ESP32: How to Log Data (10 Different Ways)

In this article, we’ll share ten different methods to log and save data with the ESP32. Would you like to monitor a specific sensor with the ESP32 and keep track of the data over time? Would you like to save all your data records but you don’t know how to do it or which method to use? Here we’ll show you different ways to save data permanently with the ESP32. We’ll show you methods that rely on third-party databases, services, hardware solutions, and others.

ESP32 How to Log Data 9 Different Ways

Table of Contents:

Throughout this article, we’ll cover the following methods to save data using the ESP32:

  1. MicroSD Card
  2. ESP32 Filesystem (SPIFFS or LittleFS)
  3. Firebase Realtime Database and Firebase Storage
  4. InfluxDB Time Series Database
  5. Deta Base: Free and Unlimited Database
  6. Thingspeak Channels
  7. Google Sheets (using IFTTT)
  8. Google Sheets (using Google Service Account)
  9. MySQL on a Cloud Server
  10. MySQL on a Local Server

1) ESP32 Datalogging using a MicroSD Card

ESP32 microSD card module wiring breadboard

Using a microSD card with the ESP32 is a great way to save data permanently. You can save big amounts of data in txt or in other formats—as much as the microSD card size allows you to. You can also save pictures if you’re using an ESP32-CAM.

To interface a microSD card with the ESP32, you can use a microSD card module that communicates with the board via SPI communication protocol. These modules are usually pretty cheap. You can check some microSD card modules here.

Some ESP32 development boards already come with a built-in microSD card slot, so you won’t need any extra circuitry or hardware.

We have a complete guide showing how to interface a microSD card with the ESP32 and how to handle files—read, write, create, delete, append, and much more.

If you want a specific datalogging example with sensors, we also have the following projects (you can easily modify the projects to use other sensors):

Advantages of using a microSD card with the ESP32 for datalogging:

  • Low-cost solution: microSD card modules are very cheap.
  • Huge storage capacity: microSD cards are widely available and can save huge amounts of data.
  • Easy to use: it is easy to connect and easy to program with the ESP32.
  • Doesn’t require a connection to the internet: so it’s an interesting solution in remote places.
  • It can save data in different file formats.

However, you need to take into account some of the following disadvantages:

  • Extra hardware: unless your board already comes with a built-in microSD card slot, you’ll need to get a microSD card module.
  • Data corruption: after long periods of using the microSD card, it might be corrupted and you’ll lose all your data;
  • Data is not accessible in real-time: you need to remove the microSD card from the board and insert it into your computer to access the data (unless you build a webserver that serves the microSD card files online).

2) Save Files on the ESP32 Filesystem (SPIFFS or LittleFS)

The ESP32 contains a Serial Peripheral Interface Flash File System (SPIFFS). SPIFFS is a lightweight filesystem created for microcontrollers with a flash chip (like the ESP32) that allows you to store files in the flash memory.

Here are some tutorials using the ESP32 filesystem:

SPIFFS lets you access the flash memory like you would do in a normal filesystem in your computer, but simpler and more limited. You can read, write, close, and delete files. The SPIFFS library that allows you to interact with the flash chip doesn’t support directories, so everything is saved on a flat structure.

LittleFS is a new filesystem that you can use with the ESP32 that supports directories, is faster, and has some other improvements over SPIFFS.

You can create files to save data on the ESP32 filesystem as you would do with a microSD card. The advantage is that you don’t need any extra hardware. However, it’s not appropriate for saving big amounts of data or for long-term datalogging applications and you’re limited to the size dedicated to the filesystem on the ESP2 partition.

Advantages of using SPIFFS or LittleFS with the ESP32 for datalogging:

  • Easy to use: it is compatible with the FS.h library —it’s also used with the microSD card to create and handle files.
  • No extra hardware needed: all ESP32 boards comes with a flash chip, so you can use SPIFFS or LittleFS straight away.
  • Doesn’t require internet connection.

Disadvantages:

  • Storage limit: the number and size of files you can save will depend on the flash chip memory size. Most will be in the range of 4MB, so the SPIFFS/LittleFS partition will be lower than that. Additonally, you can’t use all SPIFFS partition memory for storage, only about 75%—check the docs.
  • Not suitable for long term datalogging or very frequent writes: read and write to the filesystem might become slower as the available memory decreases.
  • Not easy to visualize your data: if you need to access all your data to process it later, you’ll need to write a sketch that gets all data saved in spiffs, or a server that provides access to the filesystem files.

3) ESP32 Save Data to the Firebase Realtime Database

Firebase logo

Firebase is Google’s mobile application development platform that provides several tools to save data:

  • Realtime Database: realtime, cloud-hosted, NoSQL database; data is stored in a JSON structure.
  • Cloud Firestore: realtime, cloud-hosted, NoSQL database; data is stored in “documents”.
  • Cloud Storage: scalable file storage to upload and download files.

After saving your data on Firebase, you can access all the data online by going to your Firebase console. Additionally, you can easily interact with their storage and database services using javascript and other programming languages, so you can create your own web application to display that data online as we do in our Firebase Web App with the ESP32 and ESP8266 eBook.

Firebase Realtime Database Datalogging

We prefer using the Realtime Database for datalogging projects, because, in our opinion, it’s easier to handle and program than Cloud Firestore. We have the following guides to learn how to get started datalogging using Firebase Realtime Database.

Firebase Cloud Storage is another service that can be used to save files. Imagine that you have several files saved on the microSD card. The ESP32 can connect once a day to the internet to backup those files on Firebase Cloud Storage. We also have used Firebase Cloud Storage to store pictures taken with the ESP32-CAM:

Advantages of using Firebase Realtime Database with the ESP32 for datalogging:

  • Access from anywhere: after submitting the data, it is available online and you can check it from anywhere in the world;
  • Store and retrieve data in real-time: the data submitted to the Firebase Realtime Database synchronizes in real-time with other devices and services (if you have several boards or a web app listening to that database, it receives the changes almost instantly);
  • Free usage for small projects: for personal projects, you can use Firebase tools for free until a predefined storage limit;
  • Flexibility: It’s great if you want to build a web app for your IoT projects and you need to save different types of data;
  • Users and authentication: Firebase services provide tools to easily handle users and authentication.

Disadvantages:

  • Might be difficult to get started: You need to set up a Firebase project with authentication methods, database rules, and other settings that might be difficult for beginners (but if you follow our Firebase eBook, you’ll get everything set up in no time);
  • Requires internet connection: the ESP32 needs to be connected to the internet to connect with Firebase services, if the ESP32 is used in an environment without internet access, it will not be able to communicate with Firebase.
  • Additional costs: Firebase has a free plan that’s usually more than enough for personal applications, but if you need more data storage, you’ll need to get a paid plan. However, the free plan was always more than enough for our projects.

4) InfluxDB Time Series Database

ESP32 ESP8266 Sensor Readings Dashboard InfluxDB

InfluxDB is one of my favorite databases for datalogging. InfluxDB is an open-source high-performance time series database (TSDB) that can store large amounts of data per second. Each data point you submit to the database is associated with a particular timestamp. So, it is ideal for IoT datalogging projects like storing data from your weather station sensors.

You can run InfluxDB in InfluxDB Cloud, or locally on your laptop or Raspberry Pi.

InfluxDB cloud is a great alternative because it hosts your data that can be accessed from anywhere. However, the free cloud plan only lets you save data until the last 30 days. Depending on your project application, that might be a good alternative. Or if you’re willing to pay for a premium plan, InfluxDB cloud might be the best solution.

To get started datalogging with InfluxDB and the ESP32, you can follow the next tutorials.

Besides being a database, InfluxDB also allows you to build dashboards with different types of graphs, charts, gauges, and more to display your data. You can do all of this using their interface. You don’t need to write any code to build the charts or dashboard.

Advantages of using InfluxDB for Datalogging with the ESP32:

  • It’s a time-series database: each reading you submit to the database will be associated with a specific timestamp. You don’t need to get the date and time on the ESP32.
  • Dashboard with customizable charts: InfluxDB user interface provides tools for building custom dashboards to visualize your data. Without any knowledge of web programming, you can build awesome dashboards with customizable charts.
  • It’s available online: if you use InfluxDB cloud storage, you can access your data from anywhere in the world by accessing your InfluxDB account.
  • Also available offline: if you want to have full control over your data, you can also install InfluxDB on your own cloud server, or locally on a Raspberry Pi.

Disadvantages:

  • Requires installation: you need to install and/or setup InfluxDB first, which might require an extra step on your project;
  • Only time-based data: all data is associated with a specific timestamp, so it might not be the best database if you want to save data that doesn’t require a timestamp like JSON data.
  • Requires internet connection: the ESP32 needs to be connected to the internet to connect with InfluxDB, unless you install it locally.
  • Retention period: the free cloud plan only lets you save data until the last 30 days.

5) Use Deta Base to Save Your Data

deta base with ESP32 UI demonstration

Deta Base is a NoSQL database. It is unlimited, free, and easy to use. Additionally, it requires minimal setup. So, it’s perfect for your hobbyist projects and prototyping. It offers a UI through which you can easily see, query, update and delete records in the database.

The biggest advantage of Deta Base over other database solutions is that it requires very little setup. Once you sign up for Deta Base, it’s ready to use.

As with Firebase, you can create your own web applications to interact with the database.

If you want to learn how to use Deta Base with the ESP32, you can follow our guide:

Advantages of using Deta Base with the ESP32:

  • It’s free.
  • It requires minimal setup, once you sign up to Deta Base, you can start using it right away to store data.
  • It’s fast and scalable.
  • You can check your data online from anywhere from your account.

Disadvantages:

  • Small community: it’s still in the early stages, so if you need help, there’s still little information online.
  • Beta version: it’s in beta version, and you might encounter unexpected bugs.
  • Requires internet connection: like the other cloud solutions, the ESP32 needs to have access to the internet.

6) Use ThingSpeak Channels

ESP32 board ThingSpeak channels

ThingSpeak is an IoT platform in which you can create channels to store data. It provides visualization tools and different widgets to display your data like charts, gauges, or numeric displays. The submitted data is also associated with a timestamp, which is useful if you want to display it on charts to see how it behaves over time.

You can have multiple devices publishing data to your Thingspeak account. Usually, each device will require a channel. The free plan is limited to four channels.

In my opinion, the visual interface is easy to use, but it allows very little customization when compared with the tools provided by InfluxDB. However, it is perfect if you just want a simple visualization without having to worry about formatting details (like series colors, charts background colors, etc.).

The data is stored in the cloud, so you can access your data anywhere from your account,

We have two different tutorials that show different methods to publish data to Thingspeak. The following shows how to make an HTTP POST request with data to Thingspeak services (you need to write the request manually on the code):

An easier way to publish data to Thingspeak with the ESP32 is to use a library. You just need to call a function and pass as an argument the data you want to send.

Advantages of using ThinsSpeak with the ESP32:

  • Suitable for multiple IoT devices: you can have multiple ESP32 boards and other IoT devices publishing data to your Thinspeak account.
  • Free to use: it provides a free plan with up to four channels. Usually, each device will require a different channel on ThingSpeak.
  • Minimal setup: you don’t need to install anything on your computer or set up any database settings; once you create an account and a new channel, it’s ready to use.
  • Simple dashboard: the data will be displayed on the widget of your choice, without having to program anything on the user interface.

Disadvantages:

  • Limited free plan: the free plan is limited to four channels, which might not be enough for your project.
  • Little customization: the number of available widgets is limited and allows very little customization.
  • Requires internet connection: like the other cloud solutions, the ESP32 needs to have access to the internet.

7) Save ESP32 Data to Google Sheets (IFTTT)

ESP32 publish data to google sheets

Another great alternative to log data with the ESP32 is to use Google Sheets. Then, you can analyze, manage, and display your data using the features provided in Google Sheets. You can have access to your data from anywhere using your Google account. This is a good method if you need to store data that needs processing later on.

We have a guide showing how to log sensor readings to Google Sheets using IFTTT services. The advantage of using IFTTT services is that you don’t need to create any scripts or additional configurations in your Google account. However, it has some limitations in the number of requests you can make with the free account and with how much information you can save at once. Here’s the tutorial:

If you don’t want to rely on a third-party service to publish data to Google Sheets with the ESP32, you can use this method that uses a Google Service Account.

Advantages of saving ESP32 data to Google Sheets:

  • Easy data management: if you need to process your data after collecting it, you can use all the functionalities of google sheets to process data.
  • Access from anywhere: you can easily access your data from your google account.
  • Shareable: you can share the google sheet file with other people by providing access to their specific email.

Disadvantages:

  • It doesn’t display the data visually automatically. You need to know how to create charts and graphs (which is not difficult, so I don’t think this is a big limitation).
  • If using a third-party service like IFTTT (free account) you’re limited to the number of requests and fields you can pass at a time.
  • Requires internet connection: like the other cloud solutions, the ESP32 needs to have access to the internet.

8) Save ESP32 Data to Google Sheets (Google Service Account)

A great method to log data to Google Sheets is using a Google Service Account and the Google Sheets API. A service account, identified by its unique email address, is a special kind of account that is typically used by an application or compute workload, like a Compute Engine instance, rather than being associated with a person.

Using a Google Service Account is one of the safest methods and is recommended by Google to interact with your Google Sheets.

Learn how to log data with the ESP32 to Google Sheets using a Google Service account and the Google Sheets API:


9) ESP32 Save Data to MySQL Database on a Cloud Server

ESP32 Save Data to MySQL on a Cloud Server

You can install a MySQL Database on a cloud server and then, make requests with the ESP32 to publish and retrieve data. This method is great for those who like programming, setting up their own servers, and want to have full control over their data storage. If you’re already familiar with MySQL and PHP, you’ll certainly like this method.

The biggest disadvantage of this method is that you need to set up everything on your own from scratch which is easily prone to errors. Additionally, you’ll need to pay for hosting and a domain name.

You can follow the next tutorials to set up a MySQL database on your own server and then, learn how to create a web page to display the data on charts:

  1. ESP32/ESP8266 Insert Data into MySQL Database using PHP and Arduino IDE
  2. Visualize Your Sensor Readings from Anywhere in the World (ESP32/ESP8266 + MySQL + PHP)

10) MySQL Database on a Local Server (Raspberry Pi)

This is an alternative to the previous solution. If you want to set up a MySQL database locally on a Raspberry Pi to save the data coming from the ESP32, you’ll need to follow the next tutorial first:

Then, you can follow the next tutorial to learn how to actually publish data with the ESP32 to your local server and how to display it on a web page:

Wrapping Up

In this tutorial, we compiled different methods that you can use to save and log data permanently using your ESP32. Depending on your expertise level and project requirements, one solution might be more suitable than the others. There are many other methods besides the ones we covered here. We created a compilation of the tutorials we already have about this subject.

We hope you found this compilation guide useful. Let us know what’s your favorite storage method and for which applications you use it.

Learn more about the ESP32 with our resources:

Thanks for reading.



Learn how to build a home automation system and we’ll cover the following main subjects: Node-RED, Node-RED Dashboard, Raspberry Pi, ESP32, ESP8266, MQTT, and InfluxDB database DOWNLOAD »
Learn how to build a home automation system and we’ll cover the following main subjects: Node-RED, Node-RED Dashboard, Raspberry Pi, ESP32, ESP8266, MQTT, and InfluxDB database DOWNLOAD »

Enjoyed this project? Stay updated by subscribing our newsletter!

21 thoughts on “ESP32: How to Log Data (10 Different Ways)”

  1. Nice article but I am surprised RRDTool was not mentioned.
    A while back I did a project to automate/monitor my greenhouse using a raspberry pi. The RRD was nice because once setup there was virtually no maintenance of the db necessary.

    Reply
    • Hi.
      Thanks for the suggestion.
      Can you share a link to your project? What are the mains advantages of that method?
      Thank you.
      Regards,
      Sara

      Reply
  2. That’s a great round-up! Another option worth considering is that if you’re already running home assistant it’s super easy to create sensors with esphome, and it will log the data directly into home assistant (no need for mqtt either if you just use the native api). Then you can use the reporting and automation tools and various graphing features built-in to HA or through add-ons. I’m using timeseriesDB and grafana on top of that and it’s pretty cool.

    Reply
    • Hi John.
      Thanks for the suggestion.
      As we mentioned in the article, there are endless possibilities to save data with the ESP32 using many different methods.
      In this tutorial, we’re only compiling the ones we already have covered on our website.
      Regards,
      Sara

      Reply
  3. I am constantly impressed by your performance sister Sara and Team
    Your creative and innovative thinking has put you in your own class. No one else can compare.

    Reply
      • Hi Sara,

        I work for the company behind Astra DB (albeit in a different product). I’ve used Astra in a small project and I’m planning to use it next coming weeks for a esp32-related personal project.

        Reply
  4. I have tried to use the SPIFFS to log small files for the advantages you mentioned. However, I never found a useful way to download the files to my laptop when they are done. It is easy to upload files to the SPIFFS but is there a way to download them? Am I missing something?

    Reply
  5. Does any mentioned methods allow a simple back controll od ESP32 based device? E.g. LED swith on/off, PWM duty cycle set… by send some information to ESP32.

    Reply

Leave a Comment

Download Our Free eBooks and Resources

Get instant access to our FREE eBooks, Resources, and Exclusive Electronics Projects by entering your email address below.