ESP32 Datalogger: Download Data File via Web Server (Arduino IDE)

In this guide, we’ll build an ESP32 datalogger that also hosts a web server so that you can access and download the data remotely. The ESP32 will log data to a file hosted on a microSD card. You can access the web server with your computer or smartphone and download the file with the data remotely without the need to remove the microSD card from the ESP32.

Table of Contents:

• Project Overview
• Prerequisites
  • Arduino IDE
  • Installing Libraries
  • Formatting the MicroSD Card
  • Parts Required
  • Circuit Diagram
• HTML File
• Datalogger and Web Server Code
• Demonstration

Project Overview

Here’s a quick overview of the features of this project.

• The ESP32 will log data and the corresponding timestamp to a file on a microSD card.

• We’ll log data from a BME280 sensor, but you can use any other sensor or sensors.

• We’ll save the timestamp in epoch time. Then, when processing the data, you can convert it to your local time.

• The ESP32 will also host a web server that you can access on your computer or smartphone as long as they are on the same network.

• The web server allows you to visualize the data saved on the file, download the data file to your computer, or delete the file from the microSD card.

• The HTML file to build the ESP32 web server will also be hosted on the microSD card.

In this example, we’ll get the time from the internet (NTP server), so the ESP32 needs to be connected to a router. If the board doesn’t have access to the internet, you can set it as an access point, and get the time from an RTC module (for example DS1307 or DS3231).

Prerequisites

Before proceeding, make sure you check the follow the next prerequisites.

Arduino IDE

We’ll program the ESP32 using Arduino IDE. So make sure you have the ESP32 boards installed. You can use Arduino IDE 2 or the legacy version.

• Installing ESP32 Board in Arduino IDE 2
• Installing the ESP32 Board in Arduino IDE (Arduino 1.8.x)

Libraries

You also need to install the following libraries.

• Adafruit BME280 Library (Arduino Library Manager)
• Adafruit Unified Sensor Driver (it will install automatically with the Adafruit BME280, otherwise, you need to install it manually)
• ESPAsyncWebServer (.zip folder)
• AsyncTCP (.zip folder)

You can install the first two libraries using the Arduino Library Manager. Go to Sketch > Include Library > Manage Libraries and search for the library name.

The ESPAsyncWebServer and AsynTCP libraries aren’t available to install through the Arduino Library Manager. You need to download the library .zip file and then, in your Arduino IDE, go to Sketch > Include Library > Add .zip Library and select the libraries you’ve just downloaded.

Formatting the MicroSD Card

Before proceeding with the tutorial, make sure you format your microSD card as FAT32. Follow the next instructions to format your microSD card or use a software tool like SD Card Formatter (compatible with Windows and Mac OS).

1. Insert the microSD card into your computer. Go to My Computer and right-click on the SD card. Select Format as shown in the figure below.

2. A new window pops up. Select FAT32, press Start to initialize the formatting process, and follow the onscreen instructions.

Parts Required

To follow this tutorial, you need the following parts:

• ESP32 development board (read: Best ESP32 development boards)
• BME280 sensor
• MicroSD Card Module
• MicroSD Card
• Jumper Wires
• Breadboard

You can use the preceding links or go directly to MakerAdvisor.com/tools to find all the parts for your projects at the best price!

Circuit Diagram

For this example, wire the microSD card module and the BME280 sensor to the ESP32. Follow the next schematic diagram or the tables below to wire the circuit.

You can also take a look at the following tables:

BME280	ESP32
VIN	3V3
GND	GND
SCL	GPIO 22
SDA	GPIO 21

microSD card module	ESP32
3V3	3.3V
CS	GPIO 5
MOSI	GPIO 23
CLK	GPIO 18
MISO	GPIO 19
GND	GND

Not familiar with the BME280 sensor? Read the following guide

• ESP32 with BME280 Sensor using Arduino IDE (Pressure, Temperature, Humidity)

Not familiar with using a microSD card with the ESP32? Check the following tutorials:

• ESP32: Guide for MicroSD Card Module using Arduino IDE
• ESP32 Web Server Hosting Files from MicroSD Card
• ESP32 Data Logging Temperature to MicroSD Card

HTML File

The following HTML file will build the web server page that we can interact with to manage the data on the ESP32.

<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>ESP32 Datalogger</title>
    <style>
        html {
            font-family: Arial, Helvetica, sans-serif;
        }

        body {
            background-color: #f4f4f4;
            margin: 0;
            padding: 0;
        }

        .container {
            max-width: 800px;
            margin: 50px auto;
            text-align: center;
        }

        h1 {
            color: #333;
        }

        .button {
            display: inline-block;
            padding: 10px 20px;
            margin: 10px;
            font-size: 16px;
            border: none;
            border-radius: 5px;
            cursor: pointer;
            transition-duration: 0.4s;    }

        .button-data {
            background-color: #858585;
            color: #fff;
        }

        .button-delete {
            background-color: #780320;
            color: #fff;
        }

        .button:hover {
            background-color: #0056b3;
        }
    </style>
</head>
<body>
    <div class="container">
        <h1>ESP32 Datalogger - Manage Data</h1>
        <a href="view-data"><button class="button button-data">View Data</button></a>
        <a href="download"><button class="button button-data">Download Data</button></a>
        <a href="delete"><button class="button button-delete">Delete Data</button></a>
    </div>
</body>
</html>

View raw code

The web page has three buttons:

• View Data: will show the data file contents — makes a request on the /view-data path;
• Download Data: downloads the data file to your device — makes a request on the /download path;
• Delete Data: deletes the data file from the microSD card — makes a request on the /delete path.

Copy the HTML File to the microSD card

1. Create a file called index.html and copy the HTML text we’ve shown you previously.
2. Copy that file to the microSD card.
3. Insert the microSD card into the microSD card module that should be connected to the ESP32.

Datalogger and Web Server Code

The following code creates an ESP32 datalogger that will log BME280 sensor data and the corresponding timestamp to a file called data.txt on the microSD card. At the same time, it also hosts a web server and shows a web page (built from the HTML file you copied to the microSD card) that you can access to manage the data.

/*********
  Rui Santos & Sara Santos - Random Nerd Tutorials
  Complete project details at https://RandomNerdTutorials.com/esp32-datalogger-download-data-file/

  Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files.
  The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*********/

#include <Arduino.h>
#include <WiFi.h>
#include <AsyncTCP.h>
#include <ESPAsyncWebServer.h>
#include "FS.h"
#include "SD.h"
#include "SPI.h"
#include <Adafruit_BME280.h>
#include <Adafruit_Sensor.h>
#include "time.h"
#include <WiFiUdp.h>

// Replace with your network credentials
const char* ssid = "REPLACE_WITH_YOUR_SSID";
const char* password = "REPLACE_WITH_YOUR_PASSWORD";

// NTP server to request epoch time
const char* ntpServer = "pool.ntp.org";

// Variable to save current epoch time
unsigned long epochTime; 

// Variables to hold sensor readings
float temp;
float hum;
float pres;
String dataMessage;

// File name where readings will be saved
const char* dataPath = "/data.txt";

// Timer variables
unsigned long lastTime = 0;
unsigned long timerDelay = 30000;

// Create AsyncWebServer object on port 80
AsyncWebServer server(80);

// BME280 connect to ESP32 I2C (GPIO 21 = SDA, GPIO 22 = SCL)
Adafruit_BME280 bme;

// Init BME280
void initBME(){
  if (!bme.begin(0x76)) {
    Serial.println("Could not find a valid BME280 sensor, check wiring!");
    while (1);
  }
}

// Init microSD card
void initSDCard(){
  if(!SD.begin()){
    Serial.println("Card Mount Failed");
    return;
  }
  uint8_t cardType = SD.cardType();

  if(cardType == CARD_NONE){
    Serial.println("No SD card attached");
    return;
  }

  Serial.print("SD Card Type: ");
  if(cardType == CARD_MMC){
    Serial.println("MMC");
  } else if(cardType == CARD_SD){
    Serial.println("SDSC");
  } else if(cardType == CARD_SDHC){
    Serial.println("SDHC");
  } else {
    Serial.println("UNKNOWN");
  }
  uint64_t cardSize = SD.cardSize() / (1024 * 1024);
  Serial.printf("SD Card Size: %lluMB\n", cardSize);
}

// Write to the SD card
void writeFile(fs::FS &fs, const char * path, const char * message) {
  Serial.printf("Writing file: %s\n", path);

  File file = fs.open(path, FILE_WRITE);
  if(!file) {
    Serial.println("Failed to open file for writing");
    return;
  }
  if(file.print(message)) {
    Serial.println("File written");
  } else {
    Serial.println("Write failed");
  }
  file.close();
}

// Append data to the SD card
void appendFile(fs::FS &fs, const char * path, const char * message) {
  Serial.printf("Appending to file: %s\n", path);

  File file = fs.open(path, FILE_APPEND);
  if(!file) {
    Serial.println("Failed to open file for appending");
    return;
  }
  if(file.print(message)) {
    Serial.println("Message appended");
  } else {
    Serial.println("Append failed");
  }
  file.close();
}

// Delete file
void deleteFile(fs::FS &fs, const char * path){
  Serial.printf("Deleting file: %s\r\n", path);
  if(fs.remove(path)){
    Serial.println("- file deleted");
  } else {
    Serial.println("- delete failed");
  }
}

// Function that gets current epoch time
unsigned long getTime() {
  time_t now;
  struct tm timeinfo;
  if (!getLocalTime(&timeinfo)) {
   //Serial.println("Failed to obtain time");
    return(0);
  }
  time(&now);
  return now;
}

// Function that initializes wi-fi
void initWiFi() {
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  Serial.print("Connecting to WiFi ..");
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print('.');
    delay(1000);
  }
  Serial.println(WiFi.localIP());
}

void setup() {
  Serial.begin(115200);
  initWiFi();
  initBME();
  initSDCard();
  configTime(0, 0, ntpServer);

  // If the data.txt file doesn't exist
  // Create a file on the SD card and write the data labels
  File file = SD.open("/data.txt");
  if(!file) {
    Serial.println("File doesn't exist");
    Serial.println("Creating file...");
    writeFile(SD, "/data.txt", "Epoch Time, Temperature, Humidity, Pressure \r\n");
  }
  else {
    Serial.println("File already exists");  
  }
  file.close();

  // Handle the root URL
  server.on("/", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(SD, "/index.html", "text/html");
  });

  // Handle the download button
  server.on("/download", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(SD, "/data.txt", String(), true);
  });

  // Handle the View Data button
  server.on("/view-data", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(SD, "/data.txt", "text/plain", false);
  });

  // Handle the delete button
  server.on("/delete", HTTP_GET, [](AsyncWebServerRequest *request){
    deleteFile(SD, dataPath);
    request->send(200, "text/plain", "data.txt was deleted.");
  });

  // Uncomment the following line if you need to serve more static files like CSS and javascript or favicon
  //server.serveStatic("/", SD, "/");

  server.begin();
}

void loop() {
  if ((millis() - lastTime) > timerDelay) {
    //Get epoch time
    epochTime = getTime();
    
    //Get sensor readings
    temp = bme.readTemperature();
    //temp = 1.8*bme.readTemperature() + 32;
    hum = bme.readHumidity();
    pres = bme.readPressure()/100.0F;

    //Concatenate all info separated by commas
    dataMessage = String(epochTime) + "," + String(temp) + "," + String(hum) + "," + String(pres)+ "\r\n";
    Serial.print("Saving data: ");
    Serial.println(dataMessage);

    //Append the data to file
    appendFile(SD, "/data.txt", dataMessage.c_str());

    lastTime = millis();
  }
}

View raw code

Before uploading the code to the board, you need to insert your network credentials on the following variables.

// Replace with your network credentials
const char* ssid = "REPLACE_WITH_YOUR_SSID";
const char* password = "REPLACE_WITH_YOUR_PASSWORD";

How the Code Works

Continue reading to learn how the code works or skip to the Demonstration section.

Including Libraries

Start by including the required libraries. We include the libraries to connect to Wi-Fi, create the web server, handle files, communicate with the microSD card, communicate with the BME280 sensor and get the time from an NTP server.

#include <Arduino.h>
#include <WiFi.h>
#include <AsyncTCP.h>
#include <ESPAsyncWebServer.h>
#include "FS.h"
#include "SD.h"
#include "SPI.h"
#include <Adafruit_BME280.h>
#include <Adafruit_Sensor.h>
#include "time.h"
#include <WiFiUdp.h>

Network Credentials

Insert your network credentials on the following variables so that the ESP32 can connect to your local network.

// Replace with your network credentials
const char* ssid = "REPLACE_WITH_YOUR_SSID";
const char* password = "REPLACE_WITH_YOUR_PASSWORD";

Initialize Variables

Then, we initialize some variables that we’ll use throughout the code.

the ntpServer saves the URL of the NTP server we’ll use to get the time.

const char* ntpServer = "pool.ntp.org";

We’ll save the timestamp in epoch time on the epochTime variable.

// Variable to save current epoch time
unsigned long epochTime; 

The following variables will hold BME280 sensor readings.

float temp;
float hum;
float pres;

The dataMessage variable will hold the concatenation of all readings separated by commas to be inserted in the data file.

String dataMessage;

The data will be saved on a file called data.txt on the root of the microSD card. The dataPath variable saves the name and path of that file.

const char* dataPath = "/data.txt";

We’ll get new data every 30 seconds (30000milliseconds). You can change the sampling period on the timerDelay variable.

// Timer variables
unsigned long lastTime = 0;
unsigned long timerDelay = 30000;

Create an AsyncWebServer instance on port 80.

// Create AsyncWebServer object on port 80
AsyncWebServer server(80);

And create an Adafruit_BME280 object called bme. It will use the ESP32 default I2C pins (GPIO 21 (SDA), and GPIO 22 (SCL))

// BME280 connect to ESP32 I2C (GPIO 21 = SDA, GPIO 22 = SCL)
Adafruit_BME280 bme;

Initialize the BME280 Sensor

The initBME() function will initialize the BME280 sensor. We’re setting the I2C address to 0x76, which is usually the address for these sensors. However, it might be different. You can check that using an I2C scanner.

// Init BME280
void initBME(){
  if (!bme.begin(0x76)) {
    Serial.println("Could not find a valid BME280 sensor, check wiring!");
    while (1);
  }
}

Initialize the MicroSD Card

The following function will initialize the communication with microSD card on the ESP32 default SPI pins. It will also print some information about the microSD card type and size.

// Init microSD card
void initSDCard(){
  if(!SD.begin()){
    Serial.println("Card Mount Failed");
    return;
  }
  uint8_t cardType = SD.cardType();

  if(cardType == CARD_NONE){
    Serial.println("No SD card attached");
    return;
  }

  Serial.print("SD Card Type: ");
  if(cardType == CARD_MMC){
    Serial.println("MMC");
  } else if(cardType == CARD_SD){
    Serial.println("SDSC");
  } else if(cardType == CARD_SDHC){
    Serial.println("SDHC");
  } else {
    Serial.println("UNKNOWN");
  }
  uint64_t cardSize = SD.cardSize() / (1024 * 1024);
  Serial.printf("SD Card Size: %lluMB\n", cardSize);
}

Write Data to a File

The following function allows us to write data to a file. To use this function you must pass as arguments the filesystem type, the file path and the message to be written.

// Write to the SD card
void writeFile(fs::FS &fs, const char * path, const char * message) {
  Serial.printf("Writing file: %s\n", path);

  File file = fs.open(path, FILE_WRITE);
  if(!file) {
    Serial.println("Failed to open file for writing");
    return;
  }
  if(file.print(message)) {
    Serial.println("File written");
  } else {
    Serial.println("Write failed");
  }
  file.close();
}

Append Data to a File

The writeFile() function overwrites any existing data on a file. To append data to a file, we have the appendFile() function. It works as the previous function, but will append data instead of overwritting.

// Append data to the SD card
void appendFile(fs::FS &fs, const char * path, const char * message) {
  Serial.printf("Appending to file: %s\n", path);

  File file = fs.open(path, FILE_APPEND);
  if(!file) {
    Serial.println("Failed to open file for appending");
    return;
  }
  if(file.print(message)) {
    Serial.println("Message appended");
  } else {
    Serial.println("Append failed");
  }
  file.close();
}

Delete a File

The deleteFile() function deletes a file from the specified filesystem and path.

// Delete file
void deleteFile(fs::FS &fs, const char * path){
  Serial.printf("Deleting file: %s\r\n", path);
  if(fs.remove(path)){
    Serial.println("- file deleted");
  } else {
    Serial.println("- delete failed");
  }
}

Get Time

The following function gets and returns the current epoch time.

// Function that gets current epoch time
unsigned long getTime() {
  time_t now;
  struct tm timeinfo;
  if (!getLocalTime(&timeinfo)) {
   //Serial.println("Failed to obtain time");
    return(0);
  }
  time(&now);
  return now;
}

Initialize Wi-Fi

The initWiFi() function will connect your ESP32 to your local network using the SSID and password you inserted at the beginning of the code.

void initWiFi() {
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  Serial.print("Connecting to WiFi ..");
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print('.');
    delay(1000);
  }
  Serial.println(WiFi.localIP());
}

setup()

In the setup(), initialize the Serial Monitor, connect the ESP32 to Wi-Fi, initialize the BME280 sensor and the microSD card and configure the time server.

Serial.begin(115200);
initWiFi();
initBME();
initSDCard();
configTime(0, 0, ntpServer);

Then, we create a new file on the microSD card called data.txt where we’ll save the data (if it doesn’t exist yet) and we’ll write the data headers to the file.

// If the data.txt file doesn't exist
// Create a file on the SD card and write the data labels
File file = SD.open("/data.txt");
if(!file) {
  Serial.println("File doesn't exist");
  Serial.println("Creating file...");
  writeFile(SD, "/data.txt", "Epoch Time, Temperature, Humidity, Pressure \r\n");
}
else {
  Serial.println("File already exists");  
}
file.close();

Handle Requests

Finally, we need to handle what happens when we click on the web page buttons. As we’ve seen previously:

• View Data: will show the data file contents — makes a request on the /view-data path;
• Download Data: downloads the data file to your device — makes a request on the /download path;
• Delete Data: deletes the data file from the microSD card — makes a request on the /delete path.

The following line will serve the index.html file saved on the microSD card to show up the web page.

server.on("/", HTTP_GET, [](AsyncWebServerRequest *request){
  request->send(SD, "/index.html", "text/html");
});

You may also like reading: ESP32 Web Server Hosting Files from MicroSD Card

When we click on the Download button, the ESP32 receives a request on the /download URL, when that happens, we handle the request as follows.

// Handle the download button
server.on("/download", HTTP_GET, [](AsyncWebServerRequest *request){
  request->send(SD, "/data.txt", String(), true);
});

To download the file, we need to pass the following arguments to the send() function: filesystem, filepath, String(), and a boolean variable that indicates true = download). For more information you can check this use case on the library documentation here.

When the ESP32 receives a request on the /view-data path, we’ll respond with the contents of the data.txt file as follows:

// Handle the View Data button
server.on("/view-data", HTTP_GET, [](AsyncWebServerRequest *request){
  request->send(SD, “/data.txt”, “text/plain”, false);
});

Finally, when the Delete button is clicked, we’ll call the deleteFile() function to detect the file from the microSD card.

server.on("/delete", HTTP_GET, [](AsyncWebServerRequest *request){
  deleteFile(SD, dataPath);
  request->send(200, "text/plain", "data.txt was deleted.");
});

At the end of the setup(), we need to initialize the server with server.begin().

server.begin();

loop()

In the loop(), we get the time and new data from the sensor every 30 seconds and append it to the data.txt file on the microSD card.

void loop() {
  if ((millis() - lastTime) > timerDelay) {
    //Get epoch time
    epochTime = getTime();
    
    //Get sensor readings
    temp = bme.readTemperature();
    //temp = 1.8*bme.readTemperature() + 32;
    hum = bme.readHumidity();
    pres = bme.readPressure()/100.0F;

    //Concatenate all info separated by commas
    dataMessage = String(epochTime) + "," + String(temp) + "," + String(hum) + "," + String(pres)+ "\r\n";
    Serial.print("Saving data: ");
    Serial.println(dataMessage);

    //Append the data to file
    appendFile(SD, "/data.txt", dataMessage.c_str());

    lastTime = millis();
  }
}

Demonstration

After inserting your network credentials in the code, you can upload it to your ESP32 board.

After uploading, open the Serial Monitor at a baud rate of 115200 and press the ESP32 onboard RST button. It will connect to Wi-Fi and print its IP address.

After that, it will create the file on the microSD card and it should start appending data to the file every 30 seconds.

Let the code run for a while until the ESP32 gathers some data. Then, you can manage the data on the microSD card that the ESP32 is connected to by accessing the web server. Open a browser on your local network and type the ESP32 IP address.

You should get the following page.

You can click on the View Data button to see the raw data on the web browser.

You can also download the file to your computer without the need to remove the microSD card from the adapter and insert it into your computer. You just need to click on the Download Data button.

Then, you can open the file on your computer and process the data as you wish.

Finally, you can also delete the file from the microSD card by clicking the Delete Data button.

And that’s basically how the project works. Feel free to adapt what you learned here to your own projects.

Wrapping Up

In this tutorial, you learned how to download a file stored on a microSD card to your computer using a web server on the ESP32. This is very useful because you don’t need physical access to the card to get the files on your computer.

We hope you found this tutorial useful. Here’s a list of related articles you may like:

• ESP32 Data Logging Temperature to MicroSD Card
• ESP32 Web Server Hosting Files from MicroSD Card
• ESP32: Guide for MicroSD Card Module using Arduino IDE
• ESP32 Web Server: Display Sensor Readings in Gauges
• ESP32/ESP8266: Firebase Data Logging Web App (Gauges, Charts, and Table)

If you want to learn more about the ESP32, check out our resources:

• Learn ESP32 with Arduino IDE (2nd Edition)
• SMART HOME with Raspberry Pi, ESP32, ESP8266
• Build Web Servers with ESP32 and ESP8266
• Firebase Web App with the ESP32 and ESP8266
• Free ESP32 Projects,Tutorials and Guides