ESP8266 and Node-RED with MQTT (Publish and Subscribe)

In this post we’re going to show you how to control ESP8266 outputs and display sensor data from the ESP8266 on Node-RED. The Node-RED software is running on a Raspberry Pi, and the communication between the ESP8266 and the Node-RED software is achieved with the MQTT communication protocol.

The following figure shows an overview of what we’re going to do in this tutorial.

First, watch the video demonstration below

Node-RED and Node-RED Dashboard

You need to have Node-RED and Node-RED Dashboard installed in your Raspberry Pi. The following blog posts are useful for getting started with Node-RED and Node-RED dashboard:

MQTT Protocol

In this tutorial we’re going to establish a communication between a Raspberry Pi running the Node-RED software and an ESP8266 using MQTT.

MQTT stands for MQ Telemetry Transport and it is a nice lightweight publish and subscribe system where you can publish and receive messages as a client. It is a simple messaging protocol, designed for constrained devices and with low-bandwidth. So, it’s the perfect solution for Internet of Things applications.

If you want to learn more about MQTT, watch the video below.

Build home automation projects with the ESP8266 Wi-Fi dev board using Arduino IDE and Lua firmware DOWNLOAD »

Build home automation projects with the ESP8266 Wi-Fi dev board using Arduino IDE and Lua firmware DOWNLOAD »

For a written version of this video and additional resources, read this blog post What is MQTT and How It Works.

Installing Mosquitto Broker

In MQTT, the broker is primarily responsible for receiving all messages, filtering the messages, decide who is interested in it and then publishing the message to all subscribed clients.

There are several brokers you can use. In this tutorial we’re going to use the Mosquitto Broker which needs to be installed on Raspberry Pi.

To install the Mosquitto broker on Raspberry Pi follow this tutorial: How to Install Mosquitto Broker on Raspberry Pi.

Testing

To see if Mosquitto broker was successfully installed, run the next command:

pi@raspberry:~ $ mosquitto -v

This returns the Mosquitto version that is currently running in your Raspberry Pi. It should be 1.4 or above.

Note: the Mosquitto command returns the Mosquitto version that is currently installed, but it also tries to initialize Mosquitto again. Since Mosquitto is already running it prompts an error message. Don’t worry Mosquitto is properly installed and running if you see a similar message.

Establishing an MQTT communication with Node-RED

In this section we’re going to establish an MQTT communication using the Node-RED nodes.

Dashboard Layout

The first step is to create the dashboard layout. In this example, we’ll have a button to control an ESP8266 output; a chart and a gauge to display temperature and humidity readings from the DHT11 sensor.

Download our Free eBooks and Resources

On the top right corner of the Node-RED window, select the Layout tab under the dashboard tab. Create a tab called Room and inside the Room tab, create two groups: Lamp and Sensor as shown in figure below.

Creating the Flow

Drag the following nodes to the flow – see figure below:

  • switch – this will control the ESP8266 output
  • mqtt output node – this will publish a message to the ESP8266 accordingly to the switch state
  • 2x mqtt input nodes – this nodes will be subscribed to the temperature and humidity topics to receive sensor data from the ESP
  • chart – will display the temperature sensor readings
  • gauge – will display the humidity sensor readings

Node-RED and the MQTT broker need to be connected. To connect the MQTT broker to Node-REd, double-click the MQTT output node. A new window pops up – as shown in figure below.

  1. Click the Add new mqtt-broker option.
  2. Type localhost in the server field
  3. All the other settings are configured properly by default.
  4. Press Add and the MQTT output node automatically connects to your broker.

Edit all the other nodes properties as shown in the following figures:

  • switch – the switch sends an on  string message when it’s on; and sends an off string message when it’s off. This node will publish on the room/lamp topic. Your ESP will then be subscribed to this topic, to receive its messages.
  • mqtt output node. This node is connected to the mosquitto broker and it will publish in the room/lamp topic.
  • mqtt input node. This node is subscribed to the room/temperature topic to receive temperature sensor data from the ESP8266. The ESP8266 will be pusblishing the temperature readings on this topic.
  • chart. The chart will display the readings received on the room/temperature topic.
  • mqtt input node. This node is subscribed to the room/humidity topic to receive humidity sensor data from the ESP8266. The ESP8266 will be pusblishing the humidity readings on this same topic.
  • gauge. The gauge will display the readings received on the room/humidity topic.

Wire your nodes as shown in the figure below.

Your Node-RED application is ready. Click the Deploy button on the top right corner.

The Node-RED application is ready. To see how your dashboard looks go to  http://your-pi-ip-address/ui.

Now, follow the next sections to prepare your ESP8266.

Preparing your Arduino IDE

We’ll program the ESP8266 using the Arduino IDE. In order to upload code to your ESP8266 using the Arduino IDE, you need to install the ESP8266 add-on (How to Install the ESP8266 Board in Arduino IDE). You’ll also need to install two additional libraries to have everything ready for your ESP8266.

Installing the PubSubClient Library

The PubSubClient library provides a client for doing simple publish/subscribe messaging with a server that supports MQTT (basically allows your ESP8266 to talk with Node-RED).

1) Click here to download the PubSubClient library. You should have a .zip folder in your Downloads folder

2) Unzip the .zip folder and you should get pubsubclient-master folder

3) Rename your folder from pubsubclient-master to pubsubclient

4) Move the pubsubclient folder to your Arduino IDE installation libraries folder

5) Then, re-open your Arduino IDE

The library comes with a number of example sketches. See File >Examples > PubSubClient within the Arduino IDE software.

Installing the DHT Sensor Library

The DHT sensor library provides an easy way of using any DHT sensor to read temperature and humidity with your ESP8266 or Arduino boards.

1) Click here to download the DHT sensor library. You should have a .zip folder in your Downloads

2) Unzip the .zip folder and you should get DHT-sensor-library-master folder

3) Rename your folder from DHT-sensor-library-master to DHT

4) Move the DHT folder to your Arduino IDE installation libraries folder

5) Then re-open your Arduino IDE

For more information about the DHT11 sensor and the ESP8266, read ESP8266 DHT11/DHT22 Temperature and Humidity Web Server with Arduino IDE.

Selecting the right board on Arduino IDE

You also need to select the right board on Arduino IDE:

1) Go to Tools and select “NodeMCU 1.0 (ESP-12E Module)”.

2) Select your ESP port number under the Tools > Port > COM4 (in my case)

Uploading code

Now, you can upload the following code to your ESP8266. This code publishes messages of the temperature and humidity from the DHT11 sensor on the room/temperature and room/humidity topics trough MQTT protocol.

The ESP is subscribed to the room/lamp topic to receive the messages published on that topic by the Node-RED application, to turn the lamp on or off.

The code is well commented on where you need to make changes. You need to edit the code with your own SSID, password and RPi IP address.

This code is also compatible with other DHT sensors – you just need to uncomment and comment the right lines of code to chose your sensor.

/*****
 
 All the resources for this project:
 https://randomnerdtutorials.com/
 
*****/

#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include "DHT.h"

// Uncomment one of the lines bellow for whatever DHT sensor type you're using!
#define DHTTYPE DHT11   // DHT 11
//#define DHTTYPE DHT21   // DHT 21 (AM2301)
//#define DHTTYPE DHT22   // DHT 22  (AM2302), AM2321

// Change the credentials below, so your ESP8266 connects to your router
const char* ssid = "REPLACE_WITH_YOUR_SSID";
const char* password = "REPLACE_WITH_YOUR_PASSWORD";

// Change the variable to your Raspberry Pi IP address, so it connects to your MQTT broker
const char* mqtt_server = "REPLACE_WITH_YOUR_RPI_IP_ADDRESS";

// Initializes the espClient. You should change the espClient name if you have multiple ESPs running in your home automation system
WiFiClient espClient;
PubSubClient client(espClient);

// DHT Sensor - GPIO 5 = D1 on ESP-12E NodeMCU board
const int DHTPin = 5;

// Lamp - LED - GPIO 4 = D2 on ESP-12E NodeMCU board
const int lamp = 4;

// Initialize DHT sensor.
DHT dht(DHTPin, DHTTYPE);

// Timers auxiliar variables
long now = millis();
long lastMeasure = 0;

// Don't change the function below. This functions connects your ESP8266 to your router
void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("WiFi connected - ESP IP address: ");
  Serial.println(WiFi.localIP());
}

// This functions is executed when some device publishes a message to a topic that your ESP8266 is subscribed to
// Change the function below to add logic to your program, so when a device publishes a message to a topic that 
// your ESP8266 is subscribed you can actually do something
void callback(String topic, byte* message, unsigned int length) {
  Serial.print("Message arrived on topic: ");
  Serial.print(topic);
  Serial.print(". Message: ");
  String messageTemp;
  
  for (int i = 0; i < length; i++) {
    Serial.print((char)message[i]);
    messageTemp += (char)message[i];
  }
  Serial.println();

  // Feel free to add more if statements to control more GPIOs with MQTT

  // If a message is received on the topic room/lamp, you check if the message is either on or off. Turns the lamp GPIO according to the message
  if(topic=="room/lamp"){
      Serial.print("Changing Room lamp to ");
      if(messageTemp == "on"){
        digitalWrite(lamp, HIGH);
        Serial.print("On");
      }
      else if(messageTemp == "off"){
        digitalWrite(lamp, LOW);
        Serial.print("Off");
      }
  }
  Serial.println();
}

// This functions reconnects your ESP8266 to your MQTT broker
// Change the function below if you want to subscribe to more topics with your ESP8266 
void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    /*
     YOU MIGHT NEED TO CHANGE THIS LINE, IF YOU'RE HAVING PROBLEMS WITH MQTT MULTIPLE CONNECTIONS
     To change the ESP device ID, you will have to give a new name to the ESP8266.
     Here's how it looks:
       if (client.connect("ESP8266Client")) {
     You can do it like this:
       if (client.connect("ESP1_Office")) {
     Then, for the other ESP:
       if (client.connect("ESP2_Garage")) {
      That should solve your MQTT multiple connections problem
    */
    if (client.connect("ESP8266Client")) {
      Serial.println("connected");  
      // Subscribe or resubscribe to a topic
      // You can subscribe to more topics (to control more LEDs in this example)
      client.subscribe("room/lamp");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

// The setup function sets your ESP GPIOs to Outputs, starts the serial communication at a baud rate of 115200
// Sets your mqtt broker and sets the callback function
// The callback function is what receives messages and actually controls the LEDs
void setup() {
  pinMode(lamp, OUTPUT);
  
  dht.begin();
  
  Serial.begin(115200);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);

}

// For this project, you don't need to change anything in the loop function. Basically it ensures that you ESP is connected to your broker
void loop() {

  if (!client.connected()) {
    reconnect();
  }
  if(!client.loop())
    client.connect("ESP8266Client");

  now = millis();
  // Publishes new temperature and humidity every 30 seconds
  if (now - lastMeasure > 30000) {
    lastMeasure = now;
    // Sensor readings may also be up to 2 seconds 'old' (its a very slow sensor)
    float h = dht.readHumidity();
    // Read temperature as Celsius (the default)
    float t = dht.readTemperature();
    // Read temperature as Fahrenheit (isFahrenheit = true)
    float f = dht.readTemperature(true);

    // Check if any reads failed and exit early (to try again).
    if (isnan(h) || isnan(t) || isnan(f)) {
      Serial.println("Failed to read from DHT sensor!");
      return;
    }

    // Computes temperature values in Celsius
    float hic = dht.computeHeatIndex(t, h, false);
    static char temperatureTemp[7];
    dtostrf(hic, 6, 2, temperatureTemp);
    
    // Uncomment to compute temperature values in Fahrenheit 
    // float hif = dht.computeHeatIndex(f, h);
    // static char temperatureTemp[7];
    // dtostrf(hic, 6, 2, temperatureTemp);
    
    static char humidityTemp[7];
    dtostrf(h, 6, 2, humidityTemp);

    // Publishes Temperature and Humidity values
    client.publish("room/temperature", temperatureTemp);
    client.publish("room/humidity", humidityTemp);
    
    Serial.print("Humidity: ");
    Serial.print(h);
    Serial.print(" %\t Temperature: ");
    Serial.print(t);
    Serial.print(" *C ");
    Serial.print(f);
    Serial.print(" *F\t Heat index: ");
    Serial.print(hic);
    Serial.println(" *C ");
    // Serial.print(hif);
    // Serial.println(" *F");
  }
} 

View raw code

After uploading the code, and with the Raspberry Pi running your Node-RED application and the Mosquitto broker, you can open the Arduino IDE serial monitor at a baud rate of 115200 and see what’s happening in real time.

This is helpful to check if the ESP has established a successful connection to your router and to the Mosquitto broker. You can also see the messages the ESP is receiving and publishing.

Building the Circuit

The following sections show you the needed parts and schematics to build the circuit for this project.

Parts required

These are the parts required to build the circuit (click the links below to find the best price at Maker Advisor):

You can use the preceding links or go directly to MakerAdvisor.com/tools to find all the parts for your projects at the best price!

Schematics

Here are the schematics for this project’s circuit.

Demonstration

Congratulations! You project is now completed.

Go to http://your-pi-ip-address/ui to control the ESP with the  Node-RED application. You can access your application in any browser on the same network that your Pi (watch the video demonstration below).

The application should look something the figure below.

Wrapping up

In this tutorial we’ve shown you the basic concepts that will allow you to turn on lights and monitor sensors on your ESP using Node-RED and the MQTT communication protocol. You can follow these basic steps to build more advanced projects.

We hope you’ve found this tutorial useful.

If you liked this project and Home Automation make sure you check our course: Build a Home Automation System for $100.


Learn how to program and build projects with the ESP32 and ESP8266 using MicroPython firmware DOWNLOAD »

Learn how to program and build projects with the ESP32 and ESP8266 using MicroPython firmware DOWNLOAD »


Enjoyed this project? Stay updated by subscribing our weekly newsletter!

23 thoughts on “ESP8266 and Node-RED with MQTT (Publish and Subscribe)”

  1. Great intro. I love it! You guys are awesome.
    One note of caution to new users though regarding using charts; I had a dashboard set up with about 6 charts and after a day or two the pi became VERY slow. I’m not sure why or if this problem was fixed in the recent node-red update but be aware that if you have a lot of charts and you pi gets very sluggish you need to remove the charts or reboot the pi.
    Bruster

    • Hi.
      You are right.
      We’ve skipped that step unintentionally.
      I updated the post with the gauge settings.
      Thanks for noticing.
      Regards
      Sara 🙂

  2. Very nice Tutorial and very well commented Sketch. Thank You! (P.S.: some Mosquitto Servers, like mine, expect Username and Password to get connected to. In that case change line 108 of the sketch to:
    if (client.connect(“ESP8266Client”,[username],[password])) {

  3. connected…yeey 🙂
    local ip
    172.20.1.78
    Attempting MQTT connection…failed, rc=-2 try again in 5 seconds
    Attempting MQTT connection…failed, rc=-2 try again in 5 seconds
    Attempting MQTT connection…failed, rc=-2 try again in 5 seconds
    Attempting MQTT connection…

    how to solve upword trouble ,itz didn’t connct to mqtt

    • Hi.
      You either need to:
      – start the mosquitto broker, or
      – you have the wrong IP address for the mosquitto broker in your Arduino code, or
      – you have multiple ESPs in your network with the same name, causing a conflict with the mosquitto broker.

    • Hi James.
      If you want to use the temperature sensor with the Rapsberry Pi, without Node-RED, you can use the DHT python library.
      Take a look at look at the library here: github.com/adafruit/Adafruit_Python_DHT
      I hope this helps.
      Regards,
      Sara 🙂

  4. Great tutorial! Thanks very much, really helped get me started! Only issue i had was with the lamp turning on and off, after a bit of investigation it turned out that the NodeRed output default of the switch node is “true” and “false” not “on” and “off”, after minor code change everything worked!

  5. Hi Rui,
    I have a general sort of question:
    – It looks to me like all projects, either using ESP8266 or Raspberry Pi rely on using a home LAN, Wifi, etc. However, what if you want to build a, say, robot to use away from Wifi out in the field what do you use then? or what if you build a project and you want to demo at your friend’s place who has a different LAN. How do you become independent of the local LAN? can you use an access point with some private IP to link all your devices to it? Thanks.

    • Good question!

      When I ask this question to myself, a couple of months ago, I ended with 2 approaches:

      1) I defined an array of ssid/passwords of some wifi networks (home, office, and so on…). The ESP8266 will try to connect to first one available. PROBLEM: you can be on a different place and so… no knew wifi to connect to…!

      2) I programmed the ESP8266 as a DHCP server with it’s own sub-net (let’s say, for example: 192.168.5.x) and the other devices will be DHCP clients of the ESP8266. PROBLEM: wired devices; if you want to connect your home desktop computer to the ESP, you’ll need to do a lot of changes…

  6. Hi maam,
    A very nice tutorial maam.
    Im having a problem with the temperature reading.
    Im not getting a reading for temperature but im getting a reading for humidity.
    Ive connected a messsage payload at the output and it returns an empty string.
    Awaiting Your reply ASAP.
    Thanking you.

    • Hi.
      That is very weird.
      Maybe the sensor is failing to get temperature readings.
      Can you try the basic example and see if the sensor is working properly: github.com/adafruit/DHT-sensor-library/blob/master/examples/DHTtester/DHTtester.ino
      Regards,
      Sara

  7. Hi.I have problem with this code when I upload this code to Node MCU .Like this .I don’t know what I need to do .
    Error: Exist status 1.Error compiling for board node MCU 1.0 🙂

    • Hi.
      Please make sure that you’ve installed all the necessary libraries.
      You also need to install this library (not mentioned in the tutorial, we need to update it): github.com/adafruit/Adafruit_Sensor
      Regards,
      Sara

Leave a Comment

Download our Free eBooks and Resources

Get instant access to our FREE eBooks, Resources, and Exclusive Electronics Projects by entering your email address below.