Raspberry Pi Publishing MQTT Messages to ESP8266

In this project you’ll create a standalone web server with a Raspberry Pi that can toggle two LEDs from an ESP8266 using MQTT protocol. You can replace those LEDs with any output (like a relay that controls a lamp).

In order to create the web server you will be using a Python microframework called Flask. Here’s the high level overview of the system:

First, watch the video demonstration

Recommended resources:

If you like home automation and you want to build a complete home automation system, I recommend downloading my home automation course.

Basic Raspberry Pi Setup

Before you continue reading this project, please make sure you have Raspbian Operating System installed in your Raspberry Pi.

You can read my Getting Started with the Raspberry Pi Guide to install Raspbian and complete the basic setup.

Run and install Mosquitto broker

The Raspberry Pi is going to interact with the ESP8266 with the MQTT protocol. Having Mosquitto broker installed, you need to have Mosquitto broker running on the background:

pi@raspberry:~ $ mosquitto -d

Python web server with Flask

We’re going to use a Python microframework called Flask to turn the Raspberry Pi into web server.

To install Flask, you’ll need to have pip installed. Run the following commands to update your Pi and install pip:

pi@raspberrypi ~ $ sudo apt-get update
pi@raspberrypi ~ $ sudo apt-get upgrade
pi@raspberrypi ~ $ sudo apt-get install python-pip python-flask

Then, you use pip to install Flask and its dependencies:

pi@raspberrypi ~ $ sudo pip install flask

Installing Python Paho-MQTT

The Paho-MQTT package provides a client class which enable applications to connect to an MQTT broker to publish messages, and to subscribe to topics and receive published messages. In this example, the Python web server is going to publish messages to the ESP8266 to turn the GPIOs on and off.

To install paho-mqtt run the following command:

pi@raspberrypi ~ $ sudo pip install paho-mqtt

Creating the Python Script

This is the core script of our application. It sets up the web server and when these buttons are pressed it publishes an MQTT message to the ESP8266.

To keep everything organized, start by creating a new folder:

pi@raspberrypi ~ $ mkdir web-server
pi@raspberrypi ~ $ cd web-server
pi@raspberrypi:~/web-server $

Create a new file called app.py.

pi@raspberrypi:~/web-server $ nano app.py

Copy and paste the following script to your Raspberry Pi

#
# Created by Rui Santos
# Complete project details: https://randomnerdtutorials.com
#

import paho.mqtt.client as mqtt
from flask import Flask, render_template, request
app = Flask(__name__)

mqttc=mqtt.Client()
mqttc.connect("localhost",1883,60)
mqttc.loop_start()

# Create a dictionary called pins to store the pin number, name, and pin state:
pins = {
   4 : {'name' : 'GPIO 4', 'board' : 'esp8266', 'topic' : 'esp8266/4', 'state' : 'False'},
   5 : {'name' : 'GPIO 5', 'board' : 'esp8266', 'topic' : 'esp8266/5', 'state' : 'False'}
   }

# Put the pin dictionary into the template data dictionary:
templateData = {
   'pins' : pins
   }

@app.route("/")
def main():
   # Pass the template data into the template main.html and return it to the user
   return render_template('main.html', **templateData)

# The function below is executed when someone requests a URL with the pin number and action in it:
@app.route("/<board>/<changePin>/<action>")

def action(board, changePin, action):
   # Convert the pin from the URL into an integer:
   changePin = int(changePin)
   # Get the device name for the pin being changed:
   devicePin = pins[changePin]['name']
   # If the action part of the URL is "on," execute the code indented below:
   if action == "1" and board == 'esp8266':
      mqttc.publish(pins[changePin]['topic'],"1")
      pins[changePin]['state'] = 'True'

   if action == "0" and board == 'esp8266':
      mqttc.publish(pins[changePin]['topic'],"0")
      pins[changePin]['state'] = 'False'

   # Along with the pin dictionary, put the message into the template data dictionary:
   templateData = {
      'pins' : pins
   }

   return render_template('main.html', **templateData)

if __name__ == "__main__":
   app.run(host='0.0.0.0', port=8181, debug=True)

View raw code

Creating the HTML File

Keeping HTML tags separated from your Python script is how you keep your project organized.

Flask uses a template engine called Jinja2 that you can use to send dynamic data from your Python script to your HTML file.

Create a new folder called templates:

pi@raspberrypi:~/web-server $ mkdir templates
pi@raspberrypi:~/web-server $ cd templates
pi@raspberrypi:~/web-server/templates $

Create a new file called main.html.

pi@raspberrypi:~/web-server/templates $ nano main.html

Copy and paste the following template to your Pi:

<!DOCTYPE html>
<head>
   <title>RPi Web Server</title>
   <!-- Latest compiled and minified CSS -->
   <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" crossorigin="anonymous">
   <!-- Optional theme -->
   <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap-theme.min.css" integrity="sha384-fLW2N01lMqjakBkx3l/M9EahuwpSfeNvV63J5ezn3uZzapT0u7EYsXMjQV+0En5r" crossorigin="anonymous">
   <!-- Latest compiled and minified JavaScript -->
   <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js" integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS" crossorigin="anonymous"></script>
   <meta name="viewport" content="width=device-width, initial-scale=1">
</head>

<body>
   <h1>RPi Web Server - ESP8266 MQTT</h1>
   {% for pin in pins %}
   <h2>{{ pins[pin].name }}
   {% if pins[pin].state == 'True' %}
      is currently <strong>on</strong></h2><div class="row"><div class="col-md-2">
      <a href="/esp8266/{{pin}}/0" class="btn btn-block btn-lg btn-default" role="button">Turn off</a></div></div>
   {% else %}
      is currently <strong>off</strong></h2><div class="row"><div class="col-md-2">
      <a href="/esp8266/{{pin}}/1" class="btn btn-block btn-lg btn-primary" role="button">Turn on</a></div></div>
   {% endif %}
   {% endfor %}
</body>
</html>

View raw code

Programming the ESP8266

For the ESP8266 to interact with the Raspberry Pi web server, you need to install PubSubClient library. This library provides a client for doing simple publish/subscribe messaging with a server that supports MQTT (basically allows your ESP8266 to talk with Python web server).

Installing the Library

1) Click here to download the PubSubClient library. You should have a .zip folder in your Downloads folder

2) Unzip the .zip folder and you should get pubsubclient-master folder

3) Rename your folder from pubsubclient-master to pubsubclient

4) Move the pubsubclient folder to your Arduino IDE installation libraries folder

5) Then, re-open your Arduino IDE

The library comes with a number of example sketches. See File > Examples > PubSubClient within the Arduino IDE software.

Uploading sketch

Finally, you can upload the full sketch to your ESP8266 (replace with your SSID, password and RPi IP address):

/*****
 
 All the resources for this project:
 https://randomnerdtutorials.com/
 
*****/

// Loading the ESP8266WiFi library and the PubSubClient library
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Change the credentials below, so your ESP8266 connects to your router
const char* ssid = "YOUR_SSID";
const char* password = "YOUR_PASSWORD";

// Change the variable to your Raspberry Pi IP address, so it connects to your MQTT broker
const char* mqtt_server = "YOUR_RPi_IP_Address";

// Initializes the espClient
WiFiClient espClient;
PubSubClient client(espClient);

// Connect an LED to each GPIO of your ESP8266
const int ledGPIO5 = 5;
const int ledGPIO4 = 4;

// Don't change the function below. This functions connects your ESP8266 to your router
void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("WiFi connected - ESP IP address: ");
  Serial.println(WiFi.localIP());
}

// This functions is executed when some device publishes a message to a topic that your ESP8266 is subscribed to
// Change the function below to add logic to your program, so when a device publishes a message to a topic that 
// your ESP8266 is subscribed you can actually do something
void callback(String topic, byte* message, unsigned int length) {
  Serial.print("Message arrived on topic: ");
  Serial.print(topic);
  Serial.print(". Message: ");
  String messageTemp;
  
  for (int i = 0; i < length; i++) {
    Serial.print((char)message[i]);
    messageTemp += (char)message[i];
  }
  Serial.println();

  // Feel free to add more if statements to control more GPIOs with MQTT

  // If a message is received on the topic home/office/esp1/gpio2, you check if the message is either 1 or 0. Turns the ESP GPIO according to the message
  if(topic=="esp8266/4"){
      Serial.print("Changing GPIO 4 to ");
      if(messageTemp == "1"){
        digitalWrite(ledGPIO4, HIGH);
        Serial.print("On");
      }
      else if(messageTemp == "0"){
        digitalWrite(ledGPIO4, LOW);
        Serial.print("Off");
      }
  }
  if(topic=="esp8266/5"){
      Serial.print("Changing GPIO 5 to ");
      if(messageTemp == "1"){
        digitalWrite(ledGPIO5, HIGH);
        Serial.print("On");
      }
      else if(messageTemp == "0"){
        digitalWrite(ledGPIO5, LOW);
        Serial.print("Off");
      }
  }
  Serial.println();
}

// This functions reconnects your ESP8266 to your MQTT broker
// Change the function below if you want to subscribe to more topics with your ESP8266 
void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
     /*
     YOU  NEED TO CHANGE THIS NEXT LINE, IF YOU'RE HAVING PROBLEMS WITH MQTT MULTIPLE CONNECTIONS
     To change the ESP device ID, you will have to give a unique name to the ESP8266.
     Here's how it looks like now:
       if (client.connect("ESP8266Client")) {
     If you want more devices connected to the MQTT broker, you can do it like this:
       if (client.connect("ESPOffice")) {
     Then, for the other ESP:
       if (client.connect("ESPGarage")) {
      That should solve your MQTT multiple connections problem

     THE SECTION IN loop() function should match your device name
    */
    if (client.connect("ESP8266Client")) {
      Serial.println("connected");  
      // Subscribe or resubscribe to a topic
      // You can subscribe to more topics (to control more LEDs in this example)
      client.subscribe("esp8266/4");
      client.subscribe("esp8266/5");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

// The setup function sets your ESP GPIOs to Outputs, starts the serial communication at a baud rate of 115200
// Sets your mqtt broker and sets the callback function
// The callback function is what receives messages and actually controls the LEDs
void setup() {
  pinMode(ledGPIO4, OUTPUT);
  pinMode(ledGPIO5, OUTPUT);
  
  Serial.begin(115200);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);
}

// For this project, you don't need to change anything in the loop function. 
// Basically it ensures that you ESP is connected to your broker
void loop() {
  if (!client.connected()) {
    reconnect();
  }
  if(!client.loop())
     /*
     YOU  NEED TO CHANGE THIS NEXT LINE, IF YOU'RE HAVING PROBLEMS WITH MQTT MULTIPLE CONNECTIONS
     To change the ESP device ID, you will have to give a unique name to the ESP8266.
     Here's how it looks like now:
       client.connect("ESP8266Client");
     If you want more devices connected to the MQTT broker, you can do it like this:
       client.connect("ESPOffice");
     Then, for the other ESP:
       client.connect("ESPGarage");
      That should solve your MQTT multiple connections problem

     THE SECTION IN recionnect() function should match your device name
    */
    client.connect("ESP8266Client");
}

View raw code

Schematics

The schematics for this project are very straightforward. Simply connect two LEDs with two resistors to your ESP8266 as shown in the figure below.

You can use the preceding links or go directly to MakerAdvisor.com/tools to find all the parts for your projects at the best price!

Launching the Web Server

To launch your Raspberry Pi web server move to the folder that contains the file app.py:

pi@raspberrypi:~/web-server/templates $ cd ..

Then, run the following command:

pi@raspberrypi:~/web-server $ sudo python app.py

Your web server should start immediately on port :8181!

python launch web server

Demonstration

Open your Raspberry Pi address in your browser by entering its IP address, in my case: http://192.168.1.98:8181

Note: you must enter your IP address followed by :8181

Here’s a video demo of the web server in action:

Wrapping up

In the next blog post, we will publish sensor readings with the ESP8266 to the Python web server.

Like home automation? Learn more about Node-RED, Raspberry Pi, ESP8266 and Arduino with my course: Build a Home Automation System for $100.

Do you have any questions? Leave a comment down below!

Thanks for reading. If you like this post probably you might like my next ones, so please support me by subscribing my blog.



Learn how to build a home automation system and we’ll cover the following main subjects: Node-RED, Node-RED Dashboard, Raspberry Pi, ESP32, ESP8266, MQTT, and InfluxDB database DOWNLOAD »
Learn how to build a home automation system and we’ll cover the following main subjects: Node-RED, Node-RED Dashboard, Raspberry Pi, ESP32, ESP8266, MQTT, and InfluxDB database DOWNLOAD »

Enjoyed this project? Stay updated by subscribing our newsletter!

18 thoughts on “Raspberry Pi Publishing MQTT Messages to ESP8266”

  1. My only doubt is, for example, I have two ESP8266 with NodeMCU firmware, which means they run Lua codes, but in this case, you write your code in other language, the one from Arduino, that it is pretty similar to C. My question: is it possible to do the same thing with my ESP? Even though they run Lua codes? I confess that I’m kinda of lost, because I have two raspberries and two ESP and I really want them to talk to each other. I followed your tutorial to make two ESp talk to each other and it worked pretty well! It seems that you use a NodeMCU board, that’s why I didn’t understand how can it run another language code. I’d appreciate if you answer me!

    Reply
  2. Nice tutorialIt’s really helpful.Hope for your next blog —-publish sensor readings to the Python web server.Thanks

    Reply
  3. Hi, I am wanting to use parts of this project for some work I am doing. We are making ‘smart’ Pedicure Chairs, where there will be 5-20+ chairs to control with a relay from a ‘main control station’ ex. the web interface. How could I do this to where I don’t have to bring the pi to their location, connect it to their network, locate the Pi’s IP address, program it into the arduino sketch, and run?
    I want to be able to program everything, ship it to them, plug it in and the script auto-runs on boot? If I set a static IP at home, and ship the Pi to them(preconfigured with their wifi) will the IP change?

    Reply
  4. Rui,

    For starters, thanks for all the work you’ve done creating your turtorials!!! I’ve been using them to learn the ESP and Arduino plateforms. However,
    I’m having issues getting this one to work. When running the python script, I get differnt output. I only two lines of output and it hangs on
    the second line, ie, doesn’t return to cmd prompt or any other output.

    python script hangs with “* Restarting with reloader”

    web-server:

    $ python -V
    Python 2.7.9

    $ ls
    app.py templates

    $ sudo python app.py
    * Running on http://0.0.0.0:8181/
    * Restarting with reloader
    [this is where it hangs]

    Reply
  5. opps, and:

    This site can’t be reached192.168.10.246 refused to connect.
    Try:

    Checking the connection
    Checking the proxy and the firewall
    ERR_CONNECTION_REFUSED

    Sounds like a priv/perm thing,

    Thank in advance

    Reply
  6. Working!!! Sorry for all the noise.

    Python3, maybe fixed it? I ran the sudo python3 app.py and reloaded the web page.
    Pi is running Jessie w/py2 and 3 pre-installed

    sudo python3 app.py
    * Running on http://0.0.0.0:8181/
    * Restarting with reloader
    192.168.10.200 – – [11/May/2020 13:59:25] “GET / HTTP/1.1” 200 –
    192.168.10.200 – – [11/May/2020 13:59:27] “GET /favicon.ico HTTP/1.1” 404 –
    192.168.10.200 – – [11/May/2020 13:59:36] “GET /esp8266/4/1 HTTP/1.1” 200 –
    192.168.10.200 – – [11/May/2020 13:59:41] “GET /esp8266/5/1 HTTP/1.1” 200 –

    Reply
  7. super tutorialIt’s really great .Hope for your next blogs —-
    1.the same project with voicecommand controlled
    2.the same one that can be controlled globally not in the same lan
    Thanks

    Reply
  8. Hi,

    can I ask you something please?

    Inside the python script, you are using:

    “”
    def action(board, changePin, action):
    # Convert the pin from the URL into an integer:
    changePin = int(changePin)
    # Get the device name for the pin being changed:
    devicePin = pins[changePin][‘name’]

    “”
    Can you tell me why you define devicePin? I guess you’re making a link to board in the next few lines, but i dont get it 🙂

    if action == “1” and board == ‘esp8266’:
    mqttc.publish(pins[changePin][‘topic’],”1″)
    pins[changePin][‘state’] = ‘True’

    thx

    Reply
  9. Hi,

    I have a problem with mqtt connection. It failed with rc=-2…

    I have the same code in both pyton and ESP8266 code.

    Can you help me with the value of rc state ?

    Thank in advance
    Thomas

    Reply
    • It was a problem to connect to rpi. I don’t know why ! If i change the port number 1883, i have this message on rpi console :
      192.168.218.149 – – [30/Nov/2021 15:01:03] code 400, message Bad HTTP/0.9 request type (‘\x10\x19\x00\x04MQTT\x04\x02\x00\x0f\x00’)
      ESP8266Client” HTTPStatus.BAD_REQUEST -03] “MQTT

      It work with this url “broker.mqtt-dashboard.com”

      Reply

Leave a Comment

Download Our Free eBooks and Resources

Get instant access to our FREE eBooks, Resources, and Exclusive Electronics Projects by entering your email address below.