This article is an introduction to the MQTT protocol. MQTT stands for Message Queuing Telemetry Transport, a simple messaging protocol suitable for communication between IoT devices.
Updated 16 December 2021
What is MQTT?
MQTT stands for Message Queuing Telemetry Transport. MQTT is a simple messaging protocol, designed for constrained devices with low bandwidth. So, it’s the perfect solution to exchange data between multiple IoT devices.
MQTT communication works as a publish and subscribe system. Devices publish messages on a specific topic. All devices that are subscribed to that topic receive the message.
Its main applications include sending messages to control outputs, read and publish data from sensor nodes and much more.
MQTT Basic Concepts
In MQTT there are a few basic concepts that you need to understand:
MQTT – Publish/Subscribe
The first concept is the publish and subscribe system. In a publish and subscribe system, a device can publish a message on a topic, or it can be subscribed to a particular topic to receive messages
- For example Device 1 publishes on a topic.
- Device 2 is subscribed to the same topic that device 1 is publishing in.
- So, device 2 receives the message.
MQTT – Messages
Messages are the information that you want to exchange between your devices. It can be a message like a command or data like sensor readings, for example.
MQTT – Topics
Another important concept is the topics. Topics are the way you register interest for incoming messages or how you specify where you want to publish the message.
Topics are represented with strings separated by a forward slash. Each forward slash indicates a topic level. Here’s an example of how you would create a topic for a lamp in your home office:
Note: topics are case-sensitive, which makes these two topics different:
If you would like to turn on a lamp in your home office using MQTT you can imagine the following scenario:
- A device publishes “on” and “off” messages on the home/office/lamp topic.
- You have a device that controls a lamp (it can be an ESP32, ESP8266, or any other board or device). The ESP32 that controls your lamp, is subscribed to that same topic: home/office/lamp.
- So, when a new message is published on that topic, the ESP32 receives the “on” or “off” messages and turns the lamp on or off.
The device that is publishing the messages can be an ESP32, an ESP8266, or an Home Automation controller platform with MQTT support like Node-RED, Home Assistant, Domoticz, or OpenHAB, for example.
MQTT – Broker
Finally, another important concept is the broker.
The MQTT broker is responsible for receiving all messages, filtering the messages, deciding who is interested in them, and then publishing the message to all subscribed clients.
There are several brokers you can use. In home automation projects, we use the Mosquitto Broker installed on a Raspberry Pi. You can also install the Mosquitto broker on your PC (which is not as convenient as using a Raspberry Pi board, because you have to keep your computer running all the time to keep the MQTT connection between your devices).
Having the Mosquitto broker installed on a Raspberry Pi on your local network allows you to exchange data between your IoT devices that are also connected to that same network.
To install Mosquitto broker on the Raspberry Pi follow our tutorial:
You can also run Mosquitto MQTT broker in the cloud. Running the MQTT Mosquitto Broker in the cloud allows you to connect several IoT devices from anywhere using different networks as long as they have an Internet connection. Check the tutorial below:
How to Use MQTT in Home Automation and IoT Projects
MQTT is great for home automation and internet of things projects. Here’s an example of how it can be used in a Home Automation System built with low-cost development boards like a Raspberry Pi, ESP32, ESP8266, and Arduino.
- A Raspberry Pi runs the Mosquitto broker, which is essential for MQTT protocol.
- The same Raspberry Pi runs Node-RED, which is a Home Automation Platform with MQTT support—this means it can subscribe to topics to receive messages from the other IoT devices, and publish messages on specific topics to send messages to other devices.
- Node-RED also allows you to build an User Interface with buttons to control outputs and charts to display sensor readings.
- The Arduino, the ESP32 and ESP8266 can act as MQTT clients that publish and subscribe to topics.
- These boards are connected to actuators like LEDs or relays, and sensors like temperature, humidity, smoke sensors, etc..
- These boars can publish data about the sensor’s state on a specific topic, that Node-RED is also subscribed to. This way, Node-RED receives the sensor readings and can display them on the user interface.
- On the other side, Node-RED can publish data on a specific topic to control outputs when you use the buttons on the interface. The other boards are also subscribed to that topic and control the outputs accordingly.
The following image shows an example of a Node-RED UI that allows you to control one output and displays temperature and humidity readings:
Here’s a quick summary of the steps you should follow to build something as described previously:
- Set up your Raspberry Pi. Follow our Getting Started Guide with Raspberry Pi.
- Enable and Connect your Raspberry Pi with SSH.
- You need Node-RED installed on your Pi and Node-RED Dashboard.
- Install the Mosquitto broker on the Raspberry Pi.
- Add the ESP8266 or the ESP32 to this system. You can follow the next MQTT tutorials:
- ESP32 and Node-RED with MQTT – Publish and Subscribe
- ESP8266 and Node-RED with MQTT – Publish and Subscribe
If you want to learn more about these subjects, we have a dedicated course on how to create your own Home Automation System using Raspberry Pi, ESP8266, Arduino, and Node-RED. Just click the following link.
>> Enroll in Build a Home Automation System for $100 <<
Wrapping Up
MQTT is a communication protocol based on a publish and subscribe system. Devices can subscribe to a topic or publish data on a topic. Devices receive messages that are published on topics they are subscribed to.
MQTT is simple to use and it is great for Internet of Things and Home Automation projects. You can check all our MQTT-related tutorials here.
We hope you’ve found this tutorial useful and you now understand what is MQTT and how it works.
Thanks for reading. If you like this article, please support our work by subscribing to my blog.
Congratulations anda thanks!!
You’re welcome!
Thanks for reading,
Rui
Great simplified instructions, as usual. Can you add info on use of passwords in next blog.
Hi John!
Thanks for the suggestion,
Rui
This is good for many because of the visual aspect. IMHO it’s not quite technical enough, as there are a couple key points such as NOT using a leading “/” in the topic that are omitted, which many folks assume is required due to typical file structure convention, but not for MQTT, so a little embellishment in those lines would make it even better.
Hi Rick, I totally agree that I could explore in depth those conventions and other aspects. I’ve tried to keep it as simple as possible, because MQTT is still a bit confusing to a lot of people and there are other resources that go in more depth. Thanks for your feedback!
Rui
Is MQTT has maximum string for a topic? Because I’ve some problem with publishing topic.
How many characters do you have? But I don’t think that’s your issue…
Would it be possible to use an arduino instead of a raspberry pi?
No, the Arduino can’t run an MQTT broker, but with a Ethernet/WiFi shield the Arduino can be an MQTT client
Great article, Rui!
A Raspberry Pi Zero at under $20 runs an MQTT broker just fine, and it’s a great way to get started with these computers. Or, you can run MQTT on Windows if Linux isn’t your cup of tea (steves-internet-guide.com/install-mosquitto-broker/).
Great!
Thanks for sharing 😀
Great! very clear! thanks
Thanks for reading!
Great work!
Thanks for reading !
Rui
Hello RUI,
could you explain, in a future lesson, how a REST.API works? Also how to use the REST.API with a site like IFTTT.COM? This based on this chapter MQTT?
I Love all the tutors
wonderful. thank you
You’re welcome. Thanks for reading,
Rui
do the node red works only with raspberry? what about Arduino?
Yes, Node-RED requires an Operating System to run.
So you need a full computer to install Node-RED like the Raspberry Pi
Thank you for this explanation
How i get pubsubclient.h and add it to andruino IDE?
Hi Andre.
You can find the pubsubclient in the following link: github.com/knolleary/pubsubclient
If you want to use MQTT with the ESP32, you can take a look at the following tutorial:
https://randomnerdtutorials.com/esp32-mqtt-publish-subscribe-arduino-ide/
I hope this helps.
Regards,
Sara 🙂
Hi. I have an mqtt server at home and one at work but I find it difficult to keep them in sync. Is there a service out “there” that provides free mqtt broker service? Thanks.
Hi,
nice project, but my dht22 doesn’t work ! So I switched to Webserver with same Pin D1 connected to DHT22 and this sample works fine but not reliable ! Pullup on D1pin 4,7kOhm, could I do anything to get a more reliable solution for this topic .
Mosquitto and node red works fine !!
Thanks Martin
Thanks for letting me know Martin! We also have a dedicated DHT Troubleshooting Guide that you might find helpful: https://randomnerdtutorials.com/solved-dht11-dht22-failed-to-read-from-dht-sensor/
Best description of MQTT
Thank you very much
Hi Rui, great explanation of this protocol. I have a question, what are the advantages for preferring mqtt instead of a classical http+mysql database (that are quite lightweight for sending data from few sensors). Actually using mqtt I can’t store the data, but I may at most download after the visualization. So is it more suitable for multiple iot devices? What’s your opinion?
Thanks
Paolo
I think I’m beginning to get this understood. Please correct me if I’m wrong.
MQTT must be run on something (R. Pi is a good system) will be the intermediary that passes requests and actions to the clients.
Node Red isn’t required unless you want to add an interface that you can interact with or that displays data.
Have I got this right? I was wondering about the difference between Node Red and MQTT.
Thank you
Bill
Hi Bill.
That’s right. To use MQTT, you need an MQTT broker—which is responsible for getting the messages and sending them to the right subscribers. We like to install the MQTT broker on a Raspberry Pi for MQTT on the local network. You can also install MQTT on your computer. Alternatively, you can also install it on the cloud.
Node-RED is a Home Automation platform that happens to have support for MQTT (send messages and receive messages from the broker). Node-RED is not essential. You can use it if you want to have an interface, but there are many other different ways to create an interface.
I hope this is clear now.
Regards,
Sara
One more question: Can the broker and clients all work if none are connected to the internet? I noticed you mentioned “local network” and I’m wondering if this separate system of broker and clients can all work together and without any internet connection.
Thanks for the education!
Bill
Hi.
Regarding your previous comments, all instructions were tested before publishing and they were working (two days ago). So, I think everything should be fine now.
All devices need to be connected to the internet.
Regards,
Sara
I have a few questions
– Is there any limit to add number of clients in MQTT?
– MQTT data is passing in which format?
Btw nice explanation…
regards,
Abhi
Thanks a lot.This explanation really helped me to work with MQTT.
Great!
Thank you for this informative article on MQTT! Your explanations are very clear.
From the 16th February all my MQTT devices failed to connect . Any idea of reason ?
All previously worked for months if not years.
ESP32 to node Red
Hi.
Check your network connection and if your MQTT broker is running. You might need to restart your Raspberry Pi.
Regards,
Sara
Discovered my mqtt server address had changed. Will have to re-program all the esp 32’s.
I this a common thing ?
Yes. You’ll have to do that to point the ESP32 to the right MQTT server.
For future projects, you may want to add OTA updates so that you can update the sketches on your boards wirelessly.
Regards,
Sara
Hi Sara,
Could you please explain briefly what are the requirements for OTA updates? Im currently using Arduino IoT cloud with portenta H7 which uses MQTT for wireless connection, I want to understand how the handshake between those work.
Discovered that the address of the MQTT server had changed. I will have to change the programs for a lot of esp 32’s.
Is this a common thing ?
Hi Sara,
Issue resolved. There is a confusion in where to put/locate the edited version of “mosquitto.conf”. The file that is in “/etc/mosquitto/” folder, after install has the following entry and when I put my final version of “mosquitto.conf”
in “/etc/mosquitto/conf.d/” it conflict with the file that comes with the install in the “/etc/mosquitto” directory. I moved the file to that folder and it worked OK and came up active.::
Place your local configuration in /etc/mosquitto/conf.d/
#
A full description of the configuration file is at
/usr/share/doc/mosquitto/examples/mosquitto.conf.example
pid_file /run/mosquitto/mosquitto.pid
persistence true
persistence_location /var/lib/mosquitto/
log_dest file /var/log/mosquitto/mosquitto.log
include_dir /etc/mosquitto/conf.d
=========================
● mosquitto.service – Mosquitto MQTT Broker
Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2023-03-22 21:28:54 EDT; 3min 29s ago
Thank you.
Givi
Great!
Thanks for the follow-up.
Regards,
Sara
I notice that you say the MQTT and clients must be connected to the internet. Is it not enough that they connect through their own WiFi? One of the ESP32 clients or the broker acting as a server?
I have a need which MQTT may or may not be the best solution. I currently use a USB cable to connect a PC to a ESP32 in order to send 1800 2 byte integer values in sequence from PC to ESP32 and back again for a validity check. I want to replace the USB cable with some wireless system IR, BlueTooth or WiFi, I am leaning toward WiFi. As the data is generated in a PC it seems to make sense to install the broker in the PC also.
So I have three questions:
1. Is MQTT a good choice for this task?
2. If so, can the broker also be a client? I am thinking that maybe from my PC software I can send the data directly to the broker for onward connection to the remote ESP32.
3. If not, then what would you recommend as a better alternative?