ec2 instance

This tutorial will show you how to publish temperature and humidity sensor data via MQTT from esp32(DHT11) to Node-Red, Influx DB and Grafana on ec2 instance one step at a time, sounds interesting let’s start.

Overview

In this tutorial we will be publishing temperature and humidity sensor data from esp32(DHT11) sensor via Mosquitto MQTT with the esp32 to Node-Red, Influx DB and Grafana in ec2 instance AWS.

Goals

  • Creating EC2 instance.
  • Installing Mosquitto MQTT, Node-Red, Influx DB and Grafana on ec2 instance.
  • Sending data from DHT11 sensor via Mosquitto MQTT with esp32 to Node-Red, Influx DB, and Grafana.

Creating EC2 Instance

Step 1:

Login to your AWS account, at the left top corner click on services.

aws

Then click on ec2 instance, this will launch the dashboard of EC2.

Here is the EC2 dashboard, you will get all the information about the AWS EC2 resource running.

aws

Step 2:

On the top right corner of the EC2 dashboard, choose the AWS region in which you want to deliver the EC2 server. Here we are selecting ap-south-1 region.

Step 3:

Once your desired region has been selected, come back to EC2 dashboard.
Click on ‘Launch Instance’ button in the top right corner of the dashboard.

aws

As soon as you click ‘Launch Instance’ your instance creation wizard page will get opened.

Step 4:

You can create your name and tag by clicking on additional tags

aws

In application and OS images select ubuntu server 20.04 LTS(HVM) SSD volume type for installation of Mosquitto MQTT, Node-Red, Influx DB, and Grafana.

In instance type select t2. Micro, which is a 1VCPU and 1GB memory server offered by AWS. In the next step enter the number of instances you want to launch at a time, we are launching one instance.

aws

In the next step you will be asked to create a key pair name to login to your instance. A key pair is a set of public-private keys. Once you are done with creating your key pair name click on create key pair.

aws

Your private key gets downloaded automatically. Make sure you download the key and keep it safe and secured in a separate folder, if it is lost you cannot download it again.
*Create a new key pair.
*Give a name to your key.
*Download and save it in your secured folder.
In the network settings allow access to SSH, HTTPS and HTTP and click on launch instance.

Installation of Mosquitto MQTT Ubuntu 20.04 server:

Mosquitto is an open-source message broker that uses the message queuing telemetry transport (MQTT) protocol.
Mosquitto is lightweight and is suitable for use on all devices from low power single board computer to full servers.

PREREQUISITES:

  • An ubuntu 20.04 server.
  • A non-root user with sudo rights.

Now go back to EC2 instance and click on instance ID.

Copy the public IPV4 address. Now open putty and paste your public IPv4 address, since we are using ubuntu for installation type ubuntu@ your public IPv4 address

Now click on SSH which is on the left side under category click on Auth. Click on browse and paste your key file which you have downloaded while creating your key pair name and click open.

putty
You’ll pull the Mosquitto package from Ubuntu’s software repository by executing the following steps. 1.SSH to your server and update the package information index.
$ sudo apt update
2.Install the Mosquitto package.
$ sudo apt install -y mosquitto
3.The Mosquitto package ships with a command-line client will allow you to test the server functionalities.
$ sudo apt install -y mosquitto-clients
4. The Mosquitto package should be loaded on your server. Confirm the status of the Mosquitto service.
$ sudo systemctl status mosquitto
Ensure the package is loaded and active.
5. Run the following command
$ sudo nano /etc/mosquitto/conf.d
6. Paste the command for unauthenticated access.
listener 1883 allow_anonymous true

Now go back to EC2 dashboard and click on instance ID ->security->security groups->edit inbound rules and attach the Mosquitto port 1883.

We will configure other ports later.

Installation of Node-RED on Ubuntu 20.04:

You need to install node.js and Node-Red.
curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash - sudo apt-get install -y nodejs build-essential sudo npm install -g --unsafe-perm node-red 

Now go back to EC2 dashboard and add port 1880 in security follow the same steps as we did for Mosquitto MQTT.
Once installed you can use the Node-Red command to start Node-Red in your terminal. To stop Node-Red you can use ctrl-c. You can then access the Node-Red editor by typing your <instance public IP address>:1880 on the browser.

Installation of InfluxDB on Ubuntu 20.04:

1.In this step, you’ll install the Influx DB server by pulling the installation files from the official repo maintained at Influxdata.com

sudo curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add -
The above command should respond with an OK command. 2. Add a new Influxdb.list repository file to the list of available packages by executing the following command:
sudo echo "deb https://repos.influxdata.com/ubuntu bionic stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
3. Then, update the package information index.
$ sudo apt update
4. Next, install the Influx DB package.
sudo apt install influxdb
5. Once installed, start the Influx DB service.
sudo systemctl start influxdb
6. check the status of the Influx DB server by executing the following command:
sudo systemctl status influxdb
The influxdb package should now be active and loaded on your server.

Now go back to EC2 dashboard and add port 8086 in security. You can then access InfluxDB editor by typing your <instance public IP address>:8086 on the browser.

Installation of Grafana on Ubuntu 20.04:

Installation of Grafana on ubuntu 20.04 server. 1.Update ubuntu:
sudo apt-get update -y
2.Install necessary packages:
sudo apt-get install wget curl gnupg2 apt-transport-https software-properties-common -y
3.Add gpg key
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
4.updo repo list of ubuntu
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
5.refresh and update apt-get
sudo apt-get update -
6.Install Grafana
sudo apt-get install grafana -y
7.Check Grafana version for successful installation
grafana-server -v
8.Start Grafana service
sudo systemctl start grafana-server
Now go back to EC2 dashboard and add port 3000 in security. You can then access Influx DB editor by typing your :3000 on the browser.

Publishing sensor data from DHT11 via MQTT with ESP32 to Node-RED, InfluxDB and Grafana:

Learn how to publish temperature and humidity readings from a esp32(DHT11) via MQTT to Node-Red, Influx DB and Grafana. In this example, we’re connecting the DHT data pin to GPIO4. However, you can use any other suitable digital pin.

#include "DHT.h"
#include 
extern "C" {
  #include "freertos/FreeRTOS.h"
  #include "freertos/timers.h"
}
#include 

#define WIFI_SSID "XXXXX"
#define WIFI_PASSWORD "XXXX"

// Raspberry Pi Mosquitto MQTT Broker
#define MQTT_HOST IPAddress(XXXXXXXXX)
// For a cloud MQTT broker, type the domain name
//#define MQTT_HOST "example.com"
#define MQTT_PORT 1883

// Temperature MQTT Topics
#define MQTT_PUB_TEMP "esp32/dht/temperature"
#define MQTT_PUB_HUM  "esp32/dht/humidity"

// Digital pin connected to the DHT sensor
#define DHTPIN 4  

// Uncomment whatever DHT sensor type you're using
#define DHTTYPE DHT11   // DHT 11
//#define DHTTYPE DHT22   // DHT 22  (AM2302), AM2321
//#define DHTTYPE DHT21   // DHT 21 (AM2301)   

// Initialize DHT sensor
DHT dht(DHTPIN, DHTTYPE);

// Variables to hold sensor readings
float temp;
float hum;

AsyncMqttClient mqttClient;
TimerHandle_t mqttReconnectTimer;
TimerHandle_t wifiReconnectTimer;

unsigned long previousMillis = 0;   // Stores last time temperature was published
const long interval = 10000;        // Interval at which to publish sensor readings

void connectToWifi() {
  Serial.println("Connecting to Wi-Fi...");
  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
}

void connectToMqtt() {
  Serial.println("Connecting to MQTT...");
  mqttClient.connect();
}

void WiFiEvent(WiFiEvent_t event) {
  Serial.printf("[WiFi-event] event: %d\n", event);
  switch(event) {
    case SYSTEM_EVENT_STA_GOT_IP:
      Serial.println("WiFi connected");
      Serial.println("IP address: ");
      Serial.println(WiFi.localIP());
      connectToMqtt();
      break;
    case SYSTEM_EVENT_STA_DISCONNECTED:
      Serial.println("WiFi lost connection");
      xTimerStop(mqttReconnectTimer, 0); // ensure we don't reconnect to MQTT while reconnecting to Wi-Fi
      xTimerStart(wifiReconnectTimer, 0);
      break;
  }
}

void onMqttConnect(bool sessionPresent) {
  Serial.println("Connected to MQTT.");
  Serial.print("Session present: ");
  Serial.println(sessionPresent);
}

void onMqttDisconnect(AsyncMqttClientDisconnectReason reason) {
  Serial.println("Disconnected from MQTT.");
  if (WiFi.isConnected()) {
    xTimerStart(mqttReconnectTimer, 0);
  }
}

/*void onMqttSubscribe(uint16_t packetId, uint8_t qos) {
  Serial.println("Subscribe acknowledged.");
  Serial.print("  packetId: ");
  Serial.println(packetId);
  Serial.print("  qos: ");
  Serial.println(qos);
}
void onMqttUnsubscribe(uint16_t packetId) {
  Serial.println("Unsubscribe acknowledged.");
  Serial.print("  packetId: ");
  Serial.println(packetId);
}*/

void onMqttPublish(uint16_t packetId) {
  Serial.print("Publish acknowledged.");
  Serial.print("  packetId: ");
  Serial.println(packetId);
}

void setup() {
  Serial.begin(115200);
  Serial.println();

  dht.begin();
  
  mqttReconnectTimer = xTimerCreate("mqttTimer", pdMS_TO_TICKS(2000), pdFALSE, (void*)0, reinterpret_cast(connectToMqtt));
  wifiReconnectTimer = xTimerCreate("wifiTimer", pdMS_TO_TICKS(2000), pdFALSE, (void*)0, reinterpret_cast(connectToWifi));

  WiFi.onEvent(WiFiEvent);

  mqttClient.onConnect(onMqttConnect);
  mqttClient.onDisconnect(onMqttDisconnect);
  //mqttClient.onSubscribe(onMqttSubscribe);
  //mqttClient.onUnsubscribe(onMqttUnsubscribe);
  mqttClient.onPublish(onMqttPublish);
  mqttClient.setServer(MQTT_HOST, MQTT_PORT);
  // If your broker requires authentication (username and password), set them below
  //mqttClient.setCredentials("REPlACE_WITH_YOUR_USER", "REPLACE_WITH_YOUR_PASSWORD");
  connectToWifi();
}

void loop() {
  unsigned long currentMillis = millis();
  // Every X number of seconds (interval = 10 seconds) 
  // it publishes a new MQTT message
  if (currentMillis - previousMillis >= interval) {
    // Save the last time a new reading was published
    previousMillis = currentMillis;
    // New DHT sensor readings
    hum = dht.readHumidity();
    // Read temperature as Celsius (the default)
    temp = dht.readTemperature();
    // Read temperature as Fahrenheit (isFahrenheit = true)
    //temp = dht.readTemperature(true);

    // Check if any reads failed and exit early (to try again).
    if (isnan(temp) || isnan(hum)) {
      Serial.println(F("Failed to read from DHT sensor!"));
      return;
    }

    // Publish an MQTT message on topic esp32/dht/temperature
    uint16_t packetIdPub1 = mqttClient.publish(MQTT_PUB_TEMP, 1, true, String(temp).c_str());                            
    Serial.printf("Publishing on topic %s at QoS 1, packetId: %i", MQTT_PUB_TEMP, packetIdPub1);
    Serial.printf("Message: %.2f \n", temp);

    // Publish an MQTT message on topic esp32/dht/humidity
    uint16_t packetIdPub2 = mqttClient.publish(MQTT_PUB_HUM, 1, true, String(hum).c_str());                            
    Serial.printf("Publishing on topic %s at QoS 1, packetId %i: ", MQTT_PUB_HUM, packetIdPub2);
    Serial.printf("Message: %.2f \n", hum);
  }
}


Code source:https://randomnerdtutorials.com/esp32-mqtt-publish-dht11-dht22-arduino/
Upload the code to your ESP32. Open the serial monitor at a baud rate of 115200 and you’ll see that the ESP32 starts publishing messages on the topic we’ve defined previously.

Preparing Node-RED dashboard:

Now we need to store the data in Influx DB from Mosquitto MQTT so we are using Node Red for transmitting the data.
The ESP32 is publishing temperature readings every 10 seconds on the esp32/dht/temperature and esp32/dht/humidity topics. As an example, we’ll create a simple flow using Node-Red to subscribe to those topics and display the readings.
Click the MQTT IN node and edit its properties:

In the server field just copy the instance IP address. The localhost will be set default to 1883. Insert the topic you want to subscribe. Here we are using esp32/dht/temperature topic.
Click on the other MQTT IN node and edit its properties with the same server, but for the other topic: esp32/dht/humidity.
For editing the Influx DB output node follow the below steps.

Creating bucket in InfluxDB:

Login to your Influx DB, in the navigation menu on the left, click on bucket->create bucket and give a name for your bucket and click on create, your bucket will be successfully created.

For generating API token click on API Tokens->custom API token create a description name and click on read and write option now click on generate you will be getting API token just copy the token and click on copy to board.

Now go back to Node-Red editor now click on Influx DB output node and edit its properties.

In the organization enter the organization name which you have created in the Influx DB then enter your bucket name. click on the server editor and paste the token which you have copied from Influx DB, Create your measurement and name.

No need to make changes in the debug node, it will display the temperature and humidity reading.
Once the flow gets deployed and debugged the readings from Mosquitto MQTT will be displayed on Node Red.

In Influx DB click on the bucket you have created and select measurement and value and click on submit.

GRAFANA:

In Grafana click on configuration->Data source->Add data source->Influx DB

Changes to be made:

NAME: Give a name to your data source.
QUERY LANGUAGE: Select Flux.
HTTP URL: http://localhost:8086.

In basic auth details enter your username and password of Influx DB.
In Influx DB details:
ORGANIZATION: Organization name of Influx DB.
TOKEN: Paste the API Token from Influx DB.
DEFAULT BUCKET: Bucket name.
Now click on save and test, once clicked you will get the notification datasource is working. 1 bucket found.

Now click on dashboards->new dashboard->Add a new panel.
Select the name which you have created for datasource. Copy and paste the script editor of Influx DB to Grafana give a tile name and click apply.

MQTT is a great communication protocol to exchange small amounts of data between devices. In this tutorial you’ve learned how to publish temperature and humidity readings from esp32(DHT 11) via MQTT to Node Red, Influx DB, Grafana.

Spread the love

Leave a Reply

Your email address will not be published. Required fields are marked *