Raspberry Pi Environmental Monitoring

Purpose

To monitor air temperature, humidity, and air quality.

Initially this was put together to run node-red and mysql on a single pi. This worked ok until the database reached ~200MB and the flash drive I was using started to choke. I also noticed the power draw for the pi, monitor, flash drive, and sensors exceeded the supply.

Work in progress

• Move database off pi to central home server.
• Move node-red to central server.
• Convert pi to pull from sensors using python instead of node-red.
• Send data to central server using MQTT instead of MySQL

PM Sensor

Base Script

A friend sent me a Nova PM Sensor. Found a script that is readily available.

• NOTE : This appears to run in python2. When attempting to run with python3 I received indentation errors. Should be easy to fix, but I am lazy and this is not a critical system.

sudo apt install git-core python-serial python-enum python-pip
mkdir -p /home/pi/build/air && cd /home/pi/build/air
sudo apt install git-core python-serial python-enum mariadb-server python-mysql.connector
wget -O aqi.py https://raw.githubusercontent.com/zefanja/aqi/master/python/aqi.py
echo [] > aqi.json
sed -i 's|/var/www/html/aqi.json|/home/pi/build/air/aqi.json|' aqi.py

At this point we can run the script, see output on the screen, and see the history written to the json file.

./aqi.py 
Y: 18, M: 1, D: 18, ID: 0xbc2d, CRC=OK
PM2.5:  0.0 , PM10:  0.0
PM2.5:  0.7 , PM10:  0.7
PM2.5:  0.6 , PM10:  0.6
PM2.5:  0.7 , PM10:  0.7

• Now remove the print command and instead create variables for pm2.5 and pm10

              #print("PM2.5: ", values[0], ", PM10: ", values[1])
	      pm25=values[0]
	      pm10=values[1]

Send values to MQTT

• Work in progress.

• Install python modules for mqtt

pip install paho-mqtt

• Update the aqi.py script to imnport the module, and configure as seen below.

NOTE : I am using authentication but not encryption in this example.

import paho.mqtt.client as mqtt

def mqttpub(pm1,pm2):
    broker_address="xxx.xxx.xxx.xxx"
    username="<username on broker>"
    password="<password on broker>"
    mqttc=mqtt.Client()
    mqttc.username_pw_set(username, password)
    mq = "0"
    while mq == "0":
        try:
            mqttc.connect(broker_address)
        except:
            print("Failed MQTT")
            time.sleep(5)
            mq="0"
        else:
            mq="1"
            mqttc.publish("air/livingroom/pm25",pm1)
            mqttc.publish("air/livingroom/pm10",pm2)

...
              mqttpub(pm25,pm10)

Configure for writing to database (LEGACY)

• NOTE : This is really not necessary except for historical data purposes, and running on the pi is a bit much.

However, I want the history written to a database. Configure mariadb anyway you want, if you have credentials keep them on hand.

create database air;
create user air;
grant all on air.* to 'air';
use air;
create table aqi (pm25 FLOAT, pm10 FLOAT, date DATETIME);
select * from aqi;
Empty set (0.00 sec)

insert into aqi values (2.1, 2.3, now());
Query OK, 1 row affected (0.02 sec)

select * from aqi;
+------+------+---------------------+
| pm25 | pm10 | date                |
+------+------+---------------------+
|  2.1 |  2.3 | 2019-12-20 13:32:18 |
+------+------+---------------------+
1 row in set (0.00 sec)

Now we edit the script to work with mysql
Please note that of this writing I the script searches the entire database for 0 values and deletes them. I need to go back and update to simply not write 0 values in the first place.

• Edit downloaded script with the following

import mysql.connector

mydb = mysql.connector.connect(
  host="localhost",
  user="air",
  database="air"
)
...
              mycursor.execute('INSERT INTO aqi (pm25, pm10, date) VALUES (%s, %s, %s)' % (pm25,pm10,'now()'))
              mydb.commit()
              mycursor.execute("delete from aqi where pm10 = 0 and pm25 = 0")
...
              mycursor.execute("delete from aqi where date < (now() - interval 1 year)")

Full Modified Script

#!/usr/bin/python -u
# coding=utf-8
# "DATASHEET": http://cl.ly/ekot
# https://gist.github.com/kadamski/92653913a53baf9dd1a8
from __future__ import print_function
import serial, struct, sys, time, json, subprocess,  mysql.connector

time.sleep(120)

mydb = mysql.connector.connect(
  host="localhost",
  user="air",
  database="air"
)

mycursor = mydb.cursor()

DEBUG = 0
CMD_MODE = 2
CMD_QUERY_DATA = 4
CMD_DEVICE_ID = 5
CMD_SLEEP = 6
CMD_FIRMWARE = 7
CMD_WORKING_PERIOD = 8
MODE_ACTIVE = 0
MODE_QUERY = 1
PERIOD_CONTINUOUS = 0

ser = serial.Serial()
ser.port = "/dev/ttyUSB0"
ser.baudrate = 9600

ser.open()
ser.flushInput()

byte, data = 0, ""

def dump(d, prefix=''):
    print(prefix + ' '.join(x.encode('hex') for x in d))

def construct_command(cmd, data=[]):
    assert len(data) <= 12
    data += [0,]*(12-len(data))
    checksum = (sum(data)+cmd-2)%256
    ret = "\xaa\xb4" + chr(cmd)
    ret += ''.join(chr(x) for x in data)
    ret += "\xff\xff" + chr(checksum) + "\xab"

    if DEBUG:
        dump(ret, '> ')
    return ret

def process_data(d):
    r = struct.unpack('<HHxxBB', d[2:])
    pm25 = r[0]/10.0
    pm10 = r[1]/10.0
    checksum = sum(ord(v) for v in d[2:8])%256
    return [pm25, pm10]
    #print("PM 2.5: {} μg/m^3  PM 10: {} μg/m^3 CRC={}".format(pm25, pm10, "OK" if (checksum==r[2] and r[3]==0xab) else "NOK"))

def process_version(d):
    r = struct.unpack('<BBBHBB', d[3:])
    checksum = sum(ord(v) for v in d[2:8])%256
    print("Y: {}, M: {}, D: {}, ID: {}, CRC={}".format(r[0], r[1], r[2], hex(r[3]), "OK" if (checksum==r[4] and r[5]==0xab) else "NOK"))

def read_response():
    byte = 0
    while byte != "\xaa":
        byte = ser.read(size=1)

    d = ser.read(size=9)

    if DEBUG:
        dump(d, '< ')
    return byte + d

def cmd_set_mode(mode=MODE_QUERY):
    ser.write(construct_command(CMD_MODE, [0x1, mode]))
    read_response()

def cmd_query_data():
    ser.write(construct_command(CMD_QUERY_DATA))
    d = read_response()
    values = []
    if d[1] == "\xc0":
        values = process_data(d)
    return values

def cmd_set_sleep(sleep):
    mode = 0 if sleep else 1
    ser.write(construct_command(CMD_SLEEP, [0x1, mode]))
    read_response()

def cmd_set_working_period(period):
    ser.write(construct_command(CMD_WORKING_PERIOD, [0x1, period]))
    read_response()

def cmd_firmware_ver():
    ser.write(construct_command(CMD_FIRMWARE))
    d = read_response()
    process_version(d)

def cmd_set_id(id):
    id_h = (id>>8) % 256
    id_l = id % 256
    ser.write(construct_command(CMD_DEVICE_ID, [0]*10+[id_l, id_h]))
    read_response()


if __name__ == "__main__":
    cmd_set_sleep(0)
    cmd_firmware_ver()
    cmd_set_working_period(PERIOD_CONTINUOUS)
    cmd_set_mode(MODE_QUERY);
    while True:
        cmd_set_sleep(0)
        for t in range(15):
            values = cmd_query_data();
            if values is not None and len(values) == 2:
#              print("PM2.5: ", values[0], ", PM10: ", values[1])
	      pm25=values[0]
	      pm10=values[1]
              mycursor.execute('INSERT INTO aqi (pm25, pm10, date) VALUES (%s, %s, %s)' % (pm25,pm10,'now()'))
              mydb.commit()
              mycursor.execute("delete from aqi where pm10 = 0 and pm25 = 0")
              time.sleep(2)


        print("Going to sleep for 1 min...")
        cmd_set_sleep(1)
        time.sleep(60)
        mycursor.execute("delete from aqi where date < (now() - interval 1 year)")

Configure to run at boot by adding the following to your crontab

@reboot /home/pi/build/air/air.py

Tempurature / Humidity

Using Python to pull from Sensor

Place holder

Using Node-Red to pull from Sensor

NOTE : See https://michaelwiki.geekgalaxy.com/index.php/Raspberry_Pi_Environmental_Monitoring#Graph_Display

For this one I decided to use node-red to pull the values directly. It supports raspberry pi GPIO and the DHT-11 sensor I have on hand.

• Install the node-red-contrib-dht-sensor packages (I used the web gui)

sudo apt install wiringpi

• Add inject module with as timestamp. I chose to inject every 5 seconds.
• Add the rpi-dht11 node to the flow, I gave it the following values
  • Sensor : DHT11
  • Pin numbering : WiringPi (rev.1)
  • Pin number : 7
• The dht11 or 22 node will break out to a function and change module.
• This breaks out to a function node that changes C to F

var tempc = msg.payload;
tempf = tempc * 9/5 + 32;
tempf = Math.round(tempf * 10) / 10;
msg.payload = tempf;
return msg;

• Now use a change module to change msg.payload to msg.humidity.
• Both the function and change modules connect to MQTT out modules that connect to my MQTT server.

Graph Display

The same pointed me to node-red for the display. What I really wanted was a nerdy ncurses display, but node-red should suffice until I learn ncurses.

Install Node-Red

When running the install script, make sure the pi settings are applied.

bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)
sudo systemctl enable --now nodered

Install the dashboard components

Configure PM 2.5 Flow

My flow consists of

1. An inject node with the timestamp injection using topic

   select round(avg(pm25)) from aqi where date < now() - interval 24 hour;

2. A mysql node with the database location and credentials.
3. A change node with set to

   msg.payload[0].pm25

4. A gauge node set for values between 0.1 and 30.

Configure Pi for Kiosk Mode

Please note that this is not the most secure configuration. In my case the terminal is located in my house, on a segregated wifi network and broadcast domain, and the information contained is air quality metrics so impact of compromise is low.

sudo apt-get install lightdm
sudo raspi-config

At this point go to boot options, desktop /CLI, then select the desktop autologin. Close the tool and reboot.

• Note : I ran into a login loop where I was unable to auto-login and manually logging in did not work. I found a forum mentioning to install a set of x11 packages as well as openbox. I went ahead with this as I used open box years ago and found it very light weight and useful for such tasks.

sudo apt-get install xserver-xorg-core xserver-xorg-input-all \
xserver-xorg-video-fbdev libx11-6 x11-common \
x11-utils x11-xkb-utils x11-xserver-utils xterm lightdm openbox

After running the previous the system now logs me into openbox. At this point we want openbox to start chrome in kiosk mode.

sudo apt install chromium-browser
mkdir -p ~/.config/openbox && cp /etc/xdg/openbox/* ~/.config/openbox
echo -e "xset s noblank\nxset s off\n\xset -dpms\nunclutter &\nchromium-browser --kiosk http://127.0.0.1:1880/ui" >> ~/.config/openbox/autostart
sudo reboot -h

Now you should be booted into openbox with chromium running in kiosk mode, loaded with the node red graphs.