From Stu2
Jump to navigation Jump to search

This page provides most of the technical details used to develop the W7IY Power Meters. The meters a ESP32 and send data to the WWI IoT system using MQTT via WiFi. Code was developed using the Visual Studio IDE and platform IO. Four meters were built to measure forward and reverse power on the 2M repeater, 450 repeater, 222 repeater and APRS node.

Power Meter - Inside



Some Planning:

  • mosquitto - can use base image. No customization really required. If moved to cloud, may want to figure out TLS and add passwords, etc.
  • node-red - customize Dockerfile to load modules. Update flows manually.
  • influxdb - need to store data in external volume
  • grafana
  • docker - create bridge, try docker compose to bundle all this stuff
  • cloud - cloud is expensive, but we could move mosquitto, influx and grafana to cloud, use flow in node red to push updates


The hardware uses an ESP-32 Devkit for the main processor. The ESP-32 has onboard 12 bit AD Converters, I2C and plenty of IO. The board is powered via the USB interface or a 12VDC buck converter. The LCD is an I2C module. Bypass capacitors are placed on the I2C power lines, which stopped the display from loosing it's mind in the high RF environment.

The case was designed using OpenSCAD and printed on my Prusa MK2+ 3D printer.

ESP32 Programming. The 'new' ESP32 dev boards are preprogrammed with a LUA script. To get an Arduino to load, press and hold the 'Boot' button while uploading the Arduino code. Hold the boot button until finished loading the code.

Wire Diagram

The power meter has three components inside the case including; the ESP32 DevKit1 board, buck power converter and I2C LCD. Here is the wiring chart:

Buck Converter (Adjust to 5VDC

12VDC IN - IN+
GND      - IN-

ESP32 Board to LCD

Vin  ->  LCD VCC
D21  ->  LCD SDA
D22  ->  LCD SCL


GND  ->  RCA Connector GND
VN   ->  RCA Rev
VP   ->  RCA FWD

Cheat sheet

docker ps -a    -lists all containers
docker stop [name]
docker container prune  -remove all containers in exit status
docker container rm [name]
docker images
docker rm (name) - get rid of a container
docker-compose up
docker-compose up -d -> run in background
docker-compose logs --tail 50
docker-compose pull [service]
docker exec -u 0 -it [DOCKERID] bash -> get a shell inside a container

Setup the network.

docker network create wwi-net

Interesting to note that we can use the docker names on this network. e.g. the MQTT server can be mqtt.

Individual Container Build Notes

The following sections have notes about how to build each container. No need to do this with docker-compose, except for node-red


Some mosquitto help.

mkdir ~/mosquitto/config
mkdir ~/mosquitto/data
mkdir ~/mosquitto/log

Download sample mosquitto.conf to ~/mosquitto/config/mosquitto.conf
Edit ~/mosquitto/config/mosquitto.conf

 persistence true
 persistence_location /mosquitto/data/
 log_dest file /mosquitto/log/mosquitto.log
 allow_anonymous true

chown -R 1883:1883 ~/mosquitto

docker run --name mqtt -d -p 1883:1883 --net wwi-net -v $(pwd)/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf -v $(pwd)/mosquitto/data:/mosquitto/data -v $(pwd)/mosquitto/log:/mosquitto/log eclipse-mosquitto

sudo apt install mosquitto-clients

mosquitto_pub -h -m "test" -t test/switch -d

https://blog.container-solutions.com/understanding-volumes-docker - notes on volumes. The eclipse-mosquitto readme should be more explicit about how to use these volumes. 


git node-red-docker as per this: https://nodered.org/docs/getting-started/docker
Create Dockfile in ~/node-red-docker/v1.0/Dockerfile. I wish I could use Alpine (:slim) :latest is 900MB. Dockerfile:

FROM nodered/node-red-docker:latest


RUN npm install node-red-dashboard
RUN npm install node-red-contrib-influxdb

Then build image:

docker build -f v1.0/Dockerfile -t rpt-nodered:1.0 .

Run to build container:
docker run -d --net wwi-net --name rpt-nodered -p 1880:1880 rpt-nodered:1.0

Once built, start and stop as usual. (docker stop rpt-nodered, docker start rpt-nodered)

Import flows into container via node-red development dashboard. I think these are stored in the container. We'll see when I move the container.


Some influxdb help:

Create directory for data. e.g. ~/influxdb
Pull out the configuration file to the present working directory (PWD) using:

docker run --rm influxdb influxd config > influxdb.conf

Modify config file, if needed.

Create container and run, which uses ~/influxdb.conf and stores data in ~/influxdb

docker run --name rpt-influxdb --net wwi-net -d -p 8086:8086 -v $(pwd)/influxdb/influxdb:/var/lib/influxdb -v $(pwd)/influxdb.conf:/etc/influxdb/influxdb.conf:ro influxdb -config /etc/influxdb/influxdb.conf

Create measurement database:

docker exec -it rpt-influxdb influx

create database meters

Note: logging needs to be fixed. Only logs to the console.


Some softether help:

Create config directory ~/softether/config and log directory ~/softether/log

docker run -d --cap-add NET_ADMIN --name softether-vpn-server -p 443:443/tcp -p 992:992/tcp -p 1194:1194/udp -p 5555:5555/tcp -v $(pwd)/softether/config:/etc/vpnserver:Z -v $(pwd)/softether/logs:/var/log/vpnserver:Z amary/softether-vpn-server

Note cap-add is a linux capability for the network.

SELINUX label - the Z option indicates that the bind mount content is private and unshared.

Manage via softether server management gui.


Reset admin password

docker exec -u 0 -it rpt-grafana bash

ln -s /var/lib/grafana  /usr/share/grafana/data
ln -s /var/log/grafana /usr/share/grafana/data/logs

grafana-cli --homepath "/usr/share/grafana" admin reset-admin-password admin


ddclient -daemon=0 -noquiet -debug
    image: linuxserver/ddclient
    container_name: ddclient
    restart: always
      - PUID=1000
      - PGID=1000
      - TZ=Americal/New_York
      - ${USERDIR}/docker-ddclient:/config
    restart: unless-stopped

This image uses S6 init. The initial scripts are in /etc/services.d, but they are copied over to /var/run/s6/services. This module uses inotifywait to watch for changes in ddclient.conf on the mounted volume, which is at /config in the container. If it changes, the config is copied to /ddclient.conf and ddclient restarts.

Docker Compose

After I figured out how to create stand alone containers, I figured out docker-composer.

Do the following:

  • Copy dc/docker-compose.yml to new computer - this is the main configuration file!
  • Create .env in dc directory, add the secret stuff
  • Copy all the data directories over to the new computer. (grafana,influxdb,node-red-data,node-red-docker,mosquitto)
  • Build node-red-docker, first. See the node-red section above. This creates rpt-nodeRed image we need, which has the influxdb and dashboard modules
  • add user grafana and change userid and groupid to 472
  • fix permission issues. (probably mosquitto)
  • install npm and node-red-admin and run node-red-admin hash-pw. Put the hash in /node-red-data/settings.js - this sets up an admin password
  • cd into dc directory and run 'docker-compose up'. Look through the output for errors.
  • docker-compose up -d then fix it so this runs on startup.

I bet I can automate all that stuff above, but will figure it out later.

Docker Compose Starup

Use 'restart: always' in docker-compose.yml.

Enable the docker service

sudo systemctl enable docker
docker-compose up -d  -> run once

Manual Programming

I switched to platform IO and vscode. Works great. However, the PIO gateway was down, so I needed to figure out how to program the ESP32 manually from the results of building under PIO. Took several hours to track down the info. I found the esptool command, by running 'pio run upload -v' on the command line in the vscode terminal. I used SCP to move the firmware files from my linux workstation the repeater computer. There are four files required. I found them in the esptool command line as follows:

--chip esp32 --port "/dev/ttyUSB4" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 /home/stu/.platformio/packages/framework-arduinoespressif32/tools/sdk/bin/bootloader_dio_40m.bin 0x8000 /data/home/stu/Documents/PlatformIO/Projects/PowerMeter/.pio/build/esp32doit-devkit-v1/partitions.bin 0xe000 /home/stu/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin 0x10000 .pio/build/esp32doit-devkit-v1/firmware.bin

Once the files are located on the repeater computer, I can program the ESP32 as follows. Note that both boot loader files don't change. They are specific to the ESP32.

For PowerMeter:

esptool --chip esp32 --port "/dev/ttyUSB0" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 /home/stu/pio/bootloader_dio_40m.bin 0x8000 /home/stu/powermeter/partitions.bin 0xe000 /home/stu/pio/boot_app0.bin 0x10000 /home/stu/powermeter/firmware.bin

For PowerMeterSetup
esptool --chip esp32 --port "/dev/ttyUSB0" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 /home/stu/pio/bootloader_dio_40m.bin 0x8000 /home/stu/powermeterSetUp/partitions.bin 0xe000 /home/stu/pio/boot_app0.bin 0x10000 /home/stu/powermeterSetUp/firmware.bin

For some reason, reading the eeprom using EEPROM.read(0); doesn't work anymore. So I hard coded the model #, recompiled, scp'ed and programmed each power meter by hand. Then, avahi handed out the wrong IP address, so I hard coded the MQTT broker in the code. Hopefully, I can figure this one out someday!

Work flow goes like this:

Modify main.cpp
Compile on local computer
SCP firmware.bin and partitions.bin to remote computer
SSH to remote computer
run esptool
verify with sudo screen /dev/ttyUSB0 115200