From Stu2
Revision as of 13:54, 26 September 2019 by Stu2 (talk | contribs) (Cheat sheet)
Jump to navigation Jump to search

ESP32 Programming. The 'new' ESP32 dev boards are preprogrammed with a LUA script. To get an Arduino to load, press and hold the 'Boot' button while uploading the Arduino code. Hold the boot button until finished loading the code.



Some Planning:

  • mosquitto - can use base image. No customization really required. If moved to cloud, may want to figure out TLS and add passwords, etc.
  • node-red - customize Dockerfile to load modules. Update flows manually.
  • influxdb - need to store data in external volume
  • grafana
  • docker - create bridge, try docker compose to bundle all this stuff
  • cloud - cloud is expensive, but we could move mosquitto, influx and grafana to cloud, use flow in node red to push updates

Cheat sheet

docker ps -a    -lists all containers
docker stop [name]
docker container prune  -remove all containers in exit status
docker container rm [name]
docker images
docker rm (name) - get rid of a container
docker-compose up
docker-compose up -d -> run in background
docker-compose logs --tail
docker exec -u 0 -it [DOCKERID] bash -> get a shell inside a container

Setup the network.

docker network create wwi-net

Interesting to note that we can use the docker names on this network. e.g. the MQTT server can be mqtt.

Individual Container Build Notes

The following sections have notes about how to build each container. No need to do this with docker-compose, except for node-red


Some mosquitto help.

mkdir ~/mosquitto/config
mkdir ~/mosquitto/data
mkdir ~/mosquitto/log

Edit ~/mosquitto/config/mosquitto.conf

 persistence true
 persistence_location /mosquitto/data/
 log_dest file /mosquitto/log/mosquitto.log
 allow_anonymous true

docker run --name mqtt -d -p 1883:1883 --net wwi-net -v $(pwd)/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf -v $(pwd)/mosquitto/data:/mosquitto/data -v $(pwd)/mosquitto/log:/mosquitto/log eclipse-mosquitto

mosquitto_pub -h -m "test" -t test/switch -d

https://blog.container-solutions.com/understanding-volumes-docker - notes on volumes. The eclipse-mosquitto readme should be more explicit about how to use these volumes. 


git node-red-docker as per this: https://nodered.org/docs/getting-started/docker
Create Dockfile in ~/node-red-docker/v1.0/Dockerfile. I wish I could use Alpine (:slim) :latest is 900MB. Dockerfile:

FROM nodered/node-red-docker:latest


RUN npm install node-red-dashboard
RUN npm install node-red-contrib-influxdb

Then build image:

docker build -f v1.0/Dockerfile -t rpt-nodered:1.0 .

Run to build container:
docker run -d --net wwi-net --name rpt-nodered -p 1880:1880 rpt-nodered:1.0

Once built, start and stop as usual. (docker stop rpt-nodered, docker start rpt-nodered)

Import flows into container via node-red development dashboard. I think these are stored in the container. We'll see when I move the container.


Some influxdb help:

Create directory for data. e.g. ~/influxdb
Pull out the configuration file to the present working directory (PWD) using:

docker run --rm influxdb influxd config > influxdb.conf

Modify config file, if needed.

Create container and run, which uses ~/influxdb.conf and stores data in ~/influxdb

docker run --name rpt-influxdb --net wwi-net -d -p 8086:8086 -v $(pwd)/influxdb/influxdb:/var/lib/influxdb -v $(pwd)/influxdb.conf:/etc/influxdb/influxdb.conf:ro influxdb -config /etc/influxdb/influxdb.conf

Create measurement database:

docker exec -it rpt-influxdb influx

create database meters


Some softether help:

Create config directory ~/softether/config and log directory ~/softether/log

docker run -d --cap-add NET_ADMIN --name softether-vpn-server -p 443:443/tcp -p 992:992/tcp -p 1194:1194/udp -p 5555:5555/tcp -v $(pwd)/softether/config:/etc/vpnserver:Z -v $(pwd)/softether/logs:/var/log/vpnserver:Z amary/softether-vpn-server

Note cap-add is a linux capability for the network.

SELINUX label - the Z option indicates that the bind mount content is private and unshared.

Manage via softether server management gui.

Docker Compose

After I figured out how to create stand alone containers, I figured out docker-composer.

Do the following:

  • Copy dc/docker-compose.yml to new computer - this is the main configuration file!
  • Create .env in dc directory, add the secret stuff
  • Copy all the data directories over to the new computer. (grafana,influxdb,node-red-data,node-red-docker,mosquitto)
  • Build node-red-docker, first. See the node-red section above. This creates rpt-nodeRed image we need, which has the influxdb and dashboard modules
  • add user grafana and change userid and groupid to 472
  • fix permission issues. (probably mosquitto)
  • install npm and node-red-admin and run node-red-admin hash-pw. Put the hash in /node-red-data/settings.js - this sets up an admin password
  • cd into dc directory and run 'docker-compose up'. Look through the output for errors.
  • docker-compose up -d then fix it so this runs on startup.

I bet I can automate all that stuff above, but will figure it out later.

Docker Compose Starup

Use 'restart: always' in docker-compose.yml.

Enable the docker service

sudo systemctl enable docker
docker-compose up -d  -> run once