Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Provision a reproducible MQTT broker Rpi that can be used off-site #1110
Rather than reopen the Issue #616 we can keep progress updated here as to where we want to go.
@ajlennon is building some kind of ballina thing that can rebuild and deploy to the rPI that is hanging on the front wall with a 7" touch screen.
Some details of what it does (not how to rebuild it) are at: https://github.com/DoESLiverpool/somebody-should/wiki/MQTT-services This includes running a DoESLiverpool-MQTT wifi, which makes the thing portable.
We've decided not to host the node-red or database instances on this rPI because we don't want failures with those to bring down the core MQTT-broker service anymore. These are going to be put on some other computer or other screen.
Now, we've got to show something useful on the screen. If the whole GUI is not going to be there, then it's either got to be a constantly scrolling mosquitto_sub -t '#' dump, or we can run this little script that is uses curses to plot a constantly updated view of the different messages going through.
OK so I've been making progress with this with @goatchurchprime ... Some status updates.
We have been taking down the 7in dashboard display on the wall to take to various "show and tells". Also we have been updating this and in the process we've managed to break it requiring reinstalls.
We would like to be in a position where we have the data from the space persisted in a reasonably reliable fashion so that if (when) things break or we modify the system we don't lose the data and we don't stop storing data.
As such we've broken the system out into two different pieces of hardware (for now - we might break things out further in future depending on how the system performs).
There are a couple of niggles to be fixed - on first boot the browser comes up before network connectivity is established. We probably want to hold off bringing up the browser until this happens. Similarly if connectivity is lost when Grafana dashboards are being displayed they seem to stop updating and we need a way to retry in this event.
The Balena/Docker configuration for this is currently here.
In theory this can be deployed trivially to give us web-displays to any number of new devices (such as Raspberry Pi's) which are supported by Balena.
Mosquitto - MQtt broker which Sonoff Espurna devices in the space are publishing to on mqtt.local
InfluxDB - NoSQL database we are experimenting with to persist data from Mosquitto
Node-Red - Which we are using to wire up flows to tranform and persist certain message data i.e. bridging from Mosquitto to InfluxDB
Grafana - Used for graphing and data-visualisation of the datapoints in InfluxDB
There's also an example "Bridge" piece of Python scripting to show how we can store MQtt messages to InfluxDB programatically but this is not currently in use.
There are also a couple of niggles with this. Docker containers are non-persistent so when you restart them you revert to the original state and lose any changes (i.e. any data). There is a capability to define a storage mount which is where data to be persisted goes. We have these defined for the containers and so shouldn't lose Node-Red flow definitions, Grafana dashboards or InfluxDB data. I am seeing that when I re-deploy I do seem to lose data though so need to work out if this is happening and if so why.
Also the data is currently persistet to a uSD card in the RPi which is bad practise. Eventually the uSD will fail to to limited flash write cycles. We need to move this to a middle-term solution such as storing to rotating media.
The Rpi is currently uncased on my desk. We should case it up (possibly with a 2.5in laptop HDD) and find a place to put it out of the way.
TODO: I also need to update the documentation here