Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provision a reproducible MQTT broker Rpi that can be used off-site #1110

Open
goatchurchprime opened this issue Apr 15, 2019 · 2 comments

Comments

Projects
None yet
3 participants
@goatchurchprime
Copy link
Contributor

commented Apr 15, 2019

Rather than reopen the Issue #616 we can keep progress updated here as to where we want to go.

@ajlennon is building some kind of ballina thing that can rebuild and deploy to the rPI that is hanging on the front wall with a 7" touch screen.

Some details of what it does (not how to rebuild it) are at: https://github.com/DoESLiverpool/somebody-should/wiki/MQTT-services This includes running a DoESLiverpool-MQTT wifi, which makes the thing portable.

We've decided not to host the node-red or database instances on this rPI because we don't want failures with those to bring down the core MQTT-broker service anymore. These are going to be put on some other computer or other screen.

Now, we've got to show something useful on the screen. If the whole GUI is not going to be there, then it's either got to be a constantly scrolling mosquitto_sub -t '#' dump, or we can run this little script that is uses curses to plot a constantly updated view of the different messages going through.

https://github.com/goatchurchprime/jupyter_micropython_developer_notebooks/blob/master/projects/mqttmonitortext.py
image

@ajlennon

This comment has been minimized.

Copy link
Contributor

commented Apr 27, 2019

OK so I've been making progress with this with @goatchurchprime ... Some status updates.

We have been taking down the 7in dashboard display on the wall to take to various "show and tells". Also we have been updating this and in the process we've managed to break it requiring reinstalls.

We would like to be in a position where we have the data from the space persisted in a reasonably reliable fashion so that if (when) things break or we modify the system we don't lose the data and we don't stop storing data.

As such we've broken the system out into two different pieces of hardware (for now - we might break things out further in future depending on how the system performs).

  • We have the Dashboard display which is configured via an OpenBalena.io server instance. This is running a Docker container with a desktop and Chromium web-browser running in kiosk mode. So we should be able to show anything on this dashboard that a modern web-browser can show. We've tested with YouTube videos and have had Grafana graphs of the workspace energy usage running on it.

There are a couple of niggles to be fixed - on first boot the browser comes up before network connectivity is established. We probably want to hold off bringing up the browser until this happens. Similarly if connectivity is lost when Grafana dashboards are being displayed they seem to stop updating and we need a way to retry in this event.

The Balena/Docker configuration for this is currently here.

https://github.com/DynamicDevices/does-rpi3-dash

In theory this can be deployed trivially to give us web-displays to any number of new devices (such as Raspberry Pi's) which are supported by Balena.

  • We have a headless Rpi3B+ unit running multiple Balena containers. Currently

Mosquitto - MQtt broker which Sonoff Espurna devices in the space are publishing to on mqtt.local

InfluxDB - NoSQL database we are experimenting with to persist data from Mosquitto

Node-Red - Which we are using to wire up flows to tranform and persist certain message data i.e. bridging from Mosquitto to InfluxDB

Grafana - Used for graphing and data-visualisation of the datapoints in InfluxDB

There's also an example "Bridge" piece of Python scripting to show how we can store MQtt messages to InfluxDB programatically but this is not currently in use.

There are also a couple of niggles with this. Docker containers are non-persistent so when you restart them you revert to the original state and lose any changes (i.e. any data). There is a capability to define a storage mount which is where data to be persisted goes. We have these defined for the containers and so shouldn't lose Node-Red flow definitions, Grafana dashboards or InfluxDB data. I am seeing that when I re-deploy I do seem to lose data though so need to work out if this is happening and if so why.

Also the data is currently persistet to a uSD card in the RPi which is bad practise. Eventually the uSD will fail to to limited flash write cycles. We need to move this to a middle-term solution such as storing to rotating media.

The Rpi is currently uncased on my desk. We should case it up (possibly with a 2.5in laptop HDD) and find a place to put it out of the way.

@ajlennon

This comment has been minimized.

Copy link
Contributor

commented Apr 27, 2019

TODO: I also need to update the documentation here

https://github.com/DoESLiverpool/somebody-should/wiki/MQTT-services

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.