Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Create a general-purpose hackable DoESLiverpool MQTT server #616
This is now a dominant protocol, so maybe it should be a component of our infrastructure, and to head off the Not Invented Here syndrome.
It should probably be on a spare RPi and documented on the wiki for anyone to access, or be on Doorbot if it is a robust enough system. It needs to have a fixed IPnumber.
Adrian has flashed the most recent version of espurna-1.10.1-itead-s20.bin onto a small S20 unit through a soldered serial port (with extra power from a 3V3 regulator).
Accessing using "picocom -b115200 /dev/ttyUSB9" allows me to see debug messages and find its IP number.
In the following example commands, the S20's IPnumber is 192.168.0.105 and my computer's IPnumber is 192.168.0.105
I've connected to its dashboard using:
This lets me set the MQTT broker IPaddress and port.
For debugability you can run your own MQTT broker like so:
In another command window you see all the messages (basically button processes) from the device with:
If you want to control the device it's very useful to have spotted in the verbose messages that:
Which means you can change the relay or the led with a command like so:
Now (with the device connecting itself to the DoESLiverpool wifi) we are ready to set the MQTT broker (in the absence of our own one) to: test.mosquitto.org port: 1883
It is now possible to be running the following (listening) subscriber when it resets and reconnects:
and see that one of the messages is (very usefully):
This is going to help when you want to go onto its dashboard and change things or reflash the code.
(At times the S20 resets itself repeatedly (seen in the serial port) it eventually makes its own access point you get to through 192.168.4.1 with the above wifi password.)
When connecting to the default public MQTT broker you can change things using:
There's some further workthroughs with:
There's a new sonoff POW unit that measures current*voltage power using an hlw8012 chip whose output interface seems to be a pulse-width frequency proportional to the power. If we get one this has been decoded in the espurna code:
Meanwhile I am still battling with the STPM11 development board that performs the same purpose, but via a dodgy implementation of the SPI protocol.
Given I had one of the old doorbot Raspberry Pi 1 boards to hand, I've set up a basic install of mosquitto on it (all scripted in Ansible, see MQTT-in-a-Box for details).
It will need securing - it accepts anonymous connections at the moment and doesn't run over TLS - but I figured I'd see how much it gets used first. It's available as mqtt.local on the default port of 1883.
Instructions for commissioning:
Look up your hardware here:
Look up your firmware here:
Flashing instructions are here:
Configuring is at:
Add your network to the Wifi list on the webpage, then you can reconnect to http://espurna_a7a528.local/# to get to your device (don't forget it's protected by a login password)
The server now has its own repo in the DoES github account - https://github.com/DoESLiverpool/DoES-IoT-server and that's just had an additional role added to install and run a NodeRED server. Ideally we'd move the generic Mosquitto stuff into its own Ansible role too, but I've not done that yet. Similarly I haven't added any login details on the NodeRED instance, @ajlennon, do you fancy adding that?
OK so NodeRed is up and running on @amcewen 's MQtt-in-a-box (was there by default!)
I've got a noddy flow running to debug output from the ESPURNA I have here.
@goatchurchprime we might want to define a topic space so we can have a root that is appropriate for DOeS devices - a well designed topic space makes all the diffference for Pub/Sub - e.g. if I want to filter on all SonOffs, all SonOffs of a particular type etc. etc.
As with smart watches people seem incapable of doing decent product design work for digital home switches.
Nothing really leaps out here
There is an interesting comment about an IFTTT recipe to control heating based on weather though @goatchurchprime as you were discussing.
@amcewen made a good point earlier that it would be nice to have a dimmer switch kind of arrangement for the physical input. Which could then publish via MQtt or LoRA or whatever I guess
So I got things going quite nicely. I have a node-red flow (on my server as it has to be public facing for Amazon)
I have a new "DoES Liverpool" skill to play with, which calls the NodeRed flow. Currently that's just working out what parameters were sent for the "intention" - i.e. what I told it - and is returning to text to be spoken back to me. Obviously we can next start to wire in some actions... I'm thinking current consumption and costs and stuff would be good. And when the coffee is ready of course.
So I used Nathan Chantrell's blog post to get through it which was really useful -
I had trouble with certs and put some notes in a comment which is presumably still undergoing moderation. Basically AWS doesn't seem to like the LetsEncrypt CA root.
The Xmas tree is using 29W.
And it's using the mqtt.local (although the IP number had to be hardcoded into the config to work).
I've made my proposal for the device to interpret the measurements into events so we can derive complex behavior without any unreliable processing:
I'm wondering if one could also add more sophisticated mqtt subscribers directly into the espurna (like if-this-then-that) so that they automatically chain. Goodnight lamp without any node-red stuff.
One could chain a whole series of electric kettles so that when one finishes boiling the next one starts boiling, so there is always one forever on the boil.
I have Amazon "skills" connected to Node-Red flows which enables me to be pretty flexible in what can be done.
However for this to work there has to be a public-facing https:// endpoint for Amazon to connect to.
We do have an internal mqtt.local box on the DoES network which @amcewen kindly set up.
I think it would be very useful to be able to work with such a box on the internal network from a control and monitoring standpoint and to connect to that from Amazon Alexa services
I am wondering if it is possible (sensible/secure) to allow a proxied https:// call from Amazon servers through to the internal box?
Or if there's another option?
Or it if is a no go... :)
So I best have a go at recapping for my own recollection if nothing else.
I made the mistake of trying to install Jasper from scratch which is my general preferred way to understand what's going on with code frameworks. Needless to say that proved to be a problem. I got Raspbian and a multitude of dependencies from the above link on there ok, including Jasper.
However the speech to text engines defeated me. On some general principles of privacy and internet independence I went with the only available offline option that doesn't need tuning. This proved complicated as the compilation of open-xxx kept locking up the RPi, I think due to lack of memory but even so it's a pretty painful failure mode. (instructions say compile as root, tried compiling as user, tried adding swap space. no joy)
After much ado I then stepped back to the prepared "this will work" image download and this promptly failed to boot on the v3 (nothing - no bootloader. I assume there's a v2 image and it doesn't work on v3?)
I moved onto some work by Matt Curry here
Downloads were crawling along so I have downloaded at home and can report it is at least running and on the WiFi.
I now need to work out whether his "All In One" image supports the offline STT and TTS we want and find a way to do a simple test....
I've got a demo working in the office from the cardboard box Rpi where "oscilloscope on" will turn the Sonoff Pow on, which is now connected to the fan.
It's a set of function calls in the src/somthing.py file (in the editor) that enable these commands.
Obviously new commands ought to be programmable in... also through a voice interface.
Next task is to put the sonoff pow in line with the coffee machine and find out how to record the last spike of energy (probably on the sonoff itself if it was programmable, because it's a pain doing it in some nodered thing)... so that it can answer "How old is the coffee?"
Jackie is looking at what handsfree headset is going to work on the PiZero (which the google thing should still work with), which gives some degree of microphone privacy, because it's only talking when you are wearing the device.
@johnmckerrell Where's the issue/wiki page about the static IP address technology your friend was doing on a Rpi in the cabinet last night?
It was an idea to put the MQTT server (as mqtt.local) onto that same Rpi to save on unnecessary Rpi deployment. .
referenced this issue
Apr 24, 2018
I have login details for the box. What is required? Just `apt-get install mqtt` ? As a crucial piece of infrastructure generally I’d like to limit the number of people who have access to the box.…
On 14 May 2018, at 16:57, Julian Todd ***@***.***> wrote: @skos-ninja <https://github.com/skos-ninja> have you left any instructions of how to log on to that loose Rpi so I can do it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#616 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABqawkV5brtxpbFZyBQJa2gKEGOPYIGks5tyalWgaJpZM4RAzOy>.
I'm summarizing the state of play at: https://github.com/DoESLiverpool/wiki/wiki/MQTT-services
This was referenced
May 18, 2018
To note here this is now not working again after it killed the SD card on the UniFi Pi. As such I would not recommend putting anything else on that Pi unless it is to do directly with networking.
This has now stopped progress on #598 as we can not setup the new switch properly as it can not be adopted by the UniFi software due to the SD card being dead.
I would recommend to @johnmckerrell that we find the old raspberry pi that was set up for this and install that again and not to reinstall it on the networking Pi.
Do you have an external server you can access? You could use SSH port forwarding to enable this as an interim measure. So from your build server you would do: ssh myserver.example.com -L 22:localhost:2225 Then you can ssh into myserver.example.com, and from there SSH into your build box (something along those lines, can’t remember the order of arguments there).…
On 1 Jun 2018, at 15:35, Alex Lennon ***@***.***> wrote: OK - I was accessing my Linux build box back in the Hanover Street days and could really do with having this working again asap... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#616 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABqa2A3mdeyY4vcIeB1LD_we8trR0-Bks5t4VEvgaJpZM4RAzOy>.
I think this is actually done now, and documented at: https://github.com/DoESLiverpool/wiki/wiki/MQTT-services