New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a general-purpose hackable DoESLiverpool MQTT server #616

Closed
goatchurchprime opened this Issue Dec 13, 2017 · 51 comments

Comments

Projects
None yet
6 participants
@goatchurchprime
Contributor

goatchurchprime commented Dec 13, 2017

This is now a dominant protocol, so maybe it should be a component of our infrastructure, and to head off the Not Invented Here syndrome.

It should probably be on a spare RPi and documented on the wiki for anyone to access, or be on Doorbot if it is a robust enough system. It needs to have a fixed IPnumber.

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Dec 13, 2017

Contributor

See also #476

Contributor

goatchurchprime commented Dec 13, 2017

See also #476

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Dec 13, 2017

Contributor

In particular, @ajlennon has a stack of these Sonoff ESP8266 switches, and there's a fine Arduino-based app that does all the basics at:
https://bitbucket.org/xoseperez/espurna

Adrian has flashed the most recent version of espurna-1.10.1-itead-s20.bin onto a small S20 unit through a soldered serial port (with extra power from a 3V3 regulator).

Accessing using "picocom -b115200 /dev/ttyUSB9" allows me to see debug messages and find its IP number.

In the following example commands, the S20's IPnumber is 192.168.0.105 and my computer's IPnumber is 192.168.0.105

I've connected to its dashboard using:
http://192.168.0.107/# user:admin password: bubbl1n0

This lets me set the MQTT broker IPaddress and port.

For debugability you can run your own MQTT broker like so:
mosquitto -v -p 9005
and put in your own computer's ipnumber (192.168.0.105) into the Espurna dashboard webpage (port 9005) to see all the verbose messages as it connects and publishes notes when you press the button.

In another command window you see all the messages (basically button processes) from the device with:
mosquitto_sub -h 192.168.0.5 -p 9005 -t "ESPURNA_A6F5FA/#" -v

If you want to control the device it's very useful to have spotted in the verbose messages that:
Received SUBSCRIBE from esp8266a6f5fa
1513178879: ESPURNA_A6F5FA/relay/+/set (QoS 0)
1513178879: ESPURNA_A6F5FA/led/+/set (QoS 0)
1513178879: ESPURNA_A6F5FA/action/set (QoS 0)

Which means you can change the relay or the led with a command like so:
mosquitto_pub -h 192.168.0.5 -p 9005 -t "ESPURNA_A6F5FA/led/0/set" -m "1" -d


Now (with the device connecting itself to the DoESLiverpool wifi) we are ready to set the MQTT broker (in the absence of our own one) to: test.mosquitto.org port: 1883

It is now possible to be running the following (listening) subscriber when it resets and reconnects:
mosquitto_sub -h test.mosquitto.org -t "ESPURNA_A6F5FA/#" -v

and see that one of the messages is (very usefully):
ESPURNA_A6F5FA/ip 192.168.0.107

This is going to help when you want to go onto its dashboard and change things or reflash the code.

(At times the S20 resets itself repeatedly (seen in the serial port) it eventually makes its own access point you get to through 192.168.4.1 with the above wifi password.)

When connecting to the default public MQTT broker you can change things using:
mosquitto_pub -h test.mosquitto.org -t "ESPURNA_A6F5FA/relay/0/set" -m "0" -d
mosquitto_pub -h test.mosquitto.org -t "ESPURNA_A6F5FA/led/0/set" -m "1" -d

There's some further workthroughs with:
http://www.steves-internet-guide.com/mosquitto_pub-sub-clients/

Contributor

goatchurchprime commented Dec 13, 2017

In particular, @ajlennon has a stack of these Sonoff ESP8266 switches, and there's a fine Arduino-based app that does all the basics at:
https://bitbucket.org/xoseperez/espurna

Adrian has flashed the most recent version of espurna-1.10.1-itead-s20.bin onto a small S20 unit through a soldered serial port (with extra power from a 3V3 regulator).

Accessing using "picocom -b115200 /dev/ttyUSB9" allows me to see debug messages and find its IP number.

In the following example commands, the S20's IPnumber is 192.168.0.105 and my computer's IPnumber is 192.168.0.105

I've connected to its dashboard using:
http://192.168.0.107/# user:admin password: bubbl1n0

This lets me set the MQTT broker IPaddress and port.

For debugability you can run your own MQTT broker like so:
mosquitto -v -p 9005
and put in your own computer's ipnumber (192.168.0.105) into the Espurna dashboard webpage (port 9005) to see all the verbose messages as it connects and publishes notes when you press the button.

In another command window you see all the messages (basically button processes) from the device with:
mosquitto_sub -h 192.168.0.5 -p 9005 -t "ESPURNA_A6F5FA/#" -v

If you want to control the device it's very useful to have spotted in the verbose messages that:
Received SUBSCRIBE from esp8266a6f5fa
1513178879: ESPURNA_A6F5FA/relay/+/set (QoS 0)
1513178879: ESPURNA_A6F5FA/led/+/set (QoS 0)
1513178879: ESPURNA_A6F5FA/action/set (QoS 0)

Which means you can change the relay or the led with a command like so:
mosquitto_pub -h 192.168.0.5 -p 9005 -t "ESPURNA_A6F5FA/led/0/set" -m "1" -d


Now (with the device connecting itself to the DoESLiverpool wifi) we are ready to set the MQTT broker (in the absence of our own one) to: test.mosquitto.org port: 1883

It is now possible to be running the following (listening) subscriber when it resets and reconnects:
mosquitto_sub -h test.mosquitto.org -t "ESPURNA_A6F5FA/#" -v

and see that one of the messages is (very usefully):
ESPURNA_A6F5FA/ip 192.168.0.107

This is going to help when you want to go onto its dashboard and change things or reflash the code.

(At times the S20 resets itself repeatedly (seen in the serial port) it eventually makes its own access point you get to through 192.168.4.1 with the above wifi password.)

When connecting to the default public MQTT broker you can change things using:
mosquitto_pub -h test.mosquitto.org -t "ESPURNA_A6F5FA/relay/0/set" -m "0" -d
mosquitto_pub -h test.mosquitto.org -t "ESPURNA_A6F5FA/led/0/set" -m "1" -d

There's some further workthroughs with:
http://www.steves-internet-guide.com/mosquitto_pub-sub-clients/

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Dec 13, 2017

Contributor

There's a new sonoff POW unit that measures current*voltage power using an hlw8012 chip whose output interface seems to be a pulse-width frequency proportional to the power. If we get one this has been decoded in the espurna code:
https://bitbucket.org/xoseperez/hlw8012

Meanwhile I am still battling with the STPM11 development board that performs the same purpose, but via a dodgy implementation of the SPI protocol.

Contributor

goatchurchprime commented Dec 13, 2017

There's a new sonoff POW unit that measures current*voltage power using an hlw8012 chip whose output interface seems to be a pulse-width frequency proportional to the power. If we get one this has been decoded in the espurna code:
https://bitbucket.org/xoseperez/hlw8012

Meanwhile I am still battling with the STPM11 development board that performs the same purpose, but via a dodgy implementation of the SPI protocol.

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Dec 14, 2017

Contributor

@ajlennon did you have a stash of S20 plugs? the one I have is missing some covers so I can't install it without leaving some mains terminals bare.

Contributor

goatchurchprime commented Dec 14, 2017

@ajlennon did you have a stash of S20 plugs? the one I have is missing some covers so I can't install it without leaving some mains terminals bare.

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 20, 2017

Contributor

Awesome. I dunno why I'm not getting notified when my @ajlennon pops up though. I'm happening across this stuff accidentally.

Yeah I've got some around. I'm keen to get the firmware upgraded to use this...

Contributor

ajlennon commented Dec 20, 2017

Awesome. I dunno why I'm not getting notified when my @ajlennon pops up though. I'm happening across this stuff accidentally.

Yeah I've got some around. I'm keen to get the firmware upgraded to use this...

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 20, 2017

Contributor

How do I work out which Sonoff units have this hlw8012 in?

Contributor

ajlennon commented Dec 20, 2017

How do I work out which Sonoff units have this hlw8012 in?

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 20, 2017

Contributor

Ah right. So the "POW" is the unit with it in. Looks good. Fair bit more expensive than the basic model.

Contributor

ajlennon commented Dec 20, 2017

Ah right. So the "POW" is the unit with it in. Looks good. Fair bit more expensive than the basic model.

@amcewen

This comment has been minimized.

Show comment
Hide comment
@amcewen

amcewen Dec 23, 2017

Member

Given I had one of the old doorbot Raspberry Pi 1 boards to hand, I've set up a basic install of mosquitto on it (all scripted in Ansible, see MQTT-in-a-Box for details).

It will need securing - it accepts anonymous connections at the moment and doesn't run over TLS - but I figured I'd see how much it gets used first. It's available as mqtt.local on the default port of 1883.

Member

amcewen commented Dec 23, 2017

Given I had one of the old doorbot Raspberry Pi 1 boards to hand, I've set up a basic install of mosquitto on it (all scripted in Ansible, see MQTT-in-a-Box for details).

It will need securing - it accepts anonymous connections at the moment and doesn't run over TLS - but I figured I'd see how much it gets used first. It's available as mqtt.local on the default port of 1883.

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 24, 2017

Contributor

How much data can I publish to it before I break it?

Contributor

ajlennon commented Dec 24, 2017

How much data can I publish to it before I break it?

@amcewen

This comment has been minimized.

Show comment
Hide comment
@amcewen

amcewen Dec 26, 2017

Member

I don't know. How much data do you want to publish to it?

Member

amcewen commented Dec 26, 2017

I don't know. How much data do you want to publish to it?

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Dec 29, 2017

Contributor

Pinouts from the button end are Programmer, 3V3, Tx, Rx, Gnd, Nc

Contributor

goatchurchprime commented Dec 29, 2017

Pinouts from the button end are Programmer, 3V3, Tx, Rx, Gnd, Nc

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Dec 29, 2017

Contributor

Instructions for commissioning:

Look up your hardware here:
https://bitbucket.org/xoseperez/espurna/wiki/Hardware.md#markdown-header-iteadstudio-s20
Solder the pins on and connect to a serial port programmer (don't forget to hold the button in when you connect it up so it enters boot mode). Do "ls /dev/ttyU*" to find its address.

Look up your firmware here:
https://bitbucket.org/xoseperez/espurna/downloads/

Flashing instructions are here:
https://bitbucket.org/xoseperez/espurna/wiki/Backup.md
https://bitbucket.org/xoseperez/espurna/wiki/Binaries.md
esptool.py --port /dev/ttyUSB1 --baud 115200 write_flash -fm dout 0x00000 espurna-1.11.1-itead-sonoff-basic.bin
The autoreboot doesn't work, so you have to powercycle it yourself.

Configuring is at:
https://bitbucket.org/xoseperez/espurna/wiki/Configuration.md
Connect to the ESPURNA wifi, wifi password is fibonacci, then go to 192.168.4.1 and login with "admin" and "fibonacci". Then it forces you to change the password, so change it to "bubbl1n0"

Add your network to the Wifi list on the webpage, then you can reconnect to http://espurna_a7a528.local/# to get to your device (don't forget it's protected by a login password)

Contributor

goatchurchprime commented Dec 29, 2017

Instructions for commissioning:

Look up your hardware here:
https://bitbucket.org/xoseperez/espurna/wiki/Hardware.md#markdown-header-iteadstudio-s20
Solder the pins on and connect to a serial port programmer (don't forget to hold the button in when you connect it up so it enters boot mode). Do "ls /dev/ttyU*" to find its address.

Look up your firmware here:
https://bitbucket.org/xoseperez/espurna/downloads/

Flashing instructions are here:
https://bitbucket.org/xoseperez/espurna/wiki/Backup.md
https://bitbucket.org/xoseperez/espurna/wiki/Binaries.md
esptool.py --port /dev/ttyUSB1 --baud 115200 write_flash -fm dout 0x00000 espurna-1.11.1-itead-sonoff-basic.bin
The autoreboot doesn't work, so you have to powercycle it yourself.

Configuring is at:
https://bitbucket.org/xoseperez/espurna/wiki/Configuration.md
Connect to the ESPURNA wifi, wifi password is fibonacci, then go to 192.168.4.1 and login with "admin" and "fibonacci". Then it forces you to change the password, so change it to "bubbl1n0"

Add your network to the Wifi list on the webpage, then you can reconnect to http://espurna_a7a528.local/# to get to your device (don't forget it's protected by a login password)

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 30, 2017

Contributor

Hi @amcewen. I can't seem to connect to http://mqtt.local:1883 from the internal network (or directly to 192.168.0.159:1883) although I can SSH to it.

Can I get my paws on an SSH authentication to have a go at adding Node-Red support please? :)

Contributor

ajlennon commented Dec 30, 2017

Hi @amcewen. I can't seem to connect to http://mqtt.local:1883 from the internal network (or directly to 192.168.0.159:1883) although I can SSH to it.

Can I get my paws on an SSH authentication to have a go at adding Node-Red support please? :)

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 30, 2017

Contributor

@goatchurchprime reading this

https://bitbucket.org/xoseperez/espurna/src

It looks like "Non-invasive current sensor using internal ADC or ADC121 or ADS1115"

That seems like we might be able to achieve the "drawing / not drawing" sensing we discussed?

Contributor

ajlennon commented Dec 30, 2017

@goatchurchprime reading this

https://bitbucket.org/xoseperez/espurna/src

It looks like "Non-invasive current sensor using internal ADC or ADC121 or ADS1115"

That seems like we might be able to achieve the "drawing / not drawing" sensing we discussed?

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 30, 2017

Contributor

@goatchurchprime when in SoftAP mode the ESPURNA_XXXXXX AP seems to need a password. Any thoughts?

[Edit: Ah that's 'fibonacci' too. OK]

Contributor

ajlennon commented Dec 30, 2017

@goatchurchprime when in SoftAP mode the ESPURNA_XXXXXX AP seems to need a password. Any thoughts?

[Edit: Ah that's 'fibonacci' too. OK]

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 30, 2017

Contributor

And also I knew I'd seen those switches I mentioned somewhere.They are here too

"IteadStudio Sonoff Touch"

https://bitbucket.org/xoseperez/espurna/src

Details here: https://www.itead.cc/sonoff-touch.html

What about these in Neo-DoES?

Contributor

ajlennon commented Dec 30, 2017

And also I knew I'd seen those switches I mentioned somewhere.They are here too

"IteadStudio Sonoff Touch"

https://bitbucket.org/xoseperez/espurna/src

Details here: https://www.itead.cc/sonoff-touch.html

What about these in Neo-DoES?

@amcewen

This comment has been minimized.

Show comment
Hide comment
@amcewen

amcewen Dec 30, 2017

Member

The server now has its own repo in the DoES github account - https://github.com/DoESLiverpool/DoES-IoT-server and that's just had an additional role added to install and run a NodeRED server. Ideally we'd move the generic Mosquitto stuff into its own Ansible role too, but I've not done that yet. Similarly I haven't added any login details on the NodeRED instance, @ajlennon, do you fancy adding that?

@ajlennon should you be able to connect to http://mqtt.local:1883? Surely you're mixing protocols there? http://mqtt.local:1880 will get you the NodeRED instance (now).

Member

amcewen commented Dec 30, 2017

The server now has its own repo in the DoES github account - https://github.com/DoESLiverpool/DoES-IoT-server and that's just had an additional role added to install and run a NodeRED server. Ideally we'd move the generic Mosquitto stuff into its own Ansible role too, but I've not done that yet. Similarly I haven't added any login details on the NodeRED instance, @ajlennon, do you fancy adding that?

@ajlennon should you be able to connect to http://mqtt.local:1883? Surely you're mixing protocols there? http://mqtt.local:1880 will get you the NodeRED instance (now).

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 30, 2017

Contributor

OK so NodeRed is up and running on @amcewen 's MQtt-in-a-box (was there by default!)

I've got a noddy flow running to debug output from the ESPURNA I have here.

@goatchurchprime we might want to define a topic space so we can have a root that is appropriate for DOeS devices - a well designed topic space makes all the diffference for Pub/Sub - e.g. if I want to filter on all SonOffs, all SonOffs of a particular type etc. etc.

image

Contributor

ajlennon commented Dec 30, 2017

OK so NodeRed is up and running on @amcewen 's MQtt-in-a-box (was there by default!)

I've got a noddy flow running to debug output from the ESPURNA I have here.

@goatchurchprime we might want to define a topic space so we can have a root that is appropriate for DOeS devices - a well designed topic space makes all the diffference for Pub/Sub - e.g. if I want to filter on all SonOffs, all SonOffs of a particular type etc. etc.

image

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 30, 2017

Contributor

As with smart watches people seem incapable of doing decent product design work for digital home switches.

Nothing really leaps out here

https://ideaing.com/ideas/best-smart-light-switches-plugs

There is an interesting comment about an IFTTT recipe to control heating based on weather though @goatchurchprime as you were discussing.

@amcewen made a good point earlier that it would be nice to have a dimmer switch kind of arrangement for the physical input. Which could then publish via MQtt or LoRA or whatever I guess

Contributor

ajlennon commented Dec 30, 2017

As with smart watches people seem incapable of doing decent product design work for digital home switches.

Nothing really leaps out here

https://ideaing.com/ideas/best-smart-light-switches-plugs

There is an interesting comment about an IFTTT recipe to control heating based on weather though @goatchurchprime as you were discussing.

@amcewen made a good point earlier that it would be nice to have a dimmer switch kind of arrangement for the physical input. Which could then publish via MQtt or LoRA or whatever I guess

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 30, 2017

Contributor

@ajlennon should you be able to connect to http://mqtt.local:1883? Surely you're mixing protocols there? http://mqtt.local:1880 will get you the NodeRED instance (now).

You're absolutely right - that was the problem!

Contributor

ajlennon commented Dec 30, 2017

@ajlennon should you be able to connect to http://mqtt.local:1883? Surely you're mixing protocols there? http://mqtt.local:1880 will get you the NodeRED instance (now).

You're absolutely right - that was the problem!

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 30, 2017

Contributor

So I got things going quite nicely. I have a node-red flow (on my server as it has to be public facing for Amazon)

image

I have a new "DoES Liverpool" skill to play with, which calls the NodeRed flow. Currently that's just working out what parameters were sent for the "intention" - i.e. what I told it - and is returning to text to be spoken back to me. Obviously we can next start to wire in some actions... I'm thinking current consumption and costs and stuff would be good. And when the coffee is ready of course.

So I used Nathan Chantrell's blog post to get through it which was really useful -

https://nathan.chantrell.net/20160328/amazon-echo-alexa-skills-kit-integration-with-node-red/

I had trouble with certs and put some notes in a comment which is presumably still undergoing moderation. Basically AWS doesn't seem to like the LetsEncrypt CA root.

Contributor

ajlennon commented Dec 30, 2017

So I got things going quite nicely. I have a node-red flow (on my server as it has to be public facing for Amazon)

image

I have a new "DoES Liverpool" skill to play with, which calls the NodeRed flow. Currently that's just working out what parameters were sent for the "intention" - i.e. what I told it - and is returning to text to be spoken back to me. Obviously we can next start to wire in some actions... I'm thinking current consumption and costs and stuff would be good. And when the coffee is ready of course.

So I used Nathan Chantrell's blog post to get through it which was really useful -

https://nathan.chantrell.net/20160328/amazon-echo-alexa-skills-kit-integration-with-node-red/

I had trouble with certs and put some notes in a comment which is presumably still undergoing moderation. Basically AWS doesn't seem to like the LetsEncrypt CA root.

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Dec 30, 2017

Contributor

Similarly I haven't added any login details on the NodeRED instance, @ajlennon, do you fancy adding that?

I think we may want to have a chat about if/how we would connect a public facing https:// endpoint to AWS to give skills access first?

Contributor

ajlennon commented Dec 30, 2017

Similarly I haven't added any login details on the NodeRED instance, @ajlennon, do you fancy adding that?

I think we may want to have a chat about if/how we would connect a public facing https:// endpoint to AWS to give skills access first?

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jan 3, 2018

Contributor

The POW has arrived so will bring it in so we can try to get it flashed with Espurna and wire it up to an extension cable so we can try some current measurement @goatchurchprime.

Contributor

ajlennon commented Jan 3, 2018

The POW has arrived so will bring it in so we can try to get it flashed with Espurna and wire it up to an extension cable so we can try some current measurement @goatchurchprime.

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Jan 5, 2018

Contributor

@ajlennon
It is flashed and operating at http://espurna_3b6f59.local/#

The Xmas tree is using 29W.

And it's using the mqtt.local (although the IP number had to be hardcoded into the config to work).
mosquitto_sub -h mqtt.local -t "ESPURNA_3B6F59/#" -v

I've made my proposal for the device to interpret the measurements into events so we can derive complex behavior without any unreliable processing:
https://bitbucket.org/xoseperez/espurna/issues/369/report-energy-use-per-event-on-the-emon

I'm wondering if one could also add more sophisticated mqtt subscribers directly into the espurna (like if-this-then-that) so that they automatically chain. Goodnight lamp without any node-red stuff.

One could chain a whole series of electric kettles so that when one finishes boiling the next one starts boiling, so there is always one forever on the boil.

Contributor

goatchurchprime commented Jan 5, 2018

@ajlennon
It is flashed and operating at http://espurna_3b6f59.local/#

The Xmas tree is using 29W.

And it's using the mqtt.local (although the IP number had to be hardcoded into the config to work).
mosquitto_sub -h mqtt.local -t "ESPURNA_3B6F59/#" -v

I've made my proposal for the device to interpret the measurements into events so we can derive complex behavior without any unreliable processing:
https://bitbucket.org/xoseperez/espurna/issues/369/report-energy-use-per-event-on-the-emon

I'm wondering if one could also add more sophisticated mqtt subscribers directly into the espurna (like if-this-then-that) so that they automatically chain. Goodnight lamp without any node-red stuff.

One could chain a whole series of electric kettles so that when one finishes boiling the next one starts boiling, so there is always one forever on the boil.

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jan 15, 2018

Contributor

Got one of these on order for adding in Alexa voice control

https://t.co/5HoLkMgR55

Contributor

ajlennon commented Jan 15, 2018

Got one of these on order for adding in Alexa voice control

https://t.co/5HoLkMgR55

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jan 15, 2018

Contributor

@johnmckerrell - @goatchurchprime and I would like to experiment a bit with Amazon Echo voice control.

I have Amazon "skills" connected to Node-Red flows which enables me to be pretty flexible in what can be done.

However for this to work there has to be a public-facing https:// endpoint for Amazon to connect to.

We do have an internal mqtt.local box on the DoES network which @amcewen kindly set up.

I think it would be very useful to be able to work with such a box on the internal network from a control and monitoring standpoint and to connect to that from Amazon Alexa services

I am wondering if it is possible (sensible/secure) to allow a proxied https:// call from Amazon servers through to the internal box?

Or if there's another option?

Or it if is a no go... :)

Contributor

ajlennon commented Jan 15, 2018

@johnmckerrell - @goatchurchprime and I would like to experiment a bit with Amazon Echo voice control.

I have Amazon "skills" connected to Node-Red flows which enables me to be pretty flexible in what can be done.

However for this to work there has to be a public-facing https:// endpoint for Amazon to connect to.

We do have an internal mqtt.local box on the DoES network which @amcewen kindly set up.

I think it would be very useful to be able to work with such a box on the internal network from a control and monitoring standpoint and to connect to that from Amazon Alexa services

I am wondering if it is possible (sensible/secure) to allow a proxied https:// call from Amazon servers through to the internal box?

Or if there's another option?

Or it if is a no go... :)

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jan 15, 2018

Contributor

I can't work out all these people who don't trust a piece of software in a box (quite possibly opensource, transparent software designed in house) that decodes audio on command but are happy to be surrounded by other peoples' laptops willy nilly

Contributor

ajlennon commented Jan 15, 2018

I can't work out all these people who don't trust a piece of software in a box (quite possibly opensource, transparent software designed in house) that decodes audio on command but are happy to be surrounded by other peoples' laptops willy nilly

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Jan 17, 2018

Contributor

If you could find a speech to tweet app, it could be wired up through the cheerlights system.

@DefProc pointed to: http://jasperproject.github.io/ as something to install and make work.

Contributor

goatchurchprime commented Jan 17, 2018

If you could find a speech to tweet app, it could be wired up through the cheerlights system.

@DefProc pointed to: http://jasperproject.github.io/ as something to install and make work.

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jan 17, 2018

Contributor

So I best have a go at recapping for my own recollection if nothing else.

I made the mistake of trying to install Jasper from scratch which is my general preferred way to understand what's going on with code frameworks. Needless to say that proved to be a problem. I got Raspbian and a multitude of dependencies from the above link on there ok, including Jasper.

However the speech to text engines defeated me. On some general principles of privacy and internet independence I went with the only available offline option that doesn't need tuning. This proved complicated as the compilation of open-xxx kept locking up the RPi, I think due to lack of memory but even so it's a pretty painful failure mode. (instructions say compile as root, tried compiling as user, tried adding swap space. no joy)

After much ado I then stepped back to the prepared "this will work" image download and this promptly failed to boot on the v3 (nothing - no bootloader. I assume there's a v2 image and it doesn't work on v3?)

I moved onto some work by Matt Curry here

http://www.mattcurry.com/projects-2/the-jasper-project/jasper-project-useful-links/

Downloads were crawling along so I have downloaded at home and can report it is at least running and on the WiFi.

I now need to work out whether his "All In One" image supports the offline STT and TTS we want and find a way to do a simple test....

Contributor

ajlennon commented Jan 17, 2018

So I best have a go at recapping for my own recollection if nothing else.

I made the mistake of trying to install Jasper from scratch which is my general preferred way to understand what's going on with code frameworks. Needless to say that proved to be a problem. I got Raspbian and a multitude of dependencies from the above link on there ok, including Jasper.

However the speech to text engines defeated me. On some general principles of privacy and internet independence I went with the only available offline option that doesn't need tuning. This proved complicated as the compilation of open-xxx kept locking up the RPi, I think due to lack of memory but even so it's a pretty painful failure mode. (instructions say compile as root, tried compiling as user, tried adding swap space. no joy)

After much ado I then stepped back to the prepared "this will work" image download and this promptly failed to boot on the v3 (nothing - no bootloader. I assume there's a v2 image and it doesn't work on v3?)

I moved onto some work by Matt Curry here

http://www.mattcurry.com/projects-2/the-jasper-project/jasper-project-useful-links/

Downloads were crawling along so I have downloaded at home and can report it is at least running and on the WiFi.

I now need to work out whether his "All In One" image supports the offline STT and TTS we want and find a way to do a simple test....

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Jan 17, 2018

Contributor

I've got a demo working in the office from the cardboard box Rpi where "oscilloscope on" will turn the Sonoff Pow on, which is now connected to the fan.

It's a set of function calls in the src/somthing.py file (in the editor) that enable these commands.

Obviously new commands ought to be programmable in... also through a voice interface.

Next task is to put the sonoff pow in line with the coffee machine and find out how to record the last spike of energy (probably on the sonoff itself if it was programmable, because it's a pain doing it in some nodered thing)... so that it can answer "How old is the coffee?"

Jackie is looking at what handsfree headset is going to work on the PiZero (which the google thing should still work with), which gives some degree of microphone privacy, because it's only talking when you are wearing the device.

Contributor

goatchurchprime commented Jan 17, 2018

I've got a demo working in the office from the cardboard box Rpi where "oscilloscope on" will turn the Sonoff Pow on, which is now connected to the fan.

It's a set of function calls in the src/somthing.py file (in the editor) that enable these commands.

Obviously new commands ought to be programmable in... also through a voice interface.

Next task is to put the sonoff pow in line with the coffee machine and find out how to record the last spike of energy (probably on the sonoff itself if it was programmable, because it's a pain doing it in some nodered thing)... so that it can answer "How old is the coffee?"

Jackie is looking at what handsfree headset is going to work on the PiZero (which the google thing should still work with), which gives some degree of microphone privacy, because it's only talking when you are wearing the device.

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Apr 24, 2018

Contributor

@johnmckerrell Where's the issue/wiki page about the static IP address technology your friend was doing on a Rpi in the cabinet last night?

It was an idea to put the MQTT server (as mqtt.local) onto that same Rpi to save on unnecessary Rpi deployment. .

Contributor

goatchurchprime commented Apr 24, 2018

@johnmckerrell Where's the issue/wiki page about the static IP address technology your friend was doing on a Rpi in the cabinet last night?

It was an idea to put the MQTT server (as mqtt.local) onto that same Rpi to save on unnecessary Rpi deployment. .

@skos-ninja

This comment has been minimized.

Show comment
Hide comment
@skos-ninja

skos-ninja Apr 24, 2018

Hi Julian, I will be in on Thursday evening if we want to set it up then

skos-ninja commented Apr 24, 2018

Hi Julian, I will be in on Thursday evening if we want to set it up then

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime May 2, 2018

Contributor

@skos-ninja Did you get mqtt.local in at all? (I tried pinging just in case, and nothing came up)

Contributor

goatchurchprime commented May 2, 2018

@skos-ninja Did you get mqtt.local in at all? (I tried pinging just in case, and nothing came up)

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime May 14, 2018

Contributor

@skos-ninja have you left any instructions of how to log on to that loose Rpi so I can do it?

Contributor

goatchurchprime commented May 14, 2018

@skos-ninja have you left any instructions of how to log on to that loose Rpi so I can do it?

@johnmckerrell

This comment has been minimized.

Show comment
Hide comment
@johnmckerrell

johnmckerrell May 14, 2018

Member
Member

johnmckerrell commented May 14, 2018

@johnmckerrell

This comment has been minimized.

Show comment
Hide comment
@johnmckerrell

johnmckerrell May 15, 2018

Member

I installed mosquitto and changed the hostname of the pi to mqtt.local which I've confirmed works remotely. Let me know if you can access it ok (and @skos-ninja would be good to confirm that the hostname change will be fine but it seemed ok).

Member

johnmckerrell commented May 15, 2018

I installed mosquitto and changed the hostname of the pi to mqtt.local which I've confirmed works remotely. Let me know if you can access it ok (and @skos-ninja would be good to confirm that the hostname change will be fine but it seemed ok).

@skos-ninja

This comment has been minimized.

Show comment
Hide comment
@skos-ninja

skos-ninja May 16, 2018

Yep this should be fine. I would have recommended getting the pi another IP and set it up on that though

skos-ninja commented May 16, 2018

Yep this should be fine. I would have recommended getting the pi another IP and set it up on that though

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime
Contributor

goatchurchprime commented May 18, 2018

I'm summarizing the state of play at: https://github.com/DoESLiverpool/wiki/wiki/MQTT-services

@skos-ninja

This comment has been minimized.

Show comment
Hide comment
@skos-ninja

skos-ninja May 31, 2018

To note here this is now not working again after it killed the SD card on the UniFi Pi. As such I would not recommend putting anything else on that Pi unless it is to do directly with networking.

This has now stopped progress on #598 as we can not setup the new switch properly as it can not be adopted by the UniFi software due to the SD card being dead.

I would recommend to @johnmckerrell that we find the old raspberry pi that was set up for this and install that again and not to reinstall it on the networking Pi.

skos-ninja commented May 31, 2018

To note here this is now not working again after it killed the SD card on the UniFi Pi. As such I would not recommend putting anything else on that Pi unless it is to do directly with networking.

This has now stopped progress on #598 as we can not setup the new switch properly as it can not be adopted by the UniFi software due to the SD card being dead.

I would recommend to @johnmckerrell that we find the old raspberry pi that was set up for this and install that again and not to reinstall it on the networking Pi.

@skos-ninja skos-ninja reopened this May 31, 2018

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jun 1, 2018

Contributor

Hey @skos-ninja - have we got a guest network connection that we can safely attach devices to for external visibility?

Contributor

ajlennon commented Jun 1, 2018

Hey @skos-ninja - have we got a guest network connection that we can safely attach devices to for external visibility?

@skos-ninja

This comment has been minimized.

Show comment
Hide comment
@skos-ninja

skos-ninja Jun 1, 2018

Hi @ajlennon,

We do not currently have a way for you to access devices outside of the network right now.

We will probably not be doing this for a while as we still need to go through the steps of making this a secure but easy process.

skos-ninja commented Jun 1, 2018

Hi @ajlennon,

We do not currently have a way for you to access devices outside of the network right now.

We will probably not be doing this for a while as we still need to go through the steps of making this a secure but easy process.

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jun 1, 2018

Contributor

OK - I was accessing my Linux build box back in the Hanover Street days and could really do with having this working again asap...

Contributor

ajlennon commented Jun 1, 2018

OK - I was accessing my Linux build box back in the Hanover Street days and could really do with having this working again asap...

@johnmckerrell

This comment has been minimized.

Show comment
Hide comment
@johnmckerrell

johnmckerrell Jun 1, 2018

Member
Member

johnmckerrell commented Jun 1, 2018

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jun 1, 2018

Contributor

I think I've been using does.mckerrell.net or something but I don't have the laptop to hand atm ?

Was recommended and thinking of using ngrok moving forward but wanted to make sure that wasn't going to cause a security issue

Contributor

ajlennon commented Jun 1, 2018

I think I've been using does.mckerrell.net or something but I don't have the laptop to hand atm ?

Was recommended and thinking of using ngrok moving forward but wanted to make sure that wasn't going to cause a security issue

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jun 1, 2018

Contributor

Ideally I'd like to be able to access my box again, but to have it on a DMZ network segment such that if (god forbid) anything went wrong any attacker would not have access to anything in a secure segment?

Contributor

ajlennon commented Jun 1, 2018

Ideally I'd like to be able to access my box again, but to have it on a DMZ network segment such that if (god forbid) anything went wrong any attacker would not have access to anything in a secure segment?

@skos-ninja

This comment has been minimized.

Show comment
Hide comment
@skos-ninja

skos-ninja Jun 1, 2018

This is exactly what I was just talking to @johnmckerrell about!

I'm looking to set this up this weekend so that we can start to port forward and it not be a security issue.

If you are looking to help I will be around tomorrow (2nd June) working on this.

skos-ninja commented Jun 1, 2018

This is exactly what I was just talking to @johnmckerrell about!

I'm looking to set this up this weekend so that we can start to port forward and it not be a security issue.

If you are looking to help I will be around tomorrow (2nd June) working on this.

@ajlennon

This comment has been minimized.

Show comment
Hide comment
@ajlennon

ajlennon Jun 3, 2018

Contributor

Excellent. Great to hear we're thinking along similar lines! I've been away diving but am around and about for parts of the week

Contributor

ajlennon commented Jun 3, 2018

Excellent. Great to hear we're thinking along similar lines! I've been away diving but am around and about for parts of the week

@amcewen

This comment has been minimized.

Show comment
Hide comment
@amcewen

amcewen Jun 5, 2018

Member

I've recreated the setup documented in #616 (comment) onto a micro SD card provided by @goatchurchprime and he's stuck it into a Pi (the one by the TV and the Liver Bird), so there's now an MQTT server on mqtt.local again.

Member

amcewen commented Jun 5, 2018

I've recreated the setup documented in #616 (comment) onto a micro SD card provided by @goatchurchprime and he's stuck it into a Pi (the one by the TV and the Liver Bird), so there's now an MQTT server on mqtt.local again.

@goatchurchprime

This comment has been minimized.

Show comment
Hide comment
@goatchurchprime

goatchurchprime Aug 10, 2018

Contributor

I think this is actually done now, and documented at: https://github.com/DoESLiverpool/wiki/wiki/MQTT-services

Contributor

goatchurchprime commented Aug 10, 2018

I think this is actually done now, and documented at: https://github.com/DoESLiverpool/wiki/wiki/MQTT-services

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment