Encryption und Token Auth zur MS-Kommunikation

[christianTF]

https://www.loxone.com/dede/kb/api/

Loxberry-Modul muss Auth abstrahieren
Token wird lokal abgelegt und ggf. erneuert.
Plugins müssen für verschlüsselte Kommunikation explizit LB-Modul-Funktionen nutzen

Enc ab MS-V8.1
Token ab 9.0

[mschlenstedt]

Ich habe das mal auf "Later Future" gesetzt. Ich hoffe mal, dass die Änderungen nicht die Restful-Api betreffen.

Irgendwo hatten wir auch schonmal diskutiert einen Websocket from LoxBerry aufzubauen, sodass alle Plugins darüber direkt kommunizieren können. Find eich auch eine gute Idee.

[mschlenstedt]

@Woersty hat mir per Email schon einmal erste Versuche geschickt:

Hab etwas gespielt. Mit der node-lox-ws-api von https://github.com/alladdin kann man arbeiten.
Spart man sich die Neuentwicklung der Websocket Schnittstelle. Ist schnell genug und ab März ist unser Weg ja tot...

Funxt supi:

Licht AN

node test_loxapi.js miniserver Loxone lox 0b6a92c8-016c-b196-ffffeee000000000 100.000000 |grep quit
2017-11-10T23:28:10.945Z INFO : Sending string to Loxone MS and quit: jdev/sps/io/0b6a92c8-016c-b196-ffffeee000000000/100.000000

Licht AUS

node test_loxapi.js miniserver Loxone lox 0b6a92c8-016c-b196-ffffeee000000000 0.000000 |grep quit
2017-11-10T23:28:16.304Z INFO : Sending string to Loxone MS and quit: jdev/sps/io/0b6a92c8-016c-b196-ffffeee000000000/0.000000

Einzige Anpassung erstmal zum Test in test_loxapi.js

Zeile 11/12

var input_device = process.argv[5];
var input_value = process.argv[6];

Zeile 27

return text; // No Limit

Zeile 72-78

lox.on('authorized', function() {
log_info('Loxone authorized');
//interval = setInterval(function(){ lox.send_control_command("0b52aa23-010a-16f3-ffff112233445566/AI1", 'on') }, 5000);
//setTimeout(function(){ lox.send_control_command('test_vstup_tl', 'off') }, 5000);
setTimeout(function(){ lox.send_command('jdev/cfg/version') }, 5000);
input_device = typeof input_device !== 'undefined' ? input_device : '';
input_value = typeof input_value !== 'undefined' ? input_value : '';
if ( input_device != '' && input_value != '')
{
log_info('Sending string to Loxone MS and quit: jdev/sps/io/'+input_device+'/'+input_value);
setTimeout(function(){ lox.send_command('jdev/sps/io/'+input_device+'/'+input_value);
process.exit(); }, 10);
}
});

[Woersty]

Mal schauen ob ich da noch weiter komme.
Am Liebsten wäre mir eine minimalisierte Version, die einfach aufgerufen wird,
was macht und sich wieder beendet.

Hatte auch schon überlegt, ob es Sinn ergibt, alle events abzugreifen und in einer Datenbank den Status mitzuführen um Plugins einen leichten Zugriff darauf zu ermöglichen, bin mir aber nicht sicher, ob das sinnvoll ist. Bisher fällt mir kein Anwendungsfall ein.

Sprich ein Daemon läuft mit und loggt alle Geräte und deren Status mit.
Aber ohne History, jede ID gibt's dann nur 1x mit ihrem letzten bekannten Status.

Grüße

[christianTF]

Eine stehende Verbindung wäre das Optimum, aber bisher gibt es eigentlich keine Plugins oder Plugin-Wünsche, die direkt und ständig Zustände vom MS abrufen müssten.

Was man auf jeden Fall drin haben sollte, ist, dass der Token gespeichert und wiederverwendet wird, und nur im Fehlerfall ein neuer Token abgerufen wird.

Eine externe Funktion hätte den Vorteil, dass sie von Haus aus sprachunabhängig ist. Wenn ich was in LoxBerry::System übernehme, dann geht‘s nur unter Perl, und für PHP (oder Bash oder was auch immer) müsste es wieder extra implementiert werden.

Was ich mir vorstellen könnte, wäre ein lokales Webservice (unter Apache wäre am einfachsten). Nur lokal erreichbar, mit GET, so kann jeder aus Perl, PHP oder mit Curl seine Daten senden und abrufen.

[johny-mnemonic]

I am now actually thinking about implementing the same for the monitoring stuff I am running on Loxberry.

What I have now running on Loxberry is docker with three containers: InfluxDB, Graphana and Python
In that Python container, I am running my script, which is querying MS using REST calls every minute, getting around 150 values and sending these to InfluxDB.
It works OK, but I would like to monitor even more values with higher frequency and that seems to be a bit of an issue, as even these 150 values take around 20s to fetch, so I am looking at implementing websocket to monitor MS and send all interesting updates to InfluxDB.
My trouble is, that I am not a programmer, so even putting that simple Pyhon script for REST calls together was really an endeavor:-)
Main motivation is, that SD cards in MS are dying really fast with all those statistics enabled, so I am trying to create a solution to replace all statistics in MS, to be able to turn them off.

Having open websocket already provided by Loxberry would be nice and maybe there will be more people interested in having light years better monitoring (i.e. Graphana) in Loxberry than what MS provides.

I was afraid, it will create significant load on Loxberry, but so far it is so lightweight I can hardly spot anything. In case anyone is interested I can provide tutorial and with some help, I can even try to create Loxberry plugin from this.

[christianTF]

Statistic is, in fact, a use case for a open websocket connection.
To keep in mind is, that websocket pushes - as far as I know - only visualized elements.

Michael and me had started a statistic plugin that has not reached final state and used REST as well. Honestly, I already thought of switching this plugin from RRDTool to Grafana, as the storage design and calling syntax is nearly the same (Grafana seems to have had taken some ideas from RRDTool), but it has a great, dynamic visualization engine.

For such a websocket use case, it needs a service on Loxberry to fetch and cache the websocket data, and local resources could query the data locally.

[gahujipo]

and maybe there will be more people interested in having light years better monitoring (i.e. Graphana) in Loxberry than what MS provides.

I'm pretty sure there are plenty of them. Just look at the plugin wishlist, how many wish a "Statistik Plugin".

[christianTF]

I'm not common with docker currently, and I don't know how it hits performance of a Raspberry.

I finally mistakenly said Grafana, but meant Graphite with included Carbon as the data backend. Carbon has an excellent caching engine for writing data, therefore it won't hit the SD as a normal database would do. The Graphite/Carbon engine concentrates the data like RRDTool, with different levels of granulatity, like RRDTool.
Some weeks ago I did a test setup of Graphite/Carbon on my (Non-LoxBerry) Odroid XU4.

The nice thing is, that Grafana directly can use Graphite/Carbon as data backend.

I think Michael and me would be the first to finalize the statistics plugin, but first LB0.3.x must be finished.

If Woersty will continue invenstigations in the websocket module, this would be great preparatory work for a later statistic engine.

Christian

[johny-mnemonic]

@christianTF I was in the same boat when I started, but it turned out to be pretty simple. I just executed one shell script provided by docker and it did everything.

I do have broad experience with Grafana+Graphite+Collectd in my job. We do have it with Carbon/Whisper in HA on huge cluster, ingesting something like 900k metrics. And to be honest, that was the main motivation to try different backend then Carbon/Whisper combo, as the performance of that filesystem based "database" is really awful with high metric count as you can read here.
But considering the number of metrics it will be mostly collecting on Loxberry, there won't probably be any performance issues at all...

Another thing is, that setting up Graphite stack is more complicated, than just installing Grafana+InfluxDB. This combo does have almost no dependencies, so it is really dead simple. And that was second biggest decision factor for me.

I will try to find some comparison of I/O usage of InfluxDB vs Carbon/Whisper, as preserving the SD card is more important for us, then performance and simple installation.

Anyway, we should probably move this discussion to the thread for developing that statistic plugin ;-) This is way off-topic here.

[mschlenstedt]

Funktionierendes Beispiel in Python: https://github.com/StefanHaring/E2Lox/blob/master/websocketclient.py

Funktionierendes Beispiel in Perl: https://github.com/Bluebrain2000/Loxone-Token/blob/master/loxone_get_token.pl

Diskussion und Hinweise: https://www.loxforum.com/forum/german/software-konfiguration-programm-und-visualisierung/184124-rsa-key-and-iv-exchange-failing-not-found