Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make access to bridge easier #1

Open
CONSULitAS opened this issue Mar 3, 2023 · 40 comments
Open

make access to bridge easier #1

CONSULitAS opened this issue Mar 3, 2023 · 40 comments

Comments

@CONSULitAS
Copy link

CONSULitAS commented Mar 3, 2023

Hi,

a really great documentation! I really appreciate your work.

A guy recently posted this guide: https://blog.wyraz.de/allgemein/a-brief-analysis-of-the-tibber-pulse-bridge/

If you set the parameter „webserver_force_enable“ to true while in AP mode, you can connect to the bridge even while connected to your WiFi.

  • local hostname should be: tibber-bridge
  • user: admin
  • pass: as printed on device

This makes your guide even easier, because no reset and reboot needed for reconfiguration.

Only connect to the fresh/reseted device with your phone/tablet, change the parameter and everything is quite easy.

@MSkjel
Copy link
Owner

MSkjel commented Mar 4, 2023

Thanks for the nice comment.

This documentation is based on the Norwegian version of the Tibber Pulse that connects directly to a meter using MBUS. It is not using the IR reader and the Tibber Pulse IR Bridge as it seems the guide you linked to is using. Since the power supplied from the meters is very low, I dont think tibber added the ability to keep the webserver running on the firmwares running on our version of the Pulse. I will check it later, as I havent actually checked if there are any other pages accesible during the setup of the Pulse. There might be some kind of hidden page for configuring it.

Anyways, thank you!

@MSkjel
Copy link
Owner

MSkjel commented Mar 4, 2023

I should also specify the version of Pulse this is using, as I didnt know they were supplying other versions of it in other countries. Here in Norway the electricity company were forced to switch to AMS meters that had MBUS.

@JMyrng
Copy link

JMyrng commented Mar 6, 2023

@CONSULitAS Thanks for pointing that out.

and thanks @micw for the blog post!

I have found that this makes it much easier to get the data that the Pulse IR sends.
If you send an http get request to this address
http://tibber-bridge/data.json?node_id=1
you get directly the read data back, which is also sent via MQTT.

Now we just need to find an easy way to decrypt the data.

@MSkjel
Copy link
Owner

MSkjel commented Mar 6, 2023

@JMyrng I'm guessing they send the data in binary? I dont have access to a Pulse IR so I cant really help. On the Norwegian models they just retransmit the OBIS information the Meter spits out over MBUS. You can see the lists sent via this pdf https://www.nek.no/wp-content/uploads/2018/11/Kaifa-KFM_001.pdf. You might also want to take a look at https://github.com/toreamun/amshan-homeassistant which implements the OBIS list decoders for the NO version of the Pulse. You never know, they might be lazy and just encode it like an OBIS list

@micw
Copy link

micw commented Mar 6, 2023

Hey, glad that my post was of use and I'm impressed how fast it was discovered ^^

Actually I got my pulse running (by manually pairing it with the bridge) while still waiting for tibber to solve my update problem (need to update my blog post). Meanwhile I also discovered the "data.json" endpoint. It contains a complete SML message including pre- and suffix. The main drawback is that it requires polling, so a receiver would need to sync intervals with the meter.

Actually the "data" page of a node on the bridge does exactly this - polling every second.

@micw
Copy link

micw commented Mar 6, 2023

Fore reference, here's a data package, hex encoded (it comes in binary from the endpoint):

1b1b1b1b0101010176051266dc5f62006200726500000101760101076542
5a4444330b090145425a0100091f14010163688d0076051266dc60620062
0072650000070177010b090145425a0100091f14017262016505dddb987a
77078181c78203ff010101010445425a0177070100000009ff010101010b
090145425a0100091f140177070100010800ff6401018001621e52fb6900
00029b23c909ee0177070100010801ff0101621e52fb690000029b1dae89
ee0177070100010802ff0101621e52fb6900000000061a80000177070100
020800ff6401018001621e52fb69000000000d060ba00177070100100700
ff0101621b52fe5500015d0f0177070100240700ff0101621b52fe550000
55e60177070100380700ff0101621b52fe550000aa9e01770701004c0700
ff0101621b52fe5500005c8b0101016318b80076051266dc616200620072
6500000201710163aa39000000001b1b1b1b1a039209

@JMyrng
Copy link

JMyrng commented Mar 6, 2023

Actually I got my pulse running (by manually pairing it with the bridge) while still waiting for tibber to solve my update problem (need to update my blog post). Meanwhile I also discovered the "data.json" endpoint. It contains a complete SML message including pre- and suffix. The main drawback is that it requires polling, so a receiver would need to sync intervals with the meter.

Any idea on how to decode that sml message?

@micw
Copy link

micw commented Mar 6, 2023

Have a look at https://www.openmuc.org/de/sml/.

@micw
Copy link

micw commented Mar 6, 2023

@MSkjel Great documentation. I have an additional MBus adapter on my meter so that I can read the data there. But having access to the pulse data stream is great since it's updating way more often. I will link to it in my post.

If your doc is based on my findings, I'd also appreciate a link :-)
Just saw that it's independent and for a different pulse model. But all applies also to the german model with the pulse IR.

@gerdluthe
Copy link

@micw
Hast Du das mit dem Pulse umsetzten können? MIch würde eine einfache MQTT Anbindung auch brennend interessieren da ich zu Zeit einen IR Lesekopf nutze und diesen durch den Pulse ersetzen will/muss. Ich möchte aber die Daten für meinen IoBroker haben, von daher wäre die Lösung mit der Tibber Bridge die die Daten zweigleisig senden kann, ideal !

@micw
Copy link

micw commented Mar 11, 2023

Ich bastel gerade was. Wird eine kleine Java-Anwendung (auch als Docker-Container), welche die Pulse Bridge auslesen und das Ergebnis ausgeben kann. Mein Plan:

  • Auslesen über HTTP (unmodifizierte Pulse Bridge mit aktivem Webserver)
  • Auslesen über MQTT (MQTT-Server der Bridge modifiziert nach dieser Anleitung hier)
  • Ausgabe über MQTT
  • Ausgabe direkt an Influx und/oder Prometheus

Ich schreibe hier, wenn was testbares da ist. Aktuell bin ich am SML parsen.

@micw
Copy link

micw commented Mar 11, 2023

Here is my tool: https://github.com/micw/tibber-pulse-reader
It reads from HTTP, decodes SML and publishes to MQTT. Still lot of WiP but basics are in place ;-)

Example publish:

grid/energyImportTotal 28884446.53060
grid/energyImportTariff1 28883422.53060
grid/energyImportTariff2 1024.00000
grid/energyExportTotal 2262.00390
grid/powerTotal 3767.85
grid/powerL1 3458.08
grid/powerL2 141.39
grid/powerL3 168.38

@JMyrng
Copy link

JMyrng commented Mar 12, 2023

Still lot of WiP but basics are in place ;-)

Will test it next week 👍

@malloy139
Copy link

I wrote a small Python script that also pulls the data from http://tibber-bridge/data.json?node_id=1, decodes it and sends it via mqtt. Is there an easier way of getting the data than send a GET request every second?

@Ekran
Copy link

Ekran commented Apr 15, 2023

If you are able to build hardware, then yes! It is an easier way to get the data. I have an ESP, IR-Transistor an get the raw data stream from #1 (comment). I have written a manual decoder for SML but use also Tasmota with SML decoder. But I am able to connect electronic parts with an esp and breadboard. Schematic as sketch: https://chaos.social/@e_es/110202823482139258

@micw
Copy link

micw commented Apr 15, 2023

I made a circuit for this: evcc-io/evcc#7070 (comment)
Package of the first batch is in transit and should arrive next week. The circuit runs fine on a breadboard for a few weeks now.

Edit: tried the same with a single PNP but it was unreliable and I could not figure out why. So I switched back to my initial circuit that uses 2 NPN.

Edit2: Link to the circuit and PCB: https://oshwlab.com/mwyraz/ir-meter-head-simple_copy_copy

@Ekran
Copy link

Ekran commented Apr 15, 2023

Very nice OS Hardware! The Question is now, if it is suitable for malloy139.

@micw
Copy link

micw commented Apr 15, 2023

Actually it is the easiest way to get the data in the frequency the meter sends it, without interfering to much with tibber's hardware.

@elsbrock
Copy link

I already have an optical reader for my meter and would like to use this to report hourly readings instead of the Pulse – the meter I have is not supported as it seems to be using IEC 1107.

How does Pulse report the readings?

@micw
Copy link

micw commented Apr 20, 2023

How does Pulse report the readings?

Do you mean "does Tibber support this"? AFAIK only for Norway, there is an API to auto-report this.

You can use the normal APP api to report hourly readings but you would still be in the "monthly" tariff, so it's just a replacement for monthly manual reading. I have created https://github.com/micw/tibber-meter-uploader for this before I got my Pulse ;-)

@elsbrock
Copy link

Right, but how do they distinguish? Could I buy a Pulse (and thereby "connect" it with my account), simply to enable that part of the API and make them accept the readings?

@micw
Copy link

micw commented Apr 20, 2023

Pulse uses MQTT to AWS instead of the API. But would not help, the hourly tariff requires 10s values (or so) otherwise it will auto-switch back to monthly.

@jsphuebner
Copy link

The Tibber bridge sends OBIS data to the MQTT topic then?
I also have a meter that is not supported by the pulse and have my own readout solution already working reliably. Would really like to just send its data directly to Tibbers MQTT server
https://openinverter.org/forum/viewtopic.php?t=3654

@MSkjel
Copy link
Owner

MSkjel commented May 31, 2023

The Tibber bridge sends OBIS data to the MQTT topic then? I also have a meter that is not supported by the pulse and have my own readout solution already working reliably. Would really like to just send its data directly to Tibbers MQTT server https://openinverter.org/forum/viewtopic.php?t=3654

It does send the OBIS data, but it does also include some data before the raw OBIS list. I'll see if I cant dump a list or two for you. It does also send some status updates as JSON to the Tibber MQTT server, which includes a serial number though. I dont know if what you want to achieve is possible, depends on how they validate the data server-side. All pulses sold in norway use the serial number when adding the pulse to the tibber app.

@MSkjel MSkjel pinned this issue May 31, 2023
@jsphuebner
Copy link

Thanks very much, that would be awesome!
I have purchased the Pulse so I should be able to plug in its serial number where required.

@micw
Copy link

micw commented May 31, 2023

I think you need to pair your pulse first to get the client certificate required to communicate with their mqtt broker (some AWS IoT platform). Then you need to figure out what exact data you need to send.

@MSkjel
Copy link
Owner

MSkjel commented Jun 1, 2023

Thanks very much, that would be awesome! I have purchased the Pulse so I should be able to plug in its serial number where required.

pulse_bin.zip

This file contains the JSON which is sent maybe every 2-5 minutes to the tibber MQTT server, as well as OBIS List 1 and List 2.
Here is the documentation for my meter; https://www.nek.no/wp-content/uploads/2018/11/Kaifa-KFM_001.pdf. The raw OBIS data seems to start at offset 70, I think

@jsphuebner
Copy link

jsphuebner commented Jun 2, 2023

Cool, thanks very much! So the serial number in your json is the "ID" field and the 12-digit hex string? Both json and binary OBIS data are sent to the same topic /tibber-bridge/xxx/publish in separate publish() calls or appended binary to json?
My bridge seems to send some data as well, as the tibber app added the icon back when I plugged in the brige. Will check what it sends there and report back

UPDATE: tried the local mqtt server (simple ip address, no credentials, no TLS) but it fails:
esp_tls_last_esp_err: 32789
esp_tls_stack_err: 8576
esp_tls_cert_verify_flags: 0
error_type: 1
Almost looks like it's trying to use TLS despite me having deleted the keys

And just a warning for everyone: once you've deleted the keys it seems like you cannot put them back in because of some buffer overflow. Here is the request I intercepted:
curl 'http://192.168.188.31/params.json' -X POST -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0' -H 'Accept: */*' -H 'Accept-Language: de,en-US;q=0.7,en;q=0.3' -H 'Accept-Encoding: gzip, deflate' -H 'Referer: http://192.168.188.31/params/' -H 'Content-Type: multipart/form-data; boundary=---------------------------3783084835423229909669640946' -H 'Origin: http://192.168.188.31' -H 'Authorization: Basic YWRtaW46QVJTMi1CNzlC' -H 'Connection: keep-alive' --data-binary $'-----------------------------3783084835423229909669640946\r\nContent-Disposition: form-data; name="k"\r\n\r\nca_cert\r\n-----------------------------3783084835423229909669640946\r\nContent-Disposition: form-data; name="v"\r\n\r\n-----BEGIN CERTIFICATE-----MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzUwHhcNMDYxMTA4MDAwMDAwWhcNMzYwNzE2MjM1OTU5WjCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCvJAgIKXo1nmAMqudLO07cfLw8RRy7K+D+KQL5VwijZIUVJ/XxrcgxiV0i6CqqpkKzj/i5Vbext0uz/o9+B1fs70PbZmIVYc9gDaTY3vjgw2IIPVQT60nKWVSFJuUrjxuf6/WhkcIzSdhDY2pSS9KP6HBRTdGJaXvHcPaz3BJ023tdS1bTlr8Vd6Gw9KIl8q8ckmcY5fQGBO+QueQA5N06tRn/Arr0PO7gi+s3i+z016zy9vA9r911kTMZHRxAy3QkGSGT2RT+rCpSx4/VBEnkjWNHiDxpg8v+R70rfk/Fla4OndTRQ8Bnc+MUCH7lP59zuDMKz10/NIeWiu5T6CUVAgMBAAGjgbIwga8wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwbQYIKwYBBQUHAQwEYTBfoV2gWzBZMFcwVRYJaW1hZ2UvZ2lmMCEwHzAHBgUrDgMCGgQUj+XTGoasjY5rw8+AatRIGCx7GS4wJRYjaHR0cDovL2xvZ28udmVyaXNpZ24uY29tL3ZzbG9nby5naWYwHQYDVR0OBBYEFH/TZafC3ey78DAJ80M5+gKvMzEzMA0GCSqGSIb3DQEBBQUAA4IBAQCTJEowX2LP2BqYLz3q3JktvXf2pXkiOOzEp6B4Eq1iDkVwZMXnl2YtmAl+X6/WzChl8gGqCBpH3vn5fJJaCGkgDdk+bW48DW7Y5gaRQBi5+MHt39tBquCWIMnNZBU4gcmU7qKEKQsTb47bDN0lAtukixlE0kF6BWlKWE9gyn6CagsCqiUXObXbf+eEZSqVir2G3l6BFoMtEMze/aiCKm0oHw0LxOXnGiYZ4fQRbxC1lfznQgUy286dUV4otp6F01vvpX1FQHKOtw5rDgb7MzVIcbidJ4vEZV8NhnacRHr2lVz2XTIIM6RUthg/aFzyQkqFOFSDX9HoLPKsEdao7WNq-----END CERTIFICATE-----\r\n-----------------------------3783084835423229909669640946--\r\n'

And the reply:

Saved parameter k: 'ca_cert' v: '-----BEGIN CERTIFICATE-----MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkI�5�K<��?��?
                                                                                      A?Y�?�����?�q@�M@'

Oops. Same garbage shows up in params.json

@MSkjel
Copy link
Owner

MSkjel commented Jun 15, 2023

Cool, thanks very much! So the serial number in your json is the "ID" field and the 12-digit hex string? Both json and binary OBIS data are sent to the same topic /tibber-bridge/xxx/publish in separate publish() calls or appended binary to json? My bridge seems to send some data as well, as the tibber app added the icon back when I plugged in the brige. Will check what it sends there and report back

UPDATE: tried the local mqtt server (simple ip address, no credentials, no TLS) but it fails: esp_tls_last_esp_err: 32789 esp_tls_stack_err: 8576 esp_tls_cert_verify_flags: 0 error_type: 1 Almost looks like it's trying to use TLS despite me having deleted the keys

And just a warning for everyone: once you've deleted the keys it seems like you cannot put them back in because of some buffer overflow. Here is the request I intercepted: curl 'http://192.168.188.31/params.json' -X POST -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0' -H 'Accept: */*' -H 'Accept-Language: de,en-US;q=0.7,en;q=0.3' -H 'Accept-Encoding: gzip, deflate' -H 'Referer: http://192.168.188.31/params/' -H 'Content-Type: multipart/form-data; boundary=---------------------------3783084835423229909669640946' -H 'Origin: http://192.168.188.31' -H 'Authorization: Basic YWRtaW46QVJTMi1CNzlC' -H 'Connection: keep-alive' --data-binary $'-----------------------------3783084835423229909669640946\r\nContent-Disposition: form-data; name="k"\r\n\r\nca_cert\r\n-----------------------------3783084835423229909669640946\r\nContent-Disposition: form-data; name="v"\r\n\r\n-----BEGIN CERTIFICATE-----MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzUwHhcNMDYxMTA4MDAwMDAwWhcNMzYwNzE2MjM1OTU5WjCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCvJAgIKXo1nmAMqudLO07cfLw8RRy7K+D+KQL5VwijZIUVJ/XxrcgxiV0i6CqqpkKzj/i5Vbext0uz/o9+B1fs70PbZmIVYc9gDaTY3vjgw2IIPVQT60nKWVSFJuUrjxuf6/WhkcIzSdhDY2pSS9KP6HBRTdGJaXvHcPaz3BJ023tdS1bTlr8Vd6Gw9KIl8q8ckmcY5fQGBO+QueQA5N06tRn/Arr0PO7gi+s3i+z016zy9vA9r911kTMZHRxAy3QkGSGT2RT+rCpSx4/VBEnkjWNHiDxpg8v+R70rfk/Fla4OndTRQ8Bnc+MUCH7lP59zuDMKz10/NIeWiu5T6CUVAgMBAAGjgbIwga8wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwbQYIKwYBBQUHAQwEYTBfoV2gWzBZMFcwVRYJaW1hZ2UvZ2lmMCEwHzAHBgUrDgMCGgQUj+XTGoasjY5rw8+AatRIGCx7GS4wJRYjaHR0cDovL2xvZ28udmVyaXNpZ24uY29tL3ZzbG9nby5naWYwHQYDVR0OBBYEFH/TZafC3ey78DAJ80M5+gKvMzEzMA0GCSqGSIb3DQEBBQUAA4IBAQCTJEowX2LP2BqYLz3q3JktvXf2pXkiOOzEp6B4Eq1iDkVwZMXnl2YtmAl+X6/WzChl8gGqCBpH3vn5fJJaCGkgDdk+bW48DW7Y5gaRQBi5+MHt39tBquCWIMnNZBU4gcmU7qKEKQsTb47bDN0lAtukixlE0kF6BWlKWE9gyn6CagsCqiUXObXbf+eEZSqVir2G3l6BFoMtEMze/aiCKm0oHw0LxOXnGiYZ4fQRbxC1lfznQgUy286dUV4otp6F01vvpX1FQHKOtw5rDgb7MzVIcbidJ4vEZV8NhnacRHr2lVz2XTIIM6RUthg/aFzyQkqFOFSDX9HoLPKsEdao7WNq-----END CERTIFICATE-----\r\n-----------------------------3783084835423229909669640946--\r\n'

And the reply:

Saved parameter k: 'ca_cert' v: '-----BEGIN CERTIFICATE-----MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkI�5�K<��?��?
                                                                                      A?Y�?�����?�q@�M@'

Oops. Same garbage shows up in params.json

The JSON is sent separately from the OBIS Lists, but to the same topic. Not sure about the frequency of these reports, but it shouldnt have to be often. The pulse works even without them, but it will show as offline in the app (but the power consumption is still updated). Im not sure what data is actuallt included in the raw OBIS datastream. It seems there is some data appended to the OBIS Lists. Keep in mind that the binary file I linked contains two different OBIS Lists. List 1 and List 2, in that order. They are sent individually.

@BerriJ
Copy link

BerriJ commented Jun 16, 2023

I wrote a small Python script that also pulls the data from http://tibber-bridge/data.json?node_id=1, decodes it and sends it via mqtt. Is there an easier way of getting the data than send a GET request every second?

Would you agree to share your python script?
This would be easier to implement than a java application (at least for me). It would be awesome.

All the best,
BerriJ

@malloy139
Copy link

I wrote a small Python script that also pulls the data from http://tibber-bridge/data.json?node_id=1, decodes it and sends it via mqtt. Is there an easier way of getting the data than send a GET request every second?

Would you agree to share your python script? This would be easier to implement than a java application (at least for me). It would be awesome.

All the best, BerriJ

Sure, it's far from perfect but it works for me. I run it in a screen session.
You have to look at the obis_values list which entries you need.

import paho.mqtt.client as mqtt
import requests
from smllib import SmlStreamReader
from requests.auth import HTTPBasicAuth
import time

url = 'http://bridge_ip/data.json?node_id=1'

client = mqtt.Client("Zaehler")
client.connect("mqtt_server")

while True:
        try:
                res = requests.get(url, auth=HTTPBasicAuth('admin', 'bridge_password))

                stream = SmlStreamReader()
                stream.add(res.content)
                sml_frame = stream.get_frame()
                if sml_frame is None:
                        print('Bytes missing')
                        continue
                obis_values = sml_frame.get_obis()
                msg = f'{{"MT681_Power_cur":{obis_values[7].value},"MT681_Total_in":{obis_values[1].value/10000},"MT681_Total_out":{obis_values[4].value/10000}}}'
                #print(msg)
                client.publish("tele/zaehler/SENSOR",msg)
                time.sleep(1)
        except:
                print("Fehler")

@disaster123
Copy link

Did you get your local MQTT running? I'm trying to archieve the same - but i can't get the tibber to connect to my local mqtt

@Kristian-R
Copy link

Did you get your local MQTT running? I'm trying to archieve the same - but i can't get the tibber to connect to my local mqtt

Hi,

I was having a similar problem with my german Tibber Pulse+Bridge. I did not manage to really emtpy the "cert"-entries.

I recently bought a new Tibber Pulse+Bridge and activated it without using the tibber app (via AP and Webfrontend).

Long story short: this time it worked. Either it depends on the firmware version or my guess is that only if the cert fields are really empty the bridge can connect to the local mqtt-server. Otherwise it tries to use mqtts and fails.

I just now found out how to really clear a parameter (in the webfrontend some abbreviated field-name remains).

When I open up the console and enter for example
param_set mqtt_port \
the field is really cleared.

Perhaps you can try this.

Another approach I would have tried if the above had not worked for me:

  • set up a self signed certificate in your mqtt-broker and for the pulse (perhaps shortes possible key-length to avoid problems when pasting this key into the web frontend)
  • try to connect via TLS / mqtts

Kind regards
Kristian-R

@jsphuebner
Copy link

Yes, that worked! It is now mqtt_status.connected=true
Now waiting for something to be actually sent to the broker.
I'm listening to
tibber-bridge/xxx/publish
and
$aws/rules/ingest_tibber_bridge_data/tibber-bridge/xxx/publish

@jsphuebner
Copy link

And got some data
On $aws/rules/ingest_tibber_bridge_data/tibber-bridge/xxx/publish/TJH01/3494545e84000/event

{
  "$type" : "event",
  "id" : 10001,
  "class" : "efr32_node_event",
  "name" : "NODE_EVENT_METER_PROBING",
  "args" : {
    "meter_mode" : 2,
    "node" : {
      "$type" : "node",
      "id" : 1,
      "eui" : "dc8e95fffeb82bfe",
      "model" : "TFD01",
      "version" : "986-49df5891",
      "rssi" : -27.396474838256836,
      "seen" : 23,
      "available" : true,
      "pubcnt" : 0,
      "ota_state" : "idle",
      "ota_status" : "up2date",
      "manifest_version" : "986-49df5891",
      "up2date" : true
    }
  },
  "metrics" : {
    "$type" : "metrics",
    "ssid" : "tach",
    "rssi" : -59,
    "bssid" : "3481c4e31c3f",
    "mqttcon" : 0,
    "wificon" : 0,
    "id" : "3494545e8400",
    "uptime" : 189,
    "efr_uptime" : 188,
    "vcc" : 3.497999906539917,
    "vacrms" : 225.68521118164062,
    "heap" : 116228,
    "coredump_available" : true,
    "model" : "TJH01",
    "esp_version" : "1268-f95e5239",
    "efr_version" : "557-dee09095",
    "nodes" : [ {
      "$type" : "node",
      "id" : 1,
      "eui" : "dc8e95fffeb82bfe",
      "model" : "TFD01",
      "version" : "986-49df5891",
      "rssi" : -27.396474838256836,
      "seen" : 23,
      "available" : true,
      "pubcnt" : 0,
      "ota_state" : "idle",
      "ota_status" : "up2date",
      "manifest_version" : "986-49df5891",
      "up2date" : true
    } ],
    "ota" : [ {
      "model" : "tibber-pulse-ir-hub-esp32",
      "manifest_version" : "1268-f95e5239",
      "current_version" : "1268-f95e5239",
      "state" : "up2date"
    }, {
      "model" : "tibber-pulse-ir-hub-css",
      "manifest_version" : "1268-f95e5239",
      "current_version" : "1268-f95e5239",
      "state" : "up2date"
    }, {
      "model" : "tibber-pulse-ir-hub-js",
      "manifest_version" : "1268-f95e5239",
      "current_version" : "1268-f95e5239",
      "state" : "up2date"
    }, {
      "model" : "tibber-pulse-ir-hub-efr32-fg1",
      "manifest_version" : "557-dee09095",
      "current_version" : "557-dee09095",
      "state" : "up2date"
    } ],
    "hub_ota_status" : "up2date",
    "manifest_state" : "synced",
    "netflags" : 25,
    "ip" : "192.168.188.31",
    "gateway" : "192.168.188.1",
    "dns0" : "192.168.188.1",
    "dns1" : "0.0.0.0",
    "dns2" : "0.0.0.0",
    "mac" : "34:94:54:5e:84:0",
    "ap_mac" : "34:94:54:5e:84:1"
  }
}

And on $aws/rules/ingest_tibber_bridge_data/tibber-bridge/xxx/publish/TJH01/3494545e84000/metric

{
  "$type" : "status",
  "status" : {
    "$type" : "metrics",
    "ssid" : "tach",
    "rssi" : -58,
    "bssid" : "3481c4e31c3f",
    "mqttcon" : 0,
    "wificon" : 0,
    "id" : "3494545e8400",
    "uptime" : 250,
    "efr_uptime" : 249,
    "vcc" : 3.49399995803833,
    "vacrms" : 224.85430908203125,
    "heap" : 118048,
    "coredump_available" : true,
    "model" : "TJH01",
    "esp_version" : "1268-f95e5239",
    "efr_version" : "557-dee09095",
    "nodes" : [ {
      "$type" : "node",
      "id" : 1,
      "eui" : "dc8e95fffeb82bfe",
      "model" : "TFD01",
      "version" : "986-49df5891",
      "rssi" : -26.945465087890625,
      "seen" : 0,
      "available" : true,
      "pubcnt" : 0,
      "ota_state" : "idle",
      "ota_status" : "up2date",
      "manifest_version" : "986-49df5891",
      "up2date" : true
    } ],
    "ota" : [ {
      "model" : "tibber-pulse-ir-hub-esp32",
      "manifest_version" : "1268-f95e5239",
      "current_version" : "1268-f95e5239",
      "state" : "up2date"
    }, {
      "model" : "tibber-pulse-ir-hub-css",
      "manifest_version" : "1268-f95e5239",
      "current_version" : "1268-f95e5239",
      "state" : "up2date"
    }, {
      "model" : "tibber-pulse-ir-hub-js",
      "manifest_version" : "1268-f95e5239",
      "current_version" : "1268-f95e5239",
      "state" : "up2date"
    }, {
      "model" : "tibber-pulse-ir-hub-efr32-fg1",
      "manifest_version" : "557-dee09095",
      "current_version" : "557-dee09095",
      "state" : "up2date"
    } ],
    "hub_ota_status" : "up2date",
    "manifest_state" : "synced",
    "netflags" : 25,
    "ip" : "192.168.188.31",
    "gateway" : "192.168.188.1",
    "dns0" : "192.168.188.1",
    "dns1" : "0.0.0.0",
    "dns2" : "0.0.0.0",
    "mac" : "34:94:54:5e:84:0",
    "ap_mac" : "34:94:54:5e:84:1"
  }
}

@disaster123
Copy link

a lot more data than expected. I hope to be able to simulate the pulse device - as i need a constant update rate of 1s which the tibber pulse does not provide to me...

@jsphuebner
Copy link

Yes, quite complex. There is also a number of other states besides "NODE_EVENT_METER_PROBING", especially at bootup. Also there is a separate metric for the Pulse IR node (same content as http://192.168.188.31/nodes/1/metrics). And I haven't seen data yet... Did you intercept any?

@disaster123
Copy link

Haven't tried it yet - due to lack of time... but it's on my todo list....

@jsphuebner
Copy link

jsphuebner commented Jul 17, 2023

I've configured the pulse for the impulse interface (10000 pulses/kWh) and am flashing a LED at 10 Hz at it. This gives on
$aws/rules/ingest_tibber_bridge_data/tibber-bridge/xxx/publish/TFD01/dc8e95fffeb82bfe/imp
{"$type": "imp_data", "timestamp_ms": 3246102,"delta_ms": 99,"kw":3.636364, "kwh": 0.2443}

The event messages stopped, only the metrics continue to be sent

Now lets see if the app displays anything. Currently it says "no data"

@jsphuebner
Copy link

I have continued the data format discussion in a new issue #6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests