Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checksum mismatch #2577

Closed
bkbartk opened this issue Oct 21, 2021 · 43 comments
Closed

Checksum mismatch #2577

bkbartk opened this issue Oct 21, 2021 · 43 comments
Assignees

Comments

@bkbartk
Copy link

bkbartk commented Oct 21, 2021

The problem

After installing ESPHome and update the device via home assistant I only get checksum mismatches

`[09:04:00][C][wifi_signal.sensor:009]: WiFi Signal 'Wi-Fi Signal'
[09:04:00][C][wifi_signal.sensor:009]: Device Class: 'signal_strength'
[09:04:00][C][wifi_signal.sensor:009]: State Class: 'measurement'
[09:04:00][C][wifi_signal.sensor:009]: Unit of Measurement: 'dBm'
[09:04:00][C][wifi_signal.sensor:009]: Accuracy Decimals: 0
[09:04:00][C][wifi_info:009]: WifiInfo IPAddress 'IP Address'
[09:04:00][C][wifi_info:011]: WifiInfo SSID 'Wi-Fi SSID'
[09:04:00][C][wifi_info:012]: WifiInfo BSSID 'Wi-Fi BSSID'
[09:04:00][E][dsmr:154]: !4119

^

Checksum mismatch
[09:04:01][E][dsmr:154]: !4023

^

Checksum mismatch
[09:04:02][E][dsmr:154]: !5906

^

Checksum mismatch`

Which version of ESPHome has the issue?

2021.10.0

What type of installation are you using?

Home Assistant Add-on

Which version of Home Assistant has the issue?

core-2021.9.7

What platform are you using?

ESP8266

Board

slimmelezer

Component causing the issue

No response

Example YAML snippet

https://raw.githubusercontent.com/zuidwijk/dsmr/main/slimmelezer.yaml

Anything in the logs that might be useful for us?

[09:04:00][C][wifi_signal.sensor:009]: WiFi Signal 'Wi-Fi Signal'
[09:04:00][C][wifi_signal.sensor:009]:   Device Class: 'signal_strength'
[09:04:00][C][wifi_signal.sensor:009]:   State Class: 'measurement'
[09:04:00][C][wifi_signal.sensor:009]:   Unit of Measurement: 'dBm'
[09:04:00][C][wifi_signal.sensor:009]:   Accuracy Decimals: 0
[09:04:00][C][wifi_info:009]: WifiInfo IPAddress 'IP Address'
[09:04:00][C][wifi_info:011]: WifiInfo SSID 'Wi-Fi SSID'
[09:04:00][C][wifi_info:012]: WifiInfo BSSID 'Wi-Fi BSSID'
[09:04:00][E][dsmr:154]: !4119

 ^

Checksum mismatch
[09:04:01][E][dsmr:154]: !4023

 ^

Checksum mismatch
[09:04:02][E][dsmr:154]: !5906

 ^

Checksum mismatch

Additional information

there were no isseus with v 2021.9

@bobcashflows
Copy link

I have the same issue. After upgrading to 2021.10 only checksum errors. I've reverted back to 2021.9 and everything works perfect again.

@michaelarnauts
Copy link

michaelarnauts commented Oct 21, 2021

I see the same issue, I was using the following options on 2021.9, and since 2021.10 uses glmnet/Dsmr@0.5 now by default, the change should be somewhere in the changes made in ESPHome itself.

  platformio_options:
    lib_deps: glmnet/Dsmr@0.5
    lib_ignore: glmnet/Dsmr@0.3

I'm using a DSMRv5 meter, so I should not have to use the crc_check: false option.

@michaelarnauts
Copy link

I think the issue is due to this PR: esphome/esphome#2382

@michaelarnauts
Copy link

When using crc_check: false, I get the following debug logs:

[D][sensor:113]: 'Voltage': Sending state 231.39999 V with 1 decimals of accuracy�
[D][sensor:113]: 'Current': Sending state 8.50000 A with 1 decimals of accuracy�
[D][sensor:113]: 'Voltage': Sending state 231.30000 V with 1 decimals of accuracy�
[D][sensor:113]: 'Current': Sending state 8.47000 A with 1 decimals of accuracy�
[E][dsmr:154]: 0-0:96.1.130333735383730)
            ^
Obis ID has number over 255�
[E][dsmr:154]: 0-0:96.1.11101445S)
            ^
Obis ID has number over 255�
[D][sensor:113]: 'Power Consumed': Sending state 1.78500 kW with 3 decimals of accuracy�
[D][sensor:113]: 'Voltage': Sending state 231.60001 V with 1 decimals of accuracy�
[D][sensor:113]: 'Current': Sending state 8.43000 A with 1 decimals of accuracy�
[D][sensor:113]: 'Voltage': Sending state 231.50000 V with 1 decimals of accuracy�
[D][sensor:113]: 'Current': Sending state 8.33000 A with 1 decimals of accuracy�
[E][dsmr:154]: )
^
OBIS id Empty�
[D][sensor:113]: 'Power Consumed': Sending state 1.75300 kW with 3 decimals of accuracy�
[E][dsmr:154]: 0-0:96.1.11101455S)
            ^
Obis ID has number over 255�
[E][dsmr:154]: 749*kW)
  ^
Obis ID has number over 255�
[E][dsmr:154]: 1-0:1.7.:22.7.0(00.000*kW)
        ^
Missing (�
[D][sensor:113]: 'Voltage': Sending state 231.30000 V with 1 decimals of accuracy�
[D][sensor:113]: 'Current': Sending state 8.23000 A with 1 decimals of accuracy�
[D][sensor:113]: 'Power Consumed': Sending state 1.76700 kW with 3 decimals of accuracy�
[D][sensor:113]: 'Voltage': Sending state 231.39999 V with 1 decimals of accuracy�
[D][sensor:113]: 'Current': Sending state 8.39000 A with 1 decimals of accuracy�
[D][sensor:113]: 'Voltage': Sending state 231.89999 V with 1 decimals of accuracy�
[D][sensor:113]: 'Current': Sending state 8.54000 A with 1 decimals of accuracy�
[D][sensor:113]: 'Energy Consumed': Sending state 751.44397 kWh with 3 decimals of accuracy�
[E][dsmr:154]: 1-0:1.7.:22.7.0(00.000*kW)
        ^
Missing (�
[E][dsmr:154]: 1-0:1.7.:22.7.0(00.000*kW)
        ^
Missing (�
[E][dsmr:154]: 1-0:1.7.:22.7.0(00.000*kW)
        ^
Missing (�
[D][sensor:113]: 'Power Consumed': Sending state 1.79500 kW with 3 decimals of accuracy�
[E][dsmr:154]: 0-0:96.1.11101515S)
            ^
Obis ID has number over 255�
...

@bkbartk
Copy link
Author

bkbartk commented Oct 21, 2021

thanks for the crc input.
this one does a little bit more, still I have some issues.
for some reason after restoring the backup esphome wouldn't start anymore.
but I was lucky enough to still have a local esphome installation on my windows machine. I used that one to get my sensor to work again
esphome run slimmelezer.yaml

@zuidwijk
Copy link

I've changed the yaml code https://raw.githubusercontent.com/zuidwijk/dsmr/main/slimmelezer.yaml
I had the old multiplier of 1000 and decimals 0 in it, that has changed.

@probot-esphome
Copy link

Hey there @glmnet, mind taking a look at this issue as it has been labeled with an integration (dsmr) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)

@bkbartk
Copy link
Author

bkbartk commented Oct 21, 2021

@zuidwijk
this fix makes the issue worse for me,
now I still get the error. but also the reader only reads the wifi information, in the webinterface I now don't see any info either

[edit]
After this change I needed to power the slimmelezer via usb to be able to do an OTA update again,
both OTA and file upload otherwise would loose connection. Really strange

@zuidwijk
Copy link

@bkbartk the Wemos is consuming about 180~200mA in "normal" use. When configuring wifi, that can easily got to the max 250mA or a little above and than you get brownouts. I always advice to use usb power during wifi setup.

Maybe the wifi usage is too high during transfer of OTA files.

@bkbartk
Copy link
Author

bkbartk commented Oct 21, 2021

good to know,
still the fix doesn't work for me, not sure why

@tedenda
Copy link

tedenda commented Oct 21, 2021

I see the same typ of issue as @bkbartk and @michaelarnauts:

[13:21:05][D][text_sensor:067]: 'DSMR Identification': Sending state 'ELL5\253833635_A'
[13:21:15][D][sensor:113]: 'Energy Consumed': Sending state 16217.10156 kWh with 3 decimals of accuracy
[13:21:15][D][sensor:113]: 'Voltage Phase 1': Sending state 231.70000 V with 1 decimals of accuracy
[13:21:15][D][sensor:113]: 'Voltage Phase 2': Sending state 232.50000 V with 1 decimals of accuracy
[13:21:15][D][sensor:113]: 'Voltage Phase 3': Sending state 233.10001 V with 1 decimals of accuracy
[13:21:15][D][sensor:113]: 'Current Phase 1': Sending state 6.70000 A with 1 decimals of accuracy
[13:21:15][D][sensor:113]: 'Current Phase 2': Sending state 5.20000 A with 1 decimals of accuracy
[13:21:15][D][sensor:113]: 'Current Phase 3': Sending state 6.10000 A with 1 decimals of accuracy
[13:21:15][D][sensor:113]: 'Power Consumed Phase 1': Sending state 1.25100 kW with 3 decimals of accuracy
[13:21:15][D][text_sensor:067]: 'DSMR Identification': Sending state 'ELL5\253833635_A'
[13:21:25][E][dsmr:154]: 1-0:613.7.0(0000.767kvar)
^
Obis ID has number over 255
[13:21:35][E][dsmr:154]: 1-0:21.7.0(0001.909
kvar)
^
Invalid unit
[13:21:45][D][sensor:113]: 'Energy Consumed': Sending state 16217.12793 kWh with 3 decimals of accuracy
[13:21:45][D][sensor:113]: 'Voltage Phase 1': Sending state 231.50000 V with 1 decimals of accuracy
[13:21:45][D][sensor:113]: 'Voltage Phase 2': Sending state 232.89999 V with 1 decimals of accuracy
[13:21:45][D][sensor:113]: 'Voltage Phase 3': Sending state 233.10001 V with 1 decimals of accuracy
[13:21:45][D][sensor:113]: 'Current Phase 1': Sending state 6.60000 A with 1 decimals of accuracy
[13:21:45][D][sensor:113]: 'Current Phase 2': Sending state 5.30000 A with 1 decimals of accuracy
[13:21:45][D][sensor:113]: 'Current Phase 3': Sending state 6.00000 A with 1 decimals of accuracy
[13:21:45][D][text_sensor:067]: 'DSMR Identification': Sending state 'ELL5\253833635_A'
[13:21:55][E][dsmr:154]: 1-0:41.7.0(00003.7.0(0000.795kvar)
^
Invalid number
[13:22:05][E][dsmr:154]: 1-0:1.8.0(0001628.0(00002107.983
kvarh)
^
Invalid number
[13:22:15][E][dsmr:154]: 1-0:61.7.0(0000.967kW3.7.0(0000.770kvar)
^
Extra data
[13:22:25][E][dsmr:154]: 1-0:613.7.0(0000.793*kvar)
^
Obis ID has number over 255
[13:22:35][D][sensor:113]: 'Energy Consumed': Sending state 16217.17188 kWh with 3 decimals of accuracy
[13:22:35][D][sensor:113]: 'Voltage Phase 1': Sending state 229.60001 V with 1 decimals of accuracy
[13:22:35][D][sensor:113]: 'Voltage Phase 2': Sending state 232.00000 V with 1 decimals of accuracy
[13:22:35][D][sensor:113]: 'Voltage Phase 3': Sending state 232.00000 V with 1 decimals of accuracy
[13:22:35][D][sensor:113]: 'Current Phase 1': Sending state 6.50000 A with 1 decimals of accuracy
[13:22:35][D][sensor:113]: 'Current Phase 2': Sending state 5.30000 A with 1 decimals of accuracy
[13:22:35][D][sensor:113]: 'Current Phase 3': Sending state 6.00000 A with 1 decimals of accuracy
[13:22:35][D][text_sensor:067]: 'DSMR Identification': Sending state 'ELL5\253833635_A'

@petergebruers
Copy link

Try setting "rx_buffer_size: 1000" under "uart", like this (extract from my actual setup):

uart:
  id: uart_p1
  rx_pin: RX
  baud_rate: 115200
  # ESP8266 has 128 byte hardware fifo, largest DSMR v5 data payload is 1024 bytes.
  # Add a few bytes for the headear and set rx_buffer_size to 1000 is enough to fit
  # one telegram per second.
  rx_buffer_size: 1000

My guess is you all have meters that run at 115200 bit speed and create one telegram per second, like me, and the "chunking" in PR: esphome/esphome#2382 causes long enough pauses to drop a (few?) bytes...

Have been running ESPHome version 2021.10.1 on a D1 mini ESP8266 for 15 minutes without an issue... I left "crc_check: true" under "dsmr" (which is the default).

@jozg
Copy link

jozg commented Oct 22, 2021

Hello Petergebruers,

The rx_buffer_size: 1000 saved my day.
Since the 2021.10.1 uesphome update the slimmemeter was'nt working anymore, and the rx_buffer_size: 1000 did the trick.
Thank you, and thank you all.

Regards.

@bkbartk
Copy link
Author

bkbartk commented Oct 22, 2021

the rx_buffer_size fixed the issue.
not sure why your rx_pin is different then mine, So I left it to D7.

thank you all for responding.

@michaelarnauts
Copy link

michaelarnauts commented Oct 22, 2021

This indeed fixes it. I've set mine to rx_buffer_size: 1024 since that's a nicer value :)

@zuidwijk you'll want to add this rx_buffer_size to https://raw.githubusercontent.com/zuidwijk/dsmr/main/slimmelezer.yaml also.

@petergebruers
Copy link

@jozg you thank for your feedback!

@bkbartk you say:

not sure why your rx_pin is different then mine, So I left it to D7.

And I can explain that. I am not using any converter board, just a transistor and resistor, connected to pad "RX" marked on my D1 mini board (aka U0RXD). This allows me to use the hardware UART and minimum component count. I did disable the hardware logger by setting "baud_rate: 0" under "logger:"

@michaelarnauts - ah you prefer 1024... I understand... Hahaha. There are only 10 types of people in the world – those who understand binary, and those who don’t.- try 1049 because that is a prime number, how cool is that! Thank you for your feedback!

@tedenda
Copy link

tedenda commented Oct 22, 2021

@petergebruers Thanks a lot, this fixed the problem for me to!

@mmakaay
Copy link
Member

mmakaay commented Oct 22, 2021

I created the chunking mechanism to prevent the DSMR component from continuously reading and blocking the device. There were other devices on which things went wrong because of the lack of chunking, even up to occasional reboots, because of watch dog timer issues.

But with DMSR5 meters that are sending data every second, apparently reading the data in a more friendly way breaks here. I investigated a telegram that was captured using UART debugging (i.e. an open PR, but long live external components), and there are indeed bytes missing.

Instead of increasing the read buffer (which as such is a fine solution in case processing can't keep up with the data as in this case), I think that it's also possible to increase the size of the chunk that is read per loop() cycle. While still being sure that control is handed back to the main loop once in a while, less data might drop off the bandwagon in the UART buffer.

Question now is: what would be a good chunk size?
I made a branch in which I have played with the chunk size, and I have made it 500 bytes now.
With that size, I can still get good readings, even when I'm introducing verbose logging and a test component that blocks 200ms on every loop cycle. So this would still solve the issues that the chunking was introduced for, while reading a lot more per cycle which might fix the ticket issue.

Too people with the issue (I don't have a DSMR5 myself), can you please try the following in your code, with the increased rx_buffer_size: ... line commented out?

external_components:
  - source:
      type: git
      url: https://github.com/mmakaay/esphome
      ref: fix-dsmr-read-chunk-size
    components: [ "dsmr" ]
    refresh: 60s

If this proves to be the solution, I'll submit a PR to get it updated.

@svenahrensnl
Copy link

svenahrensnl commented Oct 22, 2021

I created the chunking mechanism to prevent the DSMR component from continuously reading and blocking the device. There were other devices on which things went wrong because of the lack of chunking, even up to occasional reboots, because of watch dog timer issues.

But with DMSR5 meters that are sending data every second, apparently reading the data in a more friendly way breaks here. I investigated a telegram that was captured using UART debugging (i.e. an open PR, but long live external components), and there are indeed bytes missing.

Instead of increasing the read buffer (which as such is a fine solution in case processing can't keep up with the data as in this case), I think that it's also possible to increase the size of the chunk that is read per loop() cycle. While still being sure that control is handed back to the main loop once in a while, less data might drop off the bandwagon in the UART buffer.

Question now is: what would be a good chunk size? I made a branch in which I have played with the chunk size, and I have made it 500 bytes now. With that size, I can still get good readings, even when I'm introducing verbose logging and a test component that blocks 200ms on every loop cycle. So this would still solve the issues that the chunking was introduced for, while reading a lot more per cycle which might fix the ticket issue.

Too people with the issue (I don't have a DSMR5 myself), can you please try the following in your code, with the increased rx_buffer_size: ... line commented out?

external_components:
  - source:
      type: git
      url: https://github.com/mmakaay/esphome
      ref: fix-dsmr-read-chunk-size
    components: [ "dsmr" ]
    refresh: 60s

If this proves to be the solution, I'll submit a PR to get it updated.

Just used your code and got results!

[E][dsmr:154]: !448F
 ^
Checksum mismatch�
[D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 2585.76392 kWh with 3 decimals of accuracy�
[D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 2434.96802 kWh with 3 decimals of accuracy�
[D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy�
[D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy�
[D][sensor:113]: 'Power Consumed': Sending state 0.23900 kW with 3 decimals of accuracy�
[D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 3 decimals of accuracy�
[E][dsmr:154]: !A5EB
 ^
Checksum mismatch�
[E][dsmr:154]: !4097
 ^
Checksum mismatch�
[E][dsmr:154]: !5CEF
 ^
Checksum mismatch�
[D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 2585.76392 kWh with 3 decimals of accuracy�
[D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 2434.96802 kWh with 3 decimals of accuracy�
[D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy�
[D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy�
[D][sensor:113]: 'Power Consumed': Sending state 0.24400 kW with 3 decimals of accuracy�
[D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 3 decimals of accuracy�
[D][sensor:113]: 'Electricity Failures': Sending state 7.00000  with 0 decimals of accurac

After a while the checksum messages only occur after the DSM version message:

[D][text_sensor:067]: 'DSMR Version': Sending state '50'�
[E][dsmr:154]: !FDC4
 ^
Checksum mismatch�
[E][dsmr:154]: !E16B
 ^
Checksum mismatch�
[E][dsmr:154]: !85DA
 ^
Checksum mismatch�
[D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 2585.76392 kWh with 3 decimals of accuracy�

@mmakaay
Copy link
Member

mmakaay commented Oct 22, 2021

Results, but also still some checksum mismatches. That is a bit weird, because the max telegram size is 1500, so there should be at most 3 loop() calls to get a complete telegram. I wouldn't expect those loop calls to be that much to be interfering with the data processing.

Still looking into things, so I'll get back on this when I find something interesting. Thanks for the test @svenahrensnl

@mmakaay
Copy link
Member

mmakaay commented Oct 22, 2021

BTW, in the latest output from @svenahrensnl , it looks like the device is still busy sending data, while already new data are coming in. Not fully sure how that works.

I'm currently testing with a VERBOSE log level and with a display attached to the device. The display currently takes about 220ms for each screen update (way longer than the recommended 20-30ms). With that up and running, I do get both working and failing updates. It looks a bit like it's a wave where I get a sequence of working readings, followed by a sequence of failing ones. I'll have to investigate a bit more on that to see if there's an actual pattern going on.

For debugging, I added some extra log lines for notable events in the read loop, and one interesting thing that I saw is:

image

One of the things I log is when the read loop returns because available() returns false. As you can see, my DSMR seems to be pushing out its data in chunks, causing the read loop to have chunking behavior, even without my modification to limit the max number of bytes per loop cycle.

I also tried disabling the chunking by setting the max bytes to read per loop cycle to the max telegram size. With that enabled, I still would get some CRC errors over time. Given the log output from above, it's not a big surprise, since chunking is going on anyway. There's shouldn't be big differences in behavior.

These last tests make me wonder: is it indeed the extra added chunking, or is something else going on here?

@mmakaay
Copy link
Member

mmakaay commented Oct 23, 2021

Alright, I implemented a new mechanism, now I've seen that a smart meter might be sending its data in chunked form. When a header is seen, but when no serial data are available for reading, the code now doesn't return from the read loop as before, but instead it waits for max 200ms for additional data to come in (using delay(...) to keep the watch dog happy).
By waiting at this point, instead of returning, the dsmr component will read more data as soon as it gets available on the input, and other components cannot interfere with the reading process, causing bytes to get lost.

With this code, verbose debugging and a display that takes 220ms to update, I now get stable readings. I would be very interested to see what this new version does for DSMR5 users. To test, add the following to your YAML file:

external_components:
  - source:
      type: git
      url: https://github.com/mmakaay/esphome
      ref: fix-dsmr-read-chunk-size
    components: [ "dsmr" ]
    refresh: 60s

Related issue: missing the header of a telegram

I have identified one case in which data can still be lost. When my display is updating, while DSMR data are coming in, the starting bytes might already have dropped from the buffer by the time the dsmr component gets a chance to read them. However, this is a different case and it does not produce CRC errors. Instead, the data are simply ignored, because the header is never seen.

For this issue, the solutions are to either make sure there are no components that block the loop for too long, or increasing the rx_buffer_size so it can hold at least a full telegram.

Here's a bit of logging that shows this case in action:

image

You see ESPHome complaining that the display updates are taking too much time, and it's clear that the header of the telegram has become collateral damage here.

@mmakaay
Copy link
Member

mmakaay commented Oct 23, 2021

I got one report from a DSMR5 user, who uses all kinds of device status sensors and a display. He reported stable readings for 8 minutes, which I find very hopeful 😀

@petergebruers
Copy link

@mmakaay thank you for your elaborate analysis and detailed reports. Very much appreciated! I guess what you say, You see ESPHome complaining that the display updates are taking too much time, and it's clear that the header of the telegram has become collateral damage here. is crucial but I have found no easy way to "trace" the execution of components and functions. I am very new to ESPHome so that might be ignorance. I can read/write cpp but that does not mean I understand how it all fits together. Unfortunately I cannot test your branch right now due to priorities.

@petergebruers
Copy link

Just a quick check, this is intentional on https://github.com/mmakaay/esphome/blob/fix-dsmr-read-chunk-size/esphome/components/dsmr/dsmr.h ?

static constexpr uint32_t MAX_BYTES_PER_LOOP = 1024;

Wasn't that meant to be 60?

@mmakaay
Copy link
Member

mmakaay commented Oct 23, 2021

Never mind that. The implementation is different now and that variable is not in use anymore.

@jsuanet
Copy link

jsuanet commented Oct 23, 2021

I got one report from a DSMR5 user, who uses all kinds of device status sensors and a display. He reported stable readings for 8 minutes, which I find very hopeful 😀

And a couple hours later still no checksum errors 😀

@petergebruers
Copy link

petergebruers commented Oct 23, 2021

Very hopeful indeed, guys. Great work! Upgraded my ESP8266 on my DSMR5 meter 5 minutes ago, with default RX buffer (256) and commit 39b41818891 of branch of @mmakaay. LGTM! Have to go now, will check again in a few hours. 👍

@michaelarnauts
Copy link

Is there a downside of using a bigger RX buffer?

@mmakaay
Copy link
Member

mmakaay commented Oct 23, 2021

I don't see why a bigger RX buffer would be a problem, so I think I will add that to my own configuration as well. It will help with the issue of losing bytes at the start of a telegram, that I described above.

At the same time, I think it would be best if the readings were solid, even with the default RX buffer of 256 bytes. So that's what I'm preparing the patch for currently. For me the readings on DSMR4 are rock solid now, and others with DSMR5 are reporting good results too. So I'll add some comments explaining the new read logic, and submit a PR for it.

@mmakaay
Copy link
Member

mmakaay commented Oct 23, 2021

I have found no easy way to "trace" the execution of components and functions. I am very new to ESPHome so that might be ignorance.

If you're ignorant there, then I'm ignorant as well @petergebruers 😄

I don't think there's any analysis like that built into the core, so it's not easy to get a comparison between versions to see where time is lost. Would be good so see if some other component is taking more time with 2021.10.0, causing more of these CRC issues though, but for that I'll have to do some dirty hacking on the code base. My checking patch was a suspect before, but given the fact that there already was some form of "natural chunking" going on, by now I don't think it's the core issue here.

I did just send in a PR for the new code, so let's see what the core developers think about it.
The patch does show really good results for me, so quite happy now 👍

@defl
Copy link

defl commented Oct 30, 2021

I can confirm above issues as well (DSMR5). Increasing the UART buffer to 1024 worked as a fix for me using dsmr 0.5.0

@denevecon
Copy link

denevecon commented Oct 31, 2021

Thanks to your investigations and solution I got my update to 2020.10.3 running again. Still some Checksum mismatch errors between sensor updates. The annoying side effect is that OTA fails. See below:

[17:14:53][C][api:135]: Address: slimmelezer.local:6053
[17:14:53][C][api:139]: Using noise encryption: NO
[17:14:53][C][wifi_signal.sensor:009]: WiFi Signal 'Wi-Fi Signal'
[17:14:53][C][wifi_signal.sensor:009]: Device Class: 'signal_strength'
[17:14:53][C][wifi_signal.sensor:009]: State Class: 'measurement'
[17:14:53][C][wifi_signal.sensor:009]: Unit of Measurement: 'dBm'
[17:14:53][C][wifi_signal.sensor:009]: Accuracy Decimals: 0
[17:14:53][C][wifi_info:009]: WifiInfo IPAddress 'IP Address'
[17:14:53][C][wifi_info:011]: WifiInfo SSID 'Wi-Fi SSID'
[17:14:53][C][wifi_info:012]: WifiInfo BSSID 'Wi-Fi BSSID'
[17:14:54][E][dsmr:168]: !68E0
^
Checksum mismatch
[17:14:55][E][dsmr:168]: !6FD5
^
Checksum mismatch
[17:14:56][E][dsmr:168]: !D225
^
Checksum mismatch
[17:14:57][E][dsmr:168]: !A0D5
^
Checksum mismatch
[17:14:59][E][dsmr:168]: !8CDF
^
Checksum mismatch
[17:15:00][E][dsmr:168]: !F33F
^
Checksum mismatch
[17:15:01][E][dsmr:168]: !9F96
^
Checksum mismatch
[17:15:02][E][dsmr:168]: !B6DA
^
Checksum mismatch
[17:15:03][E][dsmr:168]: !0206
^
Checksum mismatch
[17:15:04][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.44434 kWh with 3 decimals of accuracy
[17:15:04][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:04][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:04][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:04][D][sensor:113]: 'Power Consumed': Sending state 898.00000 kW with 0 decimals of accuracy
[17:15:04][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:04][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:04][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:04][D][sensor:113]: 'Voltage Phase 1': Sending state 233.10001 V with 1 decimals of accuracy
[17:15:04][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:04][D][sensor:113]: 'Power Consumed Phase 1': Sending state 898.00000 kW with 0 decimals of accuracy
[17:15:04][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:04][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:04][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:04][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:06][E][dsmr:168]: !4C21
^
Checksum mismatch
[17:15:07][E][dsmr:168]: !78E3
^
Checksum mismatch
[17:15:08][E][dsmr:168]: !1589
^
Checksum mismatch
[17:15:09][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.44531 kWh with 3 decimals of accuracy
[17:15:09][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:09][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:09][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:09][D][sensor:113]: 'Power Consumed': Sending state 887.00000 kW with 0 decimals of accuracy
[17:15:09][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:09][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:09][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:09][D][sensor:113]: 'Voltage Phase 1': Sending state 233.00000 V with 1 decimals of accuracy
[17:15:09][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:09][D][sensor:113]: 'Power Consumed Phase 1': Sending state 910.00000 kW with 0 decimals of accuracy
[17:15:09][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:09][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:09][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:09][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:11][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.44629 kWh with 3 decimals of accuracy
[17:15:11][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:11][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:11][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:11][D][sensor:113]: 'Power Consumed': Sending state 916.00000 kW with 0 decimals of accuracy
[17:15:11][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:11][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:11][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:11][D][sensor:113]: 'Voltage Phase 1': Sending state 233.00000 V with 1 decimals of accuracy
[17:15:11][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:11][D][sensor:113]: 'Power Consumed Phase 1': Sending state 907.00000 kW with 0 decimals of accuracy
[17:15:11][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:11][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:11][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:11][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:13][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.44629 kWh with 3 decimals of accuracy
[17:15:13][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:13][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:13][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:13][D][sensor:113]: 'Power Consumed': Sending state 919.00000 kW with 0 decimals of accuracy
[17:15:13][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:13][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:13][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:13][D][sensor:113]: 'Voltage Phase 1': Sending state 233.10001 V with 1 decimals of accuracy
[17:15:13][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:13][D][sensor:113]: 'Power Consumed Phase 1': Sending state 933.00000 kW with 0 decimals of accuracy
[17:15:13][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:13][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:13][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:13][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:15][E][dsmr:168]: !D75D
^
Checksum mismatch
[17:15:16][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.44727 kWh with 3 decimals of accuracy
[17:15:16][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:16][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:16][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:16][D][sensor:113]: 'Power Consumed': Sending state 896.00000 kW with 0 decimals of accuracy
[17:15:16][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:16][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:16][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:16][D][sensor:113]: 'Voltage Phase 1': Sending state 232.89999 V with 1 decimals of accuracy
[17:15:16][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:16][D][sensor:113]: 'Power Consumed Phase 1': Sending state 908.00000 kW with 0 decimals of accuracy
[17:15:16][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:16][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:16][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:16][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:18][E][dsmr:168]: !A25C
^
Checksum mismatch
[17:15:19][E][dsmr:168]: !E64C
^
Checksum mismatch
[17:15:20][E][dsmr:168]: !0CF7
^
Checksum mismatch
[17:15:21][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.44824 kWh with 3 decimals of accuracy
[17:15:21][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:21][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:21][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:21][D][sensor:113]: 'Power Consumed': Sending state 896.00000 kW with 0 decimals of accuracy
[17:15:21][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:21][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:21][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:21][D][sensor:113]: 'Voltage Phase 1': Sending state 233.10001 V with 1 decimals of accuracy
[17:15:21][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:21][D][sensor:113]: 'Power Consumed Phase 1': Sending state 910.00000 kW with 0 decimals of accuracy
[17:15:21][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:21][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:21][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:21][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:23][E][dsmr:168]: !9F12
^
Checksum mismatch
[17:15:24][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.44922 kWh with 3 decimals of accuracy
[17:15:24][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:24][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:24][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:24][D][sensor:113]: 'Power Consumed': Sending state 923.00000 kW with 0 decimals of accuracy
[17:15:24][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:24][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:24][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:24][D][sensor:113]: 'Voltage Phase 1': Sending state 233.10001 V with 1 decimals of accuracy
[17:15:24][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:24][D][sensor:113]: 'Power Consumed Phase 1': Sending state 932.00000 kW with 0 decimals of accuracy
[17:15:24][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:24][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:24][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:24][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:26][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.45020 kWh with 3 decimals of accuracy
[17:15:26][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:26][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:26][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:26][D][sensor:113]: 'Power Consumed': Sending state 941.00000 kW with 0 decimals of accuracy
[17:15:26][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:26][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:26][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:26][D][sensor:113]: 'Voltage Phase 1': Sending state 233.20000 V with 1 decimals of accuracy
[17:15:26][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:26][D][sensor:113]: 'Power Consumed Phase 1': Sending state 941.00000 kW with 0 decimals of accuracy
[17:15:26][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:26][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:26][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:26][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:29][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.45020 kWh with 3 decimals of accuracy
[17:15:29][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:29][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:29][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:29][D][sensor:113]: 'Power Consumed': Sending state 937.00000 kW with 0 decimals of accuracy
[17:15:29][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:29][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:29][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:29][D][sensor:113]: 'Voltage Phase 1': Sending state 233.10001 V with 1 decimals of accuracy
[17:15:29][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:29][D][sensor:113]: 'Power Consumed Phase 1': Sending state 919.00000 kW with 0 decimals of accuracy
[17:15:29][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:29][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:29][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:29][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:31][E][dsmr:168]: !B3D3
^
Checksum mismatch
[17:15:32][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.45117 kWh with 3 decimals of accuracy
[17:15:32][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:32][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:32][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:32][D][sensor:113]: 'Power Consumed': Sending state 913.00000 kW with 0 decimals of accuracy
[17:15:32][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:32][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:32][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:32][D][sensor:113]: 'Voltage Phase 1': Sending state 232.80000 V with 1 decimals of accuracy
[17:15:32][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:32][D][sensor:113]: 'Power Consumed Phase 1': Sending state 918.00000 kW with 0 decimals of accuracy
[17:15:32][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:32][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:32][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:32][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:34][E][dsmr:168]: !9A2A
^
Checksum mismatch
[17:15:35][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.45215 kWh with 3 decimals of accuracy
[17:15:35][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:35][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:35][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:35][D][sensor:113]: 'Power Consumed': Sending state 932.00000 kW with 0 decimals of accuracy
[17:15:35][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:35][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:35][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:35][D][sensor:113]: 'Voltage Phase 1': Sending state 232.80000 V with 1 decimals of accuracy
[17:15:35][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:35][D][sensor:113]: 'Power Consumed Phase 1': Sending state 932.00000 kW with 0 decimals of accuracy
[17:15:35][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:35][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:35][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:35][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:36][D][sensor:113]: 'Uptime': Sending state 1779.23303 s with 0 decimals of accuracy
[17:15:38][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.45312 kWh with 3 decimals of accuracy
[17:15:38][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:38][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:38][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:38][D][sensor:113]: 'Power Consumed': Sending state 919.00000 kW with 0 decimals of accuracy
[17:15:38][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:38][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:38][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:38][D][sensor:113]: 'Voltage Phase 1': Sending state 232.30000 V with 1 decimals of accuracy
[17:15:38][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:38][D][sensor:113]: 'Power Consumed Phase 1': Sending state 919.00000 kW with 0 decimals of accuracy
[17:15:38][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:38][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:38][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:38][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:40][E][dsmr:168]: !A6D9
^
Checksum mismatch
[17:15:41][E][dsmr:168]: !7C8F
^
Checksum mismatch
[17:15:42][E][dsmr:168]: !97AF
^
Checksum mismatch
[17:15:43][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.45410 kWh with 3 decimals of accuracy
[17:15:43][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:43][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:43][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:43][D][sensor:113]: 'Power Consumed': Sending state 837.00000 kW with 0 decimals of accuracy
[17:15:43][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:43][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:43][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:43][D][sensor:113]: 'Voltage Phase 1': Sending state 232.39999 V with 1 decimals of accuracy
[17:15:43][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:43][D][sensor:113]: 'Power Consumed Phase 1': Sending state 812.00000 kW with 0 decimals of accuracy
[17:15:43][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:43][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:43][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:43][D][text_sensor:067]: 'DSMR Version': Sending state '50'
[17:15:45][E][dsmr:168]: !78C2
^
Checksum mismatch
[17:15:45][D][sensor:113]: 'Wi-Fi Signal': Sending state -34.00000 dBm with 0 decimals of accuracy
[17:15:47][E][dsmr:168]: !FAEF
^
Checksum mismatch
[17:15:48][E][dsmr:168]: !8916
^
Checksum mismatch
[17:15:49][E][dsmr:168]: !DB0E
^
Checksum mismatch
[17:15:50][E][dsmr:168]: !1757
^
Checksum mismatch
[17:15:51][E][dsmr:168]: !8082
^
Checksum mismatch
[17:15:52][E][dsmr:168]: !BA22
^
Checksum mismatch
[17:15:53][E][dsmr:168]: !E8D9
^
Checksum mismatch
[17:15:54][D][sensor:113]: 'Energy Consumed Tariff 1': Sending state 9534.45703 kWh with 3 decimals of accuracy
[17:15:54][D][sensor:113]: 'Energy Consumed Tariff 2': Sending state 10324.56836 kWh with 3 decimals of accuracy
[17:15:54][D][sensor:113]: 'Energy Produced Tariff 1': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:54][D][sensor:113]: 'Energy Produced Tariff 2': Sending state 0.00000 kWh with 3 decimals of accuracy
[17:15:54][D][sensor:113]: 'Power Consumed': Sending state 885.00000 kW with 0 decimals of accuracy
[17:15:54][D][sensor:113]: 'Power Produced': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:54][D][sensor:113]: 'Electricity Failures': Sending state 8.00000 with 0 decimals of accuracy
[17:15:54][D][sensor:113]: 'Long Electricity Failures': Sending state 2.00000 with 0 decimals of accuracy
[17:15:54][D][sensor:113]: 'Voltage Phase 1': Sending state 232.50000 V with 1 decimals of accuracy
[17:15:54][D][sensor:113]: 'Current Phase 1': Sending state 4.00000 A with 1 decimals of accuracy
[17:15:54][D][sensor:113]: 'Power Consumed Phase 1': Sending state 879.00000 kW with 0 decimals of accuracy
[17:15:54][D][sensor:113]: 'Power Produced Phase 1': Sending state 0.00000 kW with 0 decimals of accuracy
[17:15:54][D][sensor:113]: 'Gas Consumed': Sending state 5743.83301 m³ with 3 decimals of accuracy
[17:15:54][D][text_sensor:067]: 'DSMR Identification': Sending state 'ISK5\2M550E-1012'
[17:15:54][D][text_sensor:067]: 'DSMR Version': Sending state '50'

@mmakaay
Copy link
Member

mmakaay commented Oct 31, 2021

The main reason that keeps popping up for errors like this, after incorporating the fixes (rx_buffer_size increase + my patched code) is that the UART is not hardware-based but software-based.
What does your YAML configuration look like?

Two things that might disturb the use of hardware UART:

  • Not having baud_rate: 0 for the logger component
  • Using an RX pin that is not hardware-UART capable. E.g. on ESP8266 the pin ought to be GPIO13 (i.e. D7 on a D1 mini)

A quick way to check is connecting to the logger, and inspecting the setup information for the UART component. The information will show hardware or software UART.

@denevecon
Copy link

I misinterpreted the instructions and commented the rx_buffer_size increase. Changing this and the baud_rate for the logger resulted in an error-less log output. Btw the UART is hardware based. Thanks again.

@patvdleer
Copy link

patvdleer commented Oct 31, 2021

I'm running ESPHOME 2021.10.3 with the zuidwijk 1.1.1v on a Kaifa E0025 with DSMI 4.2, sadly setting the rx_buffer_size did not fix it. Even with the crc_check set to false I still get the Checksum mismatch errors.

uart:
  baud_rate: 115200
  rx_pin: D7
  rx_buffer_size: 1024

dsmr:
  id: dsmr_instance
  crc_check: false

I forgot the logger, now it does work, even with the crc check enabled.

logger:
  baud_rate: 0

@mmakaay
Copy link
Member

mmakaay commented Dec 10, 2021

Various fixes have been done to the DSMR component and these likely have fixed the reported issues.
All fixes are in the upcoming version 2021.12.0 of ESPHome. If anybody still has problems with that version, then please create a new issue for it, with information about the log output, uart debugger and meter type.

@mmakaay mmakaay closed this as completed Dec 10, 2021
@zuidwijk
Copy link

Hey @mmakaay ... I've compiled a firmware on 2021.12.0 and 2021.12.1 and notice some Checksum mismatch errors. Do you still need to add the rx_buffer_size to the uart? And do you still need to add your fix-dsmr-read-chunk-size?

@mmakaay
Copy link
Member

mmakaay commented Dec 23, 2021

No, remove any external component that you have in your config. All fixes have been incorporated.
With my changes, the rx_buffer_size >= max telegram size is not required, though it is recommended. When using tn rx_buffer_size that matches the telegram size, the code won't have to do any forced filling of the telegram buffer anymore.

@zuidwijk
Copy link

With and without your external code, and with or without rx_buffer_size, I keep getting those error messages. Opening new case? or continu an existing case?

@mmakaay
Copy link
Member

mmakaay commented Dec 24, 2021

Definitely a new one, the issues from this were fixed and it is unwieldy as is.

@zuidwijk
Copy link

Found the issue, and I don't like it...

I'm using improv_serial: and that requires the logger baud_rate to be not 0.
Disable improv_serial and set baud_rate to 0 works.

@zuidwijk
Copy link

#2873

@github-actions github-actions bot locked and limited conversation to collaborators Apr 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests