New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Checksum mismatch #2577
Comments
I have the same issue. After upgrading to 2021.10 only checksum errors. I've reverted back to 2021.9 and everything works perfect again. |
I see the same issue, I was using the following options on 2021.9, and since 2021.10 uses
I'm using a DSMRv5 meter, so I should not have to use the |
I think the issue is due to this PR: esphome/esphome#2382 |
When using
|
thanks for the crc input. |
I've changed the yaml code https://raw.githubusercontent.com/zuidwijk/dsmr/main/slimmelezer.yaml |
Hey there @glmnet, mind taking a look at this issue as it has been labeled with an integration ( |
@zuidwijk [edit] |
@bkbartk the Wemos is consuming about 180~200mA in "normal" use. When configuring wifi, that can easily got to the max 250mA or a little above and than you get brownouts. I always advice to use usb power during wifi setup. Maybe the wifi usage is too high during transfer of OTA files. |
good to know, |
I see the same typ of issue as @bkbartk and @michaelarnauts: [13:21:05][D][text_sensor:067]: 'DSMR Identification': Sending state 'ELL5\253833635_A' |
Try setting "rx_buffer_size: 1000" under "uart", like this (extract from my actual setup): uart:
id: uart_p1
rx_pin: RX
baud_rate: 115200
# ESP8266 has 128 byte hardware fifo, largest DSMR v5 data payload is 1024 bytes.
# Add a few bytes for the headear and set rx_buffer_size to 1000 is enough to fit
# one telegram per second.
rx_buffer_size: 1000 My guess is you all have meters that run at 115200 bit speed and create one telegram per second, like me, and the "chunking" in PR: esphome/esphome#2382 causes long enough pauses to drop a (few?) bytes... Have been running ESPHome version 2021.10.1 on a D1 mini ESP8266 for 15 minutes without an issue... I left "crc_check: true" under "dsmr" (which is the default). |
Hello Petergebruers, The rx_buffer_size: 1000 saved my day. Regards. |
the rx_buffer_size fixed the issue. thank you all for responding. |
This indeed fixes it. I've set mine to @zuidwijk you'll want to add this |
@jozg you thank for your feedback! @bkbartk you say:
And I can explain that. I am not using any converter board, just a transistor and resistor, connected to pad "RX" marked on my D1 mini board (aka U0RXD). This allows me to use the hardware UART and minimum component count. I did disable the hardware logger by setting "baud_rate: 0" under "logger:" @michaelarnauts - ah you prefer 1024... I understand... Hahaha. |
@petergebruers Thanks a lot, this fixed the problem for me to! |
I created the chunking mechanism to prevent the DSMR component from continuously reading and blocking the device. There were other devices on which things went wrong because of the lack of chunking, even up to occasional reboots, because of watch dog timer issues. But with DMSR5 meters that are sending data every second, apparently reading the data in a more friendly way breaks here. I investigated a telegram that was captured using UART debugging (i.e. an open PR, but long live external components), and there are indeed bytes missing. Instead of increasing the read buffer (which as such is a fine solution in case processing can't keep up with the data as in this case), I think that it's also possible to increase the size of the chunk that is read per Question now is: what would be a good chunk size? Too people with the issue (I don't have a DSMR5 myself), can you please try the following in your code, with the increased external_components:
- source:
type: git
url: https://github.com/mmakaay/esphome
ref: fix-dsmr-read-chunk-size
components: [ "dsmr" ]
refresh: 60s If this proves to be the solution, I'll submit a PR to get it updated. |
Just used your code and got results!
After a while the checksum messages only occur after the DSM version message:
|
Results, but also still some checksum mismatches. That is a bit weird, because the max telegram size is 1500, so there should be at most 3 Still looking into things, so I'll get back on this when I find something interesting. Thanks for the test @svenahrensnl |
BTW, in the latest output from @svenahrensnl , it looks like the device is still busy sending data, while already new data are coming in. Not fully sure how that works. I'm currently testing with a For debugging, I added some extra log lines for notable events in the read loop, and one interesting thing that I saw is: One of the things I log is when the read loop returns because I also tried disabling the chunking by setting the max bytes to read per loop cycle to the max telegram size. With that enabled, I still would get some CRC errors over time. Given the log output from above, it's not a big surprise, since chunking is going on anyway. There's shouldn't be big differences in behavior. These last tests make me wonder: is it indeed the extra added chunking, or is something else going on here? |
Alright, I implemented a new mechanism, now I've seen that a smart meter might be sending its data in chunked form. When a header is seen, but when no serial data are available for reading, the code now doesn't return from the read loop as before, but instead it waits for max 200ms for additional data to come in (using With this code, verbose debugging and a display that takes 220ms to update, I now get stable readings. I would be very interested to see what this new version does for DSMR5 users. To test, add the following to your YAML file: external_components:
- source:
type: git
url: https://github.com/mmakaay/esphome
ref: fix-dsmr-read-chunk-size
components: [ "dsmr" ]
refresh: 60s Related issue: missing the header of a telegram I have identified one case in which data can still be lost. When my display is updating, while DSMR data are coming in, the starting bytes might already have dropped from the buffer by the time the dsmr component gets a chance to read them. However, this is a different case and it does not produce CRC errors. Instead, the data are simply ignored, because the header is never seen. For this issue, the solutions are to either make sure there are no components that block the loop for too long, or increasing the Here's a bit of logging that shows this case in action: You see ESPHome complaining that the display updates are taking too much time, and it's clear that the header of the telegram has become collateral damage here. |
I got one report from a DSMR5 user, who uses all kinds of device status sensors and a display. He reported stable readings for 8 minutes, which I find very hopeful 😀 |
@mmakaay thank you for your elaborate analysis and detailed reports. Very much appreciated! I guess what you say, |
Just a quick check, this is intentional on https://github.com/mmakaay/esphome/blob/fix-dsmr-read-chunk-size/esphome/components/dsmr/dsmr.h ? static constexpr uint32_t MAX_BYTES_PER_LOOP = 1024; Wasn't that meant to be 60? |
Never mind that. The implementation is different now and that variable is not in use anymore. |
And a couple hours later still no checksum errors 😀 |
Very hopeful indeed, guys. Great work! Upgraded my ESP8266 on my DSMR5 meter 5 minutes ago, with default RX buffer (256) and commit 39b41818891 of branch of @mmakaay. LGTM! Have to go now, will check again in a few hours. 👍 |
Is there a downside of using a bigger RX buffer? |
I don't see why a bigger RX buffer would be a problem, so I think I will add that to my own configuration as well. It will help with the issue of losing bytes at the start of a telegram, that I described above. At the same time, I think it would be best if the readings were solid, even with the default RX buffer of 256 bytes. So that's what I'm preparing the patch for currently. For me the readings on DSMR4 are rock solid now, and others with DSMR5 are reporting good results too. So I'll add some comments explaining the new read logic, and submit a PR for it. |
If you're ignorant there, then I'm ignorant as well @petergebruers 😄 I don't think there's any analysis like that built into the core, so it's not easy to get a comparison between versions to see where time is lost. Would be good so see if some other component is taking more time with 2021.10.0, causing more of these CRC issues though, but for that I'll have to do some dirty hacking on the code base. My checking patch was a suspect before, but given the fact that there already was some form of "natural chunking" going on, by now I don't think it's the core issue here. I did just send in a PR for the new code, so let's see what the core developers think about it. |
I can confirm above issues as well (DSMR5). Increasing the UART buffer to 1024 worked as a fix for me using dsmr 0.5.0 |
Thanks to your investigations and solution I got my update to 2020.10.3 running again. Still some Checksum mismatch errors between sensor updates. The annoying side effect is that OTA fails. See below: [17:14:53][C][api:135]: Address: slimmelezer.local:6053 |
The main reason that keeps popping up for errors like this, after incorporating the fixes (rx_buffer_size increase + my patched code) is that the UART is not hardware-based but software-based. Two things that might disturb the use of hardware UART:
A quick way to check is connecting to the logger, and inspecting the setup information for the UART component. The information will show hardware or software UART. |
I misinterpreted the instructions and commented the rx_buffer_size increase. Changing this and the baud_rate for the logger resulted in an error-less log output. Btw the UART is hardware based. Thanks again. |
I'm running uart:
baud_rate: 115200
rx_pin: D7
rx_buffer_size: 1024
dsmr:
id: dsmr_instance
crc_check: false I forgot the logger, now it does work, even with the crc check enabled. logger:
baud_rate: 0 |
Various fixes have been done to the DSMR component and these likely have fixed the reported issues. |
Hey @mmakaay ... I've compiled a firmware on 2021.12.0 and 2021.12.1 and notice some Checksum mismatch errors. Do you still need to add the |
No, remove any external component that you have in your config. All fixes have been incorporated. |
With and without your external code, and with or without rx_buffer_size, I keep getting those error messages. Opening new case? or continu an existing case? |
Definitely a new one, the issues from this were fixed and it is unwieldy as is. |
Found the issue, and I don't like it... I'm using |
The problem
After installing ESPHome and update the device via home assistant I only get checksum mismatches
`[09:04:00][C][wifi_signal.sensor:009]: WiFi Signal 'Wi-Fi Signal'
[09:04:00][C][wifi_signal.sensor:009]: Device Class: 'signal_strength'
[09:04:00][C][wifi_signal.sensor:009]: State Class: 'measurement'
[09:04:00][C][wifi_signal.sensor:009]: Unit of Measurement: 'dBm'
[09:04:00][C][wifi_signal.sensor:009]: Accuracy Decimals: 0
[09:04:00][C][wifi_info:009]: WifiInfo IPAddress 'IP Address'
[09:04:00][C][wifi_info:011]: WifiInfo SSID 'Wi-Fi SSID'
[09:04:00][C][wifi_info:012]: WifiInfo BSSID 'Wi-Fi BSSID'
[09:04:00][E][dsmr:154]: !4119
^
Checksum mismatch
[09:04:01][E][dsmr:154]: !4023
^
Checksum mismatch
[09:04:02][E][dsmr:154]: !5906
^
Checksum mismatch`
Which version of ESPHome has the issue?
2021.10.0
What type of installation are you using?
Home Assistant Add-on
Which version of Home Assistant has the issue?
core-2021.9.7
What platform are you using?
ESP8266
Board
slimmelezer
Component causing the issue
No response
Example YAML snippet
https://raw.githubusercontent.com/zuidwijk/dsmr/main/slimmelezer.yaml
Anything in the logs that might be useful for us?
Additional information
there were no isseus with v 2021.9
The text was updated successfully, but these errors were encountered: