Unknown Data length

[quam426]

Hello,

When reading out my smart meter I get the following output:

09:53:11.750Z  INFO app: Listening to device: /dev/ttyUSB0
09:53:12.370Z DEBUG app: Received data 1: db08534147350000b3208201f2300000003e30b724f3c9d19fd1b584c894bb6b97755a957a66b3fd (truncated) / size: 200
09:53:12.388Z  WARN app: Unknown data length (100), ignoring data
09:53:12.656Z DEBUG app: Received data 2: 610ffab8bec94421077c3fffab10f850655722eadac1c17cbe45caaf48e755052fa4ceaa70edba0d (truncated) / size: 400
09:53:12.667Z  WARN app: Unknown data length (200), ignoring data
09:53:12.680Z DEBUG app: Received data 3: b1f1545ddfe11e57e71b9ad2b198d571c6e07fb949e5e95f109ee0c35a7564bfb3e48b9c8f25c981 (truncated) / size: 422
09:53:12.696Z  WARN app: Unknown data length (211), ignoring data
09:53:22.329Z DEBUG app: Received data 4: db08534147350000b3208201f2300000003f1da427e3da162d31ba5095e4f489659bded7ea32b9cf (truncated) / size: 200
09:53:22.345Z  WARN app: Unknown data length (100), ignoring data
09:53:22.351Z DEBUG app: Received data 5: 4a1a1ee4780c825c059b0048812b222f1dfe1c682f0a9c0ecccb190092  / size: 58
09:53:22.368Z  WARN app: Unknown data length (29), ignoring data
09:53:22.374Z DEBUG app: Received data 6: 020c60af702fbca68c0914fba89d742972d9efee2ffdf936c35e639b50547de0839c92ccf9949b89 (truncated) / size: 238
09:53:22.388Z  WARN app: Unknown data length (119), ignoring data
09:53:22.401Z DEBUG app: Received data 7: 3903900345f924f197ce2531485c6df56c5592114f7bce370b84ce920a8146c304dc5f5320c10e23 (truncated) / size: 104
09:53:22.414Z  WARN app: Unknown data length (52), ignoring data
09:53:22.427Z DEBUG app: Received data 8: 4662e133f8e03cc4d8883439ce0c9cb334b20fb0b64f118e686d4ca55d5286e0eec3646d897f18f6 (truncated) / size: 200
09:53:22.437Z  WARN app: Unknown data length (100), ignoring data
09:53:22.471Z DEBUG app: Received data 9: 77cea08f0334a186d2f601cc9ed04837fe7a65db62ffd4dd994008dfe8f7d9da75cc1d5c2f679e1e (truncated) / size: 118
09:53:22.483Z  WARN app: Unknown data length (59), ignoring data
09:53:22.494Z DEBUG app: Received data 10: 4fbc80f7a01867a447cd42ada39ac1af1b192752629c39e6585e92b99a8ad9c1a0d9e7e06984cc8c (truncated) / size: 82
09:53:22.507Z  WARN app: Unknown data length (41), ignoring data
09:53:22.517Z DEBUG app: Received data 11: ff75a0cf3e873047d96680  / size: 22
09:53:22.534Z  WARN app: Unknown data length (11), ignoring data
09:53:32.300Z DEBUG app: Received data 12: db08534147350000b3208201f230000000400f24846b  / size: 44
09:53:32.309Z  WARN app: Unknown data length (22), ignoring data
09:53:32.329Z DEBUG app: Received data 13: 986f10abfdca9a8844a4668160467193b6836e9094d36424b3287389316a1bc250dc87c22f505090 (truncated) / size: 156
09:53:32.346Z  WARN app: Unknown data length (78), ignoring data
09:53:32.366Z DEBUG app: Received data 14: 1e43a2bba44d6ca1d3d0f1fb1c10b1811411df0e11295097aa0cb70dd512a298e18d30ed5fbcfa18 (truncated) / size: 112
09:53:32.369Z  WARN app: Unknown data length (56), ignoring data
09:53:32.388Z DEBUG app: Received data 15: 27ab799c8c91a02333b6a414f862d9a0e191e596b433df229b2513c99ed0af05822b1862e3181696 (truncated) / size: 148
09:53:32.391Z  WARN app: Unknown data length (74), ignoring data
09:53:32.413Z DEBUG app: Received data 16: 676e42f294d912ef1899041fe58207e14db0e47f1ce09a9fde721e1db2c19657dd95ad2c3a8ff705 (truncated) / size: 140
09:53:32.429Z  WARN app: Unknown data length (70), ignoring data
09:53:32.449Z DEBUG app: Received data 17: 354ef4b29feafdabc911629f6ef4469ec48a09940ec1c6c96671786be3ec743393254ca094f59301 (truncated) / size: 200
09:53:32.468Z  WARN app: Unknown data length (100), ignoring data
09:53:32.494Z DEBUG app: Received data 18: e5db34e4f3251bac0e83f4067c28de8ebf22e9c697c4068d1a7ccce1bf330dde429c31f30062c72e (truncated) / size: 86
09:53:32.517Z  WARN app: Unknown data length (43), ignoring data
09:53:32.523Z DEBUG app: Received data 19: 9233220fe00f50319475568a017ca3e7652296ef72317feb6c98ef223855f65d0443dbd507a7706f (truncated) / size: 136
09:53:32.544Z  WARN app: Unknown data length (68), ignoring data

When I dump the input from the serial device I see something similar to the datagram you have in the documentation (I see the start bytes of a packet, the identifier, several separator bytes, etc.) - see attached file.

Seems that the datagrams are received in chunks of different size and not correctly re-assembled?

[rafche]

same here

[quam426]

same here

you can give my fork a try (https://github.com/quam426/sage) - I rewrote the whole messege handling part. and if you are interested I also use another backend storage - influxdb instead of mongodb. This is IMO more suitable and efficient, and you can use e.g. Grafana for nice visualizations.

best regards,
markus

[schopenhauer]

@quam426, nice work! Happy to add you as a contributor and merge your fork, in case you are interested to join forces.