log error UnicodeDecodeError

[Anto79-ops]

What happened?

Hello,

Seeing this in AD addon on Home Assistant OS (ver: 0.16.6 and 0.16.7 ) logs on HAOS.

thanks

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

2024-09-18 10:43:05.065189 INFO MQTT: Unable to decode MQTT message
2024-09-18 10:43:05.065910 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

2024-09-18 10:47:01.173652 INFO MQTT: Unable to decode MQTT message
2024-09-18 10:47:01.174345 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

2024-09-18 10:50:53.357415 INFO MQTT: Unable to decode MQTT message
2024-09-18 10:50:53.358181 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

2024-09-18 10:54:47.955705 INFO MQTT: Unable to decode MQTT message
2024-09-18 10:54:47.956193 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

2024-09-18 10:55:04.333653 INFO MQTT: Unable to decode MQTT message
2024-09-18 10:55:04.334190 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

2024-09-18 11:00:23.826721 INFO MQTT: Unable to decode MQTT message
2024-09-18 11:00:23.827232 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

2024-09-18 11:01:04.448018 INFO MQTT: Unable to decode MQTT message
2024-09-18 11:01:04.448690 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

2024-09-18 11:05:19.438034 INFO MQTT: Unable to decode MQTT message
2024-09-18 11:05:19.438766 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

2024-09-18 11:09:17.908246 INFO MQTT: Unable to decode MQTT message
2024-09-18 11:09:17.908974 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

More information here hassio-addons/app-appdaemon#350

Version

0.16.7

Installation type

Home Assistant add-on

Relevant log output

No response

Relevant code in the app or config file that caused the issue

No response

Anything else?

No response

[anandv85]

I'm also seeing the similar issue. This is continuously logging once every 10 seconds. Please let me know if anyone need more information to debug this issue. Thanks!

2024-09-28 20:16:31.285018 INFO AppDaemon: New client Admin Client connected
2024-09-28 20:16:52.383142 INFO MQTT: Unable to decode MQTT message
2024-09-28 20:16:52.384120 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

2024-09-28 20:16:52.469429 INFO MQTT: Unable to decode MQTT message
2024-09-28 20:16:52.469850 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

2024-09-28 20:17:11.707325 INFO AppDaemon: Client disconnection from Admin Client
2024-09-28 20:17:22.250166 INFO MQTT: Unable to decode MQTT message
2024-09-28 20:17:22.251115 ERROR MQTT: Unable to decode MQTT message, with Traceback: Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/appdaemon/plugins/mqtt/mqttplugin.py", line 271, in mqtt_on_message
    payload = payload.decode()
              ^^^^^^^^^^^^^^^^

[Anto79-ops]

Hi @hacscred

sorry, im not sure I quite understand (probably because im just a user of AD). I'm currently using AD as an Addon in Home Assistant, and its a supported addon.

Not sure how I can use it as a standalone, you mean in docker for example?

thanks

[Anto79-ops]

still happening in my logs.

[OptimusGREEN]

same issue here too.

[Anto79-ops]

@OptimusGREEN hopefully a new release is coming soon, from what I hear is in the works.

[acockburn]

On the face of it this doesn't look like anything specific to the HASS environment, but this is basically an issue decoding payload which could be literally anything. Some details as to what you are actually doing with MQTT to cause this issue would be helpful.

[OptimusGREEN]

The log doesn't state which topics are related to the issue so it is somewhat impossible to tell. It would be good if when the issue arises that the mqtt module mentioned the naughty topic in the log.

[Anto79-ops]

thanks @acockburn

using this, in my case: https://github.com/xaf/qolsysgw

[acockburn]

OK, we are buried in the current release and we won't get to a resolution of this issue before we get that out but lets see if we can add some additional debugging in this area which shouldn't be too big a deal so that we can start to track it down in a future release (plan is to do more smaller releases more often moving forwards).

[jsl12]

This type of error is caused by an MQTT message having a binary payload, which causes decoding it into utf-8 to fail.

Unfortunately, really fixing this is going to be kind of involved. We have many plans for future work, and redoing some of the MQTT mechanics is in there somewhere.

For the short term though, I'll commit to at least improving the error message so that it shows the offending topic.