HA (Home Assistant) and umqtt.simple

[shariltumin]

I'm just sharing my experience.

Consider this test script test_1.py

temp_s = "24 °C"
temp_b = b"24 °C"
temp_e = temp_s.encode("utf-8")

print(temp_s, temp_b, temp_e)
print(len(temp_s), len(temp_b), len(temp_e))
print(temp_s == temp_b.decode("utf-8"), temp_s == temp_e.decode("utf-8"))

Running it on an ESP32

$ mpr u0 run test_1.py
24 °C b'24 \xc2\xb0C' b'24 \xc2\xb0C'
5 6 6
True True

Notice the difference in the len values.

This gave me problems when trying to register an "MQTT device" with my HA (Home Assistant) server using HA discovery by publishing its configuration in a json.dumps(config) string. The config has { "unit_of_measurement": "°C", } among other attributes.

The publish method in the MQTTClient of the umqtt.simple library has these intial lines of code:

    def publish(self, topic, msg, retain=False, qos=0):
        pkt = bytearray(b"\x30\0\0\0")
        pkt[0] |= qos << 1 | retain
        sz = 2 + len(topic) + len(msg)
        ...

If the msg is a string as exemplified above the calculated sz will be incorrect.

I know, the example_pub.py in micropython/lib/micropython-lib/micropython/umqtt.simpl shows (to those who are willing to read it) that one should use bytes values:

    c.publish(b"foo_topic", b"hello")

for both topic and msg, but it is so easy to overlook.

I suggest we bring the conversion in the method like so:

    def publish(self, topic, msg, retain=False, qos=0):
        if isinstance(topic, str):
           topic = topic.encode('utf-8')
        if isinstance(msg, str):
           msg = msg.encode('utf-8')
        pkt = bytearray(b"\x30\0\0\0")
        pkt[0] |= qos << 1 | retain
        sz = 2 + len(topic) + len(msg)
        ...

with the above fix in place, I can confidently do:

device_info = {
    "identifiers": ["esp32_sharil_v9"],
    "name": "Sharil ESP32 Node",
    "model": "ESP32",
    "manufacturer": "Espressif"
}

sensor_config = {
    "name": "Temperature",
    "unique_id": "sharil_temp_v9",
    "state_topic": "sharil/temp/state",
    "value_template": "{{ value | float }}",
    "unit_of_measurement": "°C",
    "device_class": "temperature",
    "state_class": "measurement",
    "device": device_info
}

# Sensor path matching the device node_id
client.publish("homeassistant/sensor/esp32_sharil_v9/sharil_temp/config", json.dumps(sensor_config), retain=True)

It took me some time to identify the problem. I have now managed to register my sensor with my HA server.

[Josverl]

Before 1.29.0 the code would have crashed, but now Unicode is available I agree that the modules will need to take this into account. Your proposal seem reasonable

The doc change to only use bytes then can be updated to refer only to older versions

[sosi-deadeye]

One workaround is to provide already encoded bytes. Instead of encoding a str to bytes during runtime, you send the bytes directly.

from umqtt.simple import MQTTClient


client = MQTTClient("Test-1", "127.0.0.1")
client.connect()

TOPIC = b'\xc3\x9cbererde:'
DATA_FMT = b"%.2f \xc2\xb0C"
# old style c formatting
# In CPython the format method does not exist for str
# Mandela effect?

# opening client connection etc...
# then publish

client.publish(TOPIC, DATA_FMT % 12.42)

Output:
Übererde: 12.42 °C

Too cold.