DEVICE_TOPIC_SUFFIX config problem

[SiGmAX666]

The problem

The autodiscovery container is not using the topic format I am specifying in the DEVICE_TOPIC_SUFFIX variable. I assume I am simply doing something wrong, but I can't figure it out. In the mean time, I modified the python script and have that working fine in another container (which was turned off for testing).

I am using the following device config in the rtl_433 container, which is publishing as expected:
devices=rtl_433_2/localhost/devices[/type][/model][/subtype]/C[channel:0],events=rtl_433_2/localhost/events

The rtl_433 container posts the following notices when it starts:

MQTT: Publishing device info to MQTT topic "rtl_433_2/localhost/devices[/type][/model][/subtype]/C[channel:0]".
MQTT: Publishing events info to MQTT topic "rtl_433_2/localhost/events".

And resulting published topic:
rtl_433_2/localhost/devices/AmbientWeather-WH31B/C1/temperature_C

In the autodiscovery container config, I am using the following two relevant settings:

RTL_TOPIC: rtl_433_2/+/events
DEVICE_TOPIC_SUFFIX: devices[/type][/model][/subtype]/C[channel:0]

Autodiscovery adds the following to the homeassistant topic. Note the state_topic does not match the config - it entirely ignored what I put in.
Resulting state: rtl_433_2/localhost/devices/AmbientWeather-WH31B/1/232/temperature_C

Full message:

{"device_class": "temperature", "name": "AmbientWeather-WH31B-1-232-T", "unit_of_measurement": "\u00b0C", "value_template": "{{ value|float|round(1) }}", "state_class": "measurement", "state_topic": "rtl_433_2/localhost/devices/AmbientWeather-WH31B/1/232/temperature_C", "unique_id": "AmbientWeather-WH31B-1-232-T", "device": {"identifiers": ["AmbientWeather-WH31B-1-232"], "name": "AmbientWeather-WH31B-1-232", "model": "AmbientWeather-WH31B", "manufacturer": "rtl_433"}}

I hope y'all can see something obvious, this is super annoying and seems like it should be simple... Thanks much!

What addon are you reporting the bug for?

rtl_433_mqtt_autodiscover

What is the addon version?

0.7.0

What type of MQTT Broker are you using?

Other (details in the bug description)

Addon log messages

No response

Additional information

For reference, here is the docker compose file being used:

  rtl433_test2:
    image: hertzg/rtl_433:latest
    restart: unless-stopped
    depends_on:
      - mosquitto
    networks:
      - default
    devices:
      - /dev/bus/usb/001/005
    volumes:
      - ./rtl433:/root
    command:
      - "-Mtime:unix:usec:utc"
      - "-Mbits"
      - "-Mlevel"
      - "-Mprotocol"
      - "-Mstats:2:300"
      - "-Fmqtt://${RTL433_MQTT_HOST}:1883,retain=1,user=${RTL433_MQTT_USER},pass=${RTL433_MQTT_PASS},devices=rtl_433_2/localhost/devices[/type][/model][/subtype]/C[channel:0],events=rtl_433_2/localhost/events"
      - "-Flog"
      - '-f915.05M'
      - '-H30'
      - '-f433.5M'

  rtl433_autodiscovery_test2:
    image: ghcr.io/pbkhrv/rtl_433-hass-addons-rtl_433_mqtt_autodiscovery-aarch64:0.7.0
    platform: linux/arm64
    restart: always
    depends_on:
      - mosquitto
      - rtl433_test2
    networks:
      - default
    environment:
      MQTT_HOST: ${RTL433_MQTT_HOST}
      MQTT_USERNAME: ${RTL433_MQTT_USER}
      MQTT_PASSWORD: ${RTL433_MQTT_PASS}
      MQTT_PORT: 1883
      RTL_TOPIC: rtl_433_2/+/events
      DEVICE_TOPIC_SUFFIX: devices[/type][/model][/subtype]/C[channel:0]
      DISCOVERY_PREFIX: homeassistant
      DISCOVERY_INTERVAL: 600
      LOG_LEVEL: debug
      FORCE_UPDATE: true
      MQTT_RETAIN: true

[deviantintegral]

If you exec into the addon's docker container, do you see the -T argument in the python arguments if you run ps auxww? The relevant code that passes it in is at https://github.com/pbkhrv/rtl_433-hass-addons/blob/main/rtl_433_mqtt_autodiscovery/run.sh#L44-L48C4.

[SiGmAX666]

Ah, interesting, that explains it. No -T. When I look at the container in

$ docker exec -it rtl433_autodiscovery_test2 sh
/ # ps auxww
PID   USER     TIME  COMMAND
    1 root      0:00 /package/admin/s6/command/s6-svscan -d4 -- /run/service
   14 root      0:00 {rc.init} /bin/sh -e /run/s6/basedir/scripts/rc.init top /run.sh
   15 root      0:00 s6-supervise s6-linux-init-shutdownd
   16 root      0:00 /package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B
   24 root      0:00 s6-supervise s6rc-fdholder
   25 root      0:00 s6-supervise s6rc-oneshot-runner
   33 root      0:00 /package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --
   78 root      0:00 bash /usr/bin/bashio /run.sh
   83 root      0:00 python3 -u /rtl_433_mqtt_hass.py -H mosquitto -p 1883 -R rtl_433_2/+/events -D homeassistant -i 600 --debug
   85 root      0:00 sh
   91 root      0:00 ps auxww

When doing a docker inspect, I think that things look fine regarding the suffix (passwords removed):

$ docker inspect --format='{{range .Config.Env}}{{println .}}{{end}}' rtl433_autodiscovery_test2
LOG_LEVEL=debug
MQTT_PORT=1883
MQTT_USERNAME=rtl433
DEVICE_TOPIC_SUFFIX=devices[/type][/model][/subtype]/C[channel:0]
MQTT_RETAIN=true
DISCOVERY_PREFIX=homeassistant
MQTT_PASSWORD=
RTL_TOPIC=rtl_433_2/+/events
MQTT_HOST=mosquitto
DISCOVERY_INTERVAL=600
FORCE_UPDATE=true
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LANG=C.UTF-8
S6_BEHAVIOUR_IF_STAGE2_FAILS=2
S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
S6_CMD_WAIT_FOR_SERVICES=1
S6_SERVICES_READYTIME=50

And I checked the run.sh script for good measure, it includes the section in the linked blob, of course:

  # This is an optional parameter and we don't want to overwrite the defaults
  DEVICE_TOPIC_SUFFIX=$(bashio::config "device_topic_suffix")
  if [ ! -z $DEVICE_TOPIC_SUFFIX ]; then
    OTHER_ARGS="${OTHER_ARGS} -T ${DEVICE_TOPIC_SUFFIX}"
  fi

[deviantintegral]

I had a moment where I thought the issue was using [ vs [[ but that’s not it.

Can you add in a few echo’s on the DEVICE_TOPIC_SUFFIX variable, OTHER_ARGS, and so on? That should at least tell us which line is breaking. My guess is that the call to the config is returning an empty string.

[SiGmAX666]

OK you're on to something here.

I started with adding two echos, the first being at the top of run.sh

#!/usr/bin/with-contenv bashio

echo "DEVICE_TOPIC_SUFFIX: $DEVICE_TOPIC_SUFFIX"
echo "OTHER_ARGS: $OTHER_ARGS"

And right at the end:

echo "Starting rtl_433_mqtt_hass.py..."
echo "DEVICE_TOPIC_SUFFIX: $DEVICE_TOPIC_SUFFIX"
echo "OTHER_ARGS: $OTHER_ARGS"

python3 -u /rtl_433_mqtt_hass.py -H $MQTT_HOST -p $MQTT_PORT -R "$RTL_TOPIC" -D "$DISCOVERY_PREFIX" -i $DISCOVERY_INTERVAL $OTHER_ARGS

When I run the script, it does not include our suffix in the OTHER_ARGS:

/ # bash run.sh
DEVICE_TOPIC_SUFFIX: devices[/type][/model][/subtype]/C[channel:0]
OTHER_ARGS:
Running in stand-alone docker mode
Starting rtl_433_mqtt_hass.py...
DEVICE_TOPIC_SUFFIX: devices[/type][/model][/subtype]/C[channel:0]
OTHER_ARGS:  --debug
INFO:root:Enabling debug logging

What caught my eye was the "Running in stand-alone docker mode" which I think means it is skipping the section around row 44 where the DEVICE_TOPIC_SUFFIX is set!

So I added the following to the bottom of the first if:

  DEVICE_TOPIC_SUFFIX="${DEVICE_TOPIC_SUFFIX}"
  echo "DEVICE_TOPIC_SUFFIX: $DEVICE_TOPIC_SUFFIX"
  echo "OTHER_ARGS: $OTHER_ARGS"

  if [ ! -z $DEVICE_TOPIC_SUFFIX ]; then
    OTHER_ARGS="${OTHER_ARGS} -T ${DEVICE_TOPIC_SUFFIX}"
  fi
  echo "DEVICE_TOPIC_SUFFIX: $DEVICE_TOPIC_SUFFIX"
  echo "OTHER_ARGS: $OTHER_ARGS"

Which got us to this, which I think is progress.

/ # bash run.sh
DEVICE_TOPIC_SUFFIX: devices[/type][/model][/subtype]/C[channel:0]
OTHER_ARGS:
Running in stand-alone docker mode
DEVICE_TOPIC_SUFFIX: devices[/type][/model][/subtype]/C[channel:0]
OTHER_ARGS:  --debug
DEVICE_TOPIC_SUFFIX: devices[/type][/model][/subtype]/C[channel:0]
OTHER_ARGS:  --debug -T devices[/type][/model][/subtype]/C[channel:0]
Starting rtl_433_mqtt_hass.py...
DEVICE_TOPIC_SUFFIX: devices[/type][/model][/subtype]/C[channel:0]
OTHER_ARGS:  --debug -T devices[/type][/model][/subtype]/C[channel:0]
usage: rtl_433_mqtt_hass.py [-h] [-d] [-q] [-u USER] [-P PASSWORD] [-H HOST] [-p PORT] [-c CA_CERT] [-r] [-f] [-R RTL_TOPIC] [-D DISCOVERY_PREFIX] [-i DISCOVERY_INTERVAL] [-x EXPIRE_AFTER] [-I IDS [IDS ...]]
rtl_433_mqtt_hass.py: error: unrecognized arguments: -T devices[/type][/model][/subtype]/C[channel:0]

If I'm reading it right, the updated code is now trying to pass the -T <suffix> along, but the rtl_433_mqtt_hass.py script is saying it doesn't take that variable?

Skimming rtl_433_mqtt_hass.py, I don't see anything to process -T, and the header is missing the chunk about how to use -T, and the script overall is 100ln+ shorter than what is over at the rtl_433 project hyperlink to row 20 from merbanan/rtl_433/, which is similar but different from the script below (but is the same as what is in 22.11 which...I can't see how that handles anything with regards to the suffix, but, I'm so far from a developer, I could easily be missing it!)

AP_EPILOG="""
It is strongly recommended to run rtl_433 with "-C si" and "-M newmodel".
This script requires rtl_433 to publish both event messages and device
messages.

MQTT Username and Password can be set via the cmdline or passed in the
environment: MQTT_USERNAME and MQTT_PASSWORD.

I feel like the run.sh and rtl_433_mqtt_hass.py scripts can't actually consume the suffix, but I'm really not a programmer and might be missing something extremely obvious. I'm happy to keep poking at whatever might help, and I hope the tests today were helpful!

[SiGmAX666]

OK I tried one more thing before calling it a night.

1. I rebuilt the container using my original docker compose (in the first post here), but changing the HA topic to homeassistant2
2. I did a fresh wget for the latest rtl_433_mqtt_hass.py file, and saved it to my docker project folder.
3. I started the container, and then ran the following to pull the "stock" run.sh:

docker cp rtl433_autodiscovery_test2:/run.sh run-aarch64-0_7_0-20231228.sh

4. I added the following at row 19:

  echo "DEVICE_TOPIC_SUFFIX: $DEVICE_TOPIC_SUFFIX"
  echo "OTHER_ARGS: $OTHER_ARGS"
  echo ""
  # This is an optional parameter and we don't want to overwrite the defaults
  # DEVICE_TOPIC_SUFFIX=${DEVICE_TOPIC_SUFFIX}
  if [ ! -z $DEVICE_TOPIC_SUFFIX ]; then
    OTHER_ARGS="${OTHER_ARGS} -T ${DEVICE_TOPIC_SUFFIX}"
  fi

  echo "DEVICE_TOPIC_SUFFIX: $DEVICE_TOPIC_SUFFIX"
  echo "OTHER_ARGS: $OTHER_ARGS"
  echo ""

5. I then added two volumes to the docker-compose.yml

    volumes:
      - ./rtl_433_mqtt_hass-master-20231228.py:/rtl_433_mqtt_hass.py:ro
      - ./run-aarch64-0_7_0-20231228.sh:/run.sh:ro

I then ran the container and the resulting MQTT messages were in the correct format, eg:

{"device_class": "timestamp", "name": "Timestamp", "entity_category": "diagnostic", "enabled_by_default": false, "icon": "mdi:clock-in", "state_topic": "rtl_433_2/localhost/devices/LaCrosse-TX141W/C0/time", "unique_id": "LaCrosse-TX141W-0-UTC", "device": {"identifiers": ["LaCrosse-TX141W-0"], "name": "LaCrosse-TX141W-0", "model": "LaCrosse-TX141W", "manufacturer": "rtl_433"}}

I'm moderately sure its a problem with the 0.7.0 aarch64 build - but I'm not clear how it all goes together when building the images, etc... This dockerfile looks wrong to me, as 22.11 doesn't seem to contain the right variables to handle the suffix. But I'm really in too deep with this, I'm not a dev and can get my wires crossed pretty easily, I might be chasing shadows! (It's also really late, which never helps matters, lol.)

Thanks in advance!

[catduckgnaf]

@SiGmAX666 This is great work, do you mind helping contribute to my other repo? since this project is abandoned and unmaintained? https://github.com/catduckgnaf/rtl_433_haos_addon

[deviantintegral]

@SiGmAX666 thanks for your detailed notes. They got me on the right direction.

It turns out this is actually pretty simple. I thought merbanan/rtl_433@128755c was in the prior stable release, but it wasn't. It was only included in a tag last month. So, this feature is only working with the next version of the addon.

I updated the CHANGELOG to clarify this, and the next stable release of this addon will resolve this for you. Thanks!

[SiGmAX666]

I finally updated to 0.8.2 from my custom file I made in December. I think run.sh is still a problem.

https://github.com/pbkhrv/rtl_433-hass-addons/blob/2024.06.23.0/rtl_433_mqtt_autodiscovery/run.sh#L3

This file needs an addition to the Stand-alone Docker Mode, something like this added to the bottom of that if (without the debug echos):

  DEVICE_TOPIC_SUFFIX="${DEVICE_TOPIC_SUFFIX}"
  echo "DEVICE_TOPIC_SUFFIX: $DEVICE_TOPIC_SUFFIX"
  echo "OTHER_ARGS: $OTHER_ARGS"

  if [ ! -z $DEVICE_TOPIC_SUFFIX ]; then
    OTHER_ARGS="${OTHER_ARGS} -T ${DEVICE_TOPIC_SUFFIX}"
  fi
  echo "DEVICE_TOPIC_SUFFIX: $DEVICE_TOPIC_SUFFIX"
  echo "OTHER_ARGS: $OTHER_ARGS"