QBUSMQTT stops after +/- 0,5 hours

[BertBJ40]

Describe the bug
In version 0.7 qbha stops after 0,5 hours. Before updating never problems.

QBHA version and installation type:
v0.7.3. ADD-on

Qbus MQTT version and installation type:
v0.3
HA add-on

I20240831 08:06:06.128532 129 ConfigManager.cpp:135] Cloud connected
I20240831 08:06:10.597523 136 FileStorage.cpp:534] Saving controller info to file: /opt/ubielite/qbushw.json
I20240831 08:06:10.615765 136 QbusMain.cpp:554] Creating controller, id: 1, serial: 004442
I20240831 08:06:10.616367 124 ConfigManager.cpp:954] Store private config to file: /ubieroot/configbackup/UbieLite_2024-08-31-08-06.json
I20240831 08:06:10.626251 135 ActiveController.cpp:289] [UL1] Currently offline
E20240831 08:49:06.255863 134 FunctionBlockManager.cpp:761] Functionblock or Device not found. State id: UL18, actual Id: UL18
E20240831 08:49:10.808151 134 FunctionBlockManager.cpp:761] Functionblock or Device not found. State id: UL18, actual Id: UL18
E20240831 08:49:11.808352 134 FunctionBlockManager.cpp:761] Functionblock or Device not found. State id: UL18, actual Id: UL18
E20240831 08:49:12.808552 134 FunctionBlockManager.cpp:761] Functionblock or Device not found. State id: UL18, actual Id: UL18
W20240831 08:49:13.808398 128 StateFilter.cpp:290] Property UL18.value has timed out. age: 6309.73ms, resend count has been reached. Removing from queue

[thomasddn]

As you probably know, to use Qbus with HA it needs two components:

• QbusMqtt (communication between Qbus controller and MQTT)
• QBHA (communication between MQTT and HA)

The logs and screenshot attached are those of QbusMqtt.

To further investigate, please answer these questions:

1. Is the QbusMqtt addon actually in a stopped state, or is it just the communication that has stopped (i.e. states do not sync)?
2. Is the QBHA addon actually in a stopped state, or is it just the communication that has stopped (i.e. states do not sync)?
3. Which versions of QbusMqtt and QBHA were you using prior to updating?
4. Which controller do you have?
5. Did you also upgrade the firmware of the controller recently?
6. Reproduce and provide logs as follows:
   • Set the log level of QBHA to DEBUG (via configuration).
   • Restart QBHA and then QbusMqtt (in this order).
   • Wait until issue appears.
   • Download the QBHA log and upload it to https://www.dropbox.com/request/3HlKIyfP4NlJMgxfmVRQ.
   • Restore the log level to its previous value (default is INFO) and restart QBHA.

[BertBJ40]

there is no more communication.
files uploaded

[thomasddn]

What about question 3, 4 and 5?

[BertBJ40]

Which versions of QbusMqtt and QBHA were you using prior to updating? No idea
Which controller do you have? CTD01E
Did you also upgrade the firmware of the controller recently? Yes, because I had this problem. But still the same issue

[thomasddn]

Looks similar to #2. Can you try to downgrade the QbusMqtt addon to v0.2?

I have the same controller as you and don't experience any problems. However, I'm still on firmware v3.12 because I did have issues when upgrading the controller firmware to v3.13. So I went back to v3.12.

[BertBJ40]

Is the only way using a backup file? Or how can I install an older version

[thomasddn]

I mainly use docker and only use HAOS for testing. As I do not have access to my test environment right now, I don't know exactly. However, after reading some community posts, it seems that the only way to rollback is to restore from the backup.

[BertBJ40]

Problem solved with downgrade the QbusMqtt addon to v0.2?

thanks

[qbushome]

I can confirm reverting back to Qbus MQTT Gaveway v0.2 solves this, BUT, my situation was slightly different.

I installed Qbus MQTT Gaveway v0.3 on release day (few months ago).
Yesterday I upgraded HA from 2024.9.3 to 2024.10.1
I started having this problem (QBHA stopping after less than 1 hour).
I restored HA 2024.9.3 -> not solved
I restored Qbus MQTT Gaveway v0.2 (backup I made a few months ago) -> problem solved.

I don't know if this is helpfull or even more mysterious.

[thomasddn]

Thanks for your detailed info! I don't think the HA version has anything to do with it. It is probably related to the controller hardware and/or firmware version in combination with the qbusmqtt binaries. Those binaries are closed source, so there's not much I can do. However, I will get in touch with Qbus and kindly ask if they can have a look.

Logs are always helpful in such cases, so it would be great if you could provide them from both qbha and qbusmqtt (when it was failing). If you like to that privately, you can upload it to https://www.dropbox.com/request/UHbLIzUkt41IRinoXVQb (which is only readable to me). Please also provide controller hardware and firmware.

[qbushome]

You are right, HA version has nothing to do with it. After running for a few hours, the issue is back.
I don't understand why it was working for months and then suddenly giving a problem.

This is happening with firmware 3.12 (installed for about 1 year), and also 3.14 (installed today).
I tried downgrading to 3.09, the System Manager confirmed I could restart, but 3.14 was still the active FW.

Just restarted QBHA and the Gateway. All working again...

I'm wondering, my MQTT Broker is very busy.
Just for testing purposes, can I install a second MQTT Broker and use that one for Qbus ?
Will HA pickup the entities?
I also experimented with wk275's solution, but I can't get it working at all (sorry just desperate).

I can provide logs, just wait until it fails again.

[qbushome]

I uploaded some logs.
I created a heartbeat (to keep the connection alive). I added a new IO in Qbus.
I switch on this heartbeat-entity in HA every 20 seconds.
I added some logic to Qbus to turn off this IO in Qbus.
I added an automation in HA to check if the heartbeat-entity is on for longer than 30 seconds, or off for longer than 50 seconds. When this is the case I restart the qbha-addon.
I'll post my findings.

[thomasddn]

@qbushome Thanks again for your insights and log files! Hopefully your logs (and from others) will help identify the problem.

When this is the case I restart the qbha-addon.

I think you will need to restart the qbusmqtt add-on. Also make sure the System Manager is not connected with your controller. You can have the SM open, but it may not be connected.

I'm wondering, my MQTT Broker is very busy.

Is this the same box as where your HA, QBHA and qbusmqtt is on? Maybe there is a lack of available resources?

---

A bit off-topic:

I tried downgrading to 3.09, the System Manager confirmed I could restart, but 3.14 was still the active FW.

To downgrade you need to write the firmware by pushing on the arrow button, then press on "forceer 1x" and then reboot. The second step is important or the downgrade will not work.

For my controller (CTD01E+) with firmware v3.12 it is working.

Just for testing purposes, can I install a second MQTT Broker and use that one for Qbus ? Will HA pickup the entities?

HA and qbus must use the same broker.

I also experimented with wk275's solution, but I can't get it working at all (sorry just desperate).

You can use whatever solution you want, I don't mind. ;) wk275's solution is more of a whole ecosystem (with grafana and such), while mine is purely focussed on integrating HA with Qbus.