Desk height and all other entities always eventually become unknown in Home Assistant #28

grahamwetzler · 2023-03-29T19:10:27Z

Running version 1.0.0 with an Uplift Desk (FRM053-2). After a day or so all of the entities in Home Assistant become unknown. The buttons still work to change the desk height though. The only way to temporarily fix this is to re-plug the Upsy Desky.

I don't see any useful logs to add.

davewongillies · 2023-04-01T02:11:56Z

I'm seeing this too with 1.0.0 but in my case re-plugging the Upsy Desky doesn't bring the entities back. HA can't see any entities from it anymore besides Status LED and Re-Detect Encoder. I've also done a factory firmware flash and its made no difference.

I can still operate the Upsy Desky from its web UI. I've got this connected to a Fully Jarvis desk.

davewongillies · 2023-04-01T03:37:27Z

I'm seeing this too with 1.0.0 but in my case re-plugging the Upsy Desky doesn't bring the entities back. HA can't see any entities from it anymore besides Status LED and Re-Detect Encoder. I've also done a factory firmware flash and its made no difference.

I can still operate the Upsy Desky from its web UI. I've got this connected to a Fully Jarvis desk.

I ended up completely removing my upsy desky from HA to get it working again.

tjhorner · 2023-04-01T20:40:33Z

HA can't see any entities from it anymore besides Status LED and Re-Detect Encoder

That's strange, were the old entities "unknown" or "unavailable"?

tjhorner · 2023-04-01T20:41:22Z

@grahamwetzler In the history of the entities which become "unknown", do they become "unavailable" for a brief time beforehand?

grahamwetzler · 2023-04-01T22:35:36Z

I'm not sure. Earlier today I moved away from the ESPHome configuration method and flashed 0.4.1 from the Github releases and it's been fine.

grahamwetzler · 2023-04-07T12:41:18Z

It took a week of but it just happened to me on version 0.4.1. The entities all became unavailable then unknown. I guess I'll try version 1.0 again and see if it happens there.

grahamwetzler · 2023-05-05T14:59:38Z

Unfortunately this is still happening on the version 1.0 file I downloaded from the Github release. Even using the reboot from the web ui does not bring back the entities in Home Assistant. The only way to temporarily get it to work is to replug the RJ45 connector.

tjhorner · 2023-05-25T00:34:31Z

Sorry to leave you hanging here. If you're still experiencing this issue, I would suggest powering the device with an external USB power supply. Since the entities first become unavailable (indicates the device is totally unreachable) this suggests to me that your desk's control box may not be able to provide enough current to keep the Upsy Desky powered. Providing the additional power supply via USB-C will resolve this.

Let me know how it goes. If you still experience this after plugging in a USB-C power cable, try installing the debug firmware from the releases. Grab debug-firmware.bin and flash that via OTA. It will provide additional information in the logs, such as why the device was reset. Once the issue occurs again, grab the logs and provide them here.

grahamwetzler · 2023-06-03T17:15:23Z

I believe I've come up with a workaround. I added an automation where any time the desk height becomes unavailable, I trigger a button press on the re-detect decoder button.

grahamwetzler · 2023-06-20T16:21:36Z

Unfortunately my workaround did not work. Even if provided with an extra source of power via the USB-C port, it still goes completely unresponsive where the only fix is to unplug/replug the RJ-45 connector. I'm going to try the debug firmware again and hopefully something will show up in the log.

grahamwetzler · 2023-06-20T20:11:04Z

I do see an issue right away. When first started, Decoder detection complete. Correct decoder variant: uplift showed up in the log. A few hours later I saw the height go unavailable in Home Assistant. I pressed a preset on my desk and I saw Decoder detection complete. Correct decoder variant: omnidesk pop up in the log

tjhorner · 2023-06-20T22:15:58Z

Can you provide the information that appears in the "Device Info" sensor? I'm interested in the reset reason. It sounds like it might be restarting in the middle of the control box sending messages, and fail to detect the correct protocol. If you are familiar with ESPHome, you can also customize the configuration of the height sensor to always use the uplift decoder.

Also, what is the model number for your keypad?

grahamwetzler · 2023-06-20T22:48:45Z

This info?

Firmware: 2023.2.4 (Mar 12 2023, 21:54:19)
Hardware: 1.0.0

I have this keypad

tjhorner · 2023-06-20T23:28:25Z

There should be a sensor called "Device Info" under "Diagnostics" in Home Assistant (though, it looks like it's truncated to just "D..." because the value is so long lol):

grahamwetzler · 2023-06-21T14:32:11Z

Oh I see,

2023.5.3|Flash: 4096kB Speed:40MHz Mode:DIO|Chip: ESP32 Features:WIFI_BGN,BLE,BT, Cores:2 Revision:3|ESP-IDF: v4.4.2|EFuse MAC: 3C:E9:0E:8B:49:D0|Reset: Software Reset CPU|Wakeup: Unknown

tjhorner · 2023-06-23T16:28:17Z

Thanks for that. I'm not seeing any abnormal reset reason, which is strange. Can you try out this firmware? It bypasses detection and forces it to use the uplift decoder.

firmware.bin.zip

If possible, please keep the web UI open until the issue occurs, this way we have the full logs from start to finish as well.

If it doesn't work, then I suspect this may be a hardware fault of some kind; I can send you out another unit in that case. It could also be some difference in how the control box talks to your specific keypad — I have the same control box as you, but a very slightly different model of keypad. So I could purchase the one you have and do some testing myself.

grahamwetzler · 2023-06-23T18:07:46Z

Thanks, I installed that firmware

grahamwetzler · 2023-06-30T18:27:49Z

Unfortunately the current height and other entities are still going unknown after a few days. Once I press a key on the desk, the height does come right back at least.

amalafrozalam · 2024-01-09T23:37:50Z

Coming here to report that this problem is still occurring on Version 2.0.0 of the firmware. I have a Fully Jarvis desk setup and I can control the desk for a while, and then the Upsy goes offline. No explanation why.

brettinternet · 2024-03-09T08:40:05Z

I'm also seeing this issue with similar logs. I have the same keypad.

ESPHome appears to frequently disconnect from a wireless connect to the device, but I'm not sure that's related:

ERROR Error sending data: [Errno 104] Connection reset by peer

tjhorner · 2024-03-09T18:32:47Z

@brettinternet can you confirm whether the state is changing to unknown or unavailable?

brettinternet · 2024-03-09T20:21:04Z

@tjhorner The Home Assistant state is Unknown with periodic history (detected for a short time if I move it up or down repeatedly). And the "Desk Height" state on the web server shows NA. The only way I can get the height state to appear is by moving the desk up and down a few times as the Troubleshooting section suggests.

I am seeing the Home Assistant device's Logbook show a few of the entities (Preset #) "became unknown" every few minutes or so. Some of the other entities (Set Preset #, Status LED) show "became unavailable". Desk Height however remains "unknown". Could this be related to the ESPHome disconnects as well?

The device is on a separate IOT VLAN than my homelab VLAN but I don't have any issues with any other device. Home Assistant runs on the host network mode in a docker container. Each VLAN has access to communicate. An mDNS repeater listens on all the interfaces.

tjhorner · 2024-03-09T21:23:19Z

@brettinternet Thanks for the info. How often do you see the entities becoming unavailable? While Desk Height is unknown, could you open the ESPHome logs and press the "Re-Detect Decoder" button entity, then share what the output from the logs is?

Could you also confirm your control box model number?

tjhorner · 2024-03-09T21:49:23Z

Since multiple people have run into this issue, I created this guide just now to assist in log collection etc: https://upsy-desky.tjhorner.dev/docs/troubleshooting/connection-issues/

brettinternet · 2024-03-10T21:36:51Z

Here are some details:

Controller

Uplift FRM053-2. The desk is almost 2 years old.

Keypad

FRM066 "Advanced Comfort Angled Keypad".

Device Info

Yesterday I noticed the device was labeled as "offline" in ESPHome despite being able to connect to it via ESPHome and view the logs. It looks like this is sometimes an mDNS issue (despite running the ESPHome container in network_mode "host" and having an mDNS repeater). As a workaround, I added a manual IP in the ESPHome config and now the device remains labeled as "online".

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  manual_ip:
    static_ip: <upsy desky static IP>
    gateway: <gateway>
    subnet: <subnet>

This appeared to stabilize the connection for much longer than a mere few minutes as I was experiencing before. Only just now, almost a full day later, have I seen the entities revert back to unknown and unavailable.

The first two lines of the logs below represent when the Desk Height becomes unknown in Home Assistant (or NA on Upsy Desky state). The second decoder detection is after I've hold the desk keypad elevation arrows up and down a few times:

13:52:36 | [I] | [standing_desk_height:036] | Starting decoder detection
-- | -- | -- | --
13:52:39 | [W] | [standing_desk_height:044] | No valid decoder found. Please make sure your desk is reporting the height and you can see it on the keypad
13:57:17 | [I] | [standing_desk_height:036] | Starting decoder detection
13:57:18 | [I] | [standing_desk_height:094] | Decoder detection complete. Correct decoder variant: uplift
13:57:18 | [I] | [standing_desk_height:095] | If you want to make this change permanent, add the following to this sensor's configuration:
13:57:18 | [I] | [standing_desk_height:096] | variant: uplift

After adding the debug options, here are the complete logs: https://gist.github.com/brettinternet/5883a606cef3024fe4237492b7e419af

The issue appears to be related to a reboot, and one reboot the values are unknown. It continues to reboot with the debug configuration options you suggested. Here's my full ESPHome config now:

substitutions:
  name: upsy-desky-76d87c

packages:
  tj_horner.upsy_desky: github://tjhorner/upsy-desky/firmware/stock.yaml@v2.0.0

esphome:
  name: ${name}
  name_add_mac_suffix: false

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  manual_ip:
    static_ip: <upsy desky static IP>
    gateway: <gateway>
    subnet: <subnet>

logger:
  level: DEBUG

debug:
  update_interval: 5s

text_sensor:
  - platform: debug
    device:
      name: "Device Info"

sensor:
  - platform: debug
    free:
      name: "Heap Free"
    block:
      name: "Heap Max Block"
    loop_time:
      name: "Loop Time"

tjhorner · 2024-04-22T18:39:50Z

Hi everyone, I've got some more details that hopefully brings us closer to a fix. One person experiencing this issue was able to capture a backtrace at the time of disconnection, and it appears that the BLE component is to blame, but not in the way initially thought. Here is the trace:

0x40083815: panic_abort at panic.c:402
0x400987bd: abort at abort.c:46
0x401fe9c7: __cxxabiv1::__terminate(void (*)()) at eh_terminate.cc:47
0x401fea0e: std::terminate() at eh_terminate.cc:57
0x401feb07: __cxa_throw at eh_throw.cc:95
0x401fea66: operator new(unsigned int) at new_op.cc:54
0x400df22d: esphome::esp32_ble::ESP32BLE::gap_event_handler(esp_gap_ble_cb_event_t, esp_ble_gap_cb_param_t*) at ble.cpp:309
0x4013378d: btc_gap_ble_cb_to_app at btc_gap_ble.c:55
 (inlined by) btc_gap_ble_cb_handler at btc_gap_ble.c:1156
0x4015c69d: btc_thread_handler at btc_task.c:196
0x4015e513: osi_thread_run at thread.c:68

Initially, I thought the BLE component may be interfering with the Wi-Fi component since they share the same antenna and were causing connection issues. But as evident by the stack trace, it's actually a firmware crash. There are some issues with similar stack traces in the ESPHome repo itself (e.g., esphome/issues#4901) and the conclusion appears to be that the device is running out of RAM. This is not surprising to me, due to the overhead of the Arduino framework and all the components the stock firmware has.

I will dig a little deeper into this to verify that this is indeed the cause, but I'm just happy that we aren't flying blind anymore! I suspect a solution to this may involve trimming down the stock firmware and offering a sort of "build your own Upsy Desky firmware" tool where one can include extra features from ESPHome if one wishes.

In the meantime, I will try to replicate the issue with this new information and report back.

dwardu89 · 2024-04-22T19:22:13Z

@tjhorner So if i were to disable the bluetooth proxy feature, it should hopefully not crash? I'm going to try it out right now and ill get back to you on the outcome after a while testing.
I noticed, the display also stops showing the height and doesnt turn off the LEDs when it did before. That crash might be the cause of it.

tjhorner · 2024-06-06T21:28:44Z

Hi everyone. I just published v3.0.0 of the firmware which includes changes that are likely to solve this issue.

Please read the release notes and try it out: https://github.com/tjhorner/upsy-desky/releases/tag/v3.0.0

Thanks!

tjhorner pinned this issue Mar 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Desk height and all other entities always eventually become unknown in Home Assistant #28

Desk height and all other entities always eventually become unknown in Home Assistant #28

grahamwetzler commented Mar 29, 2023

davewongillies commented Apr 1, 2023 •

edited

davewongillies commented Apr 1, 2023 •

edited

tjhorner commented Apr 1, 2023

tjhorner commented Apr 1, 2023

grahamwetzler commented Apr 1, 2023

grahamwetzler commented Apr 7, 2023

grahamwetzler commented May 5, 2023

tjhorner commented May 25, 2023 •

edited

grahamwetzler commented Jun 3, 2023

grahamwetzler commented Jun 20, 2023

grahamwetzler commented Jun 20, 2023

tjhorner commented Jun 20, 2023 •

edited

grahamwetzler commented Jun 20, 2023

tjhorner commented Jun 20, 2023

grahamwetzler commented Jun 21, 2023

tjhorner commented Jun 23, 2023 •

edited

grahamwetzler commented Jun 23, 2023

grahamwetzler commented Jun 30, 2023

amalafrozalam commented Jan 9, 2024

brettinternet commented Mar 9, 2024

tjhorner commented Mar 9, 2024

brettinternet commented Mar 9, 2024 •

edited

tjhorner commented Mar 9, 2024 •

edited

tjhorner commented Mar 9, 2024

brettinternet commented Mar 10, 2024 •

edited

tjhorner commented Apr 22, 2024 •

edited

dwardu89 commented Apr 22, 2024

tjhorner commented Jun 6, 2024 •

edited

Desk height and all other entities always eventually become unknown in Home Assistant #28

Desk height and all other entities always eventually become unknown in Home Assistant #28

Comments

grahamwetzler commented Mar 29, 2023

davewongillies commented Apr 1, 2023 • edited

davewongillies commented Apr 1, 2023 • edited

tjhorner commented Apr 1, 2023

tjhorner commented Apr 1, 2023

grahamwetzler commented Apr 1, 2023

grahamwetzler commented Apr 7, 2023

grahamwetzler commented May 5, 2023

tjhorner commented May 25, 2023 • edited

grahamwetzler commented Jun 3, 2023

grahamwetzler commented Jun 20, 2023

grahamwetzler commented Jun 20, 2023

tjhorner commented Jun 20, 2023 • edited

grahamwetzler commented Jun 20, 2023

tjhorner commented Jun 20, 2023

grahamwetzler commented Jun 21, 2023

tjhorner commented Jun 23, 2023 • edited

grahamwetzler commented Jun 23, 2023

grahamwetzler commented Jun 30, 2023

amalafrozalam commented Jan 9, 2024

brettinternet commented Mar 9, 2024

tjhorner commented Mar 9, 2024

brettinternet commented Mar 9, 2024 • edited

tjhorner commented Mar 9, 2024 • edited

tjhorner commented Mar 9, 2024

brettinternet commented Mar 10, 2024 • edited

Controller

Keypad

Device Info

tjhorner commented Apr 22, 2024 • edited

dwardu89 commented Apr 22, 2024

tjhorner commented Jun 6, 2024 • edited

davewongillies commented Apr 1, 2023 •

edited

davewongillies commented Apr 1, 2023 •

edited

tjhorner commented May 25, 2023 •

edited

tjhorner commented Jun 20, 2023 •

edited

tjhorner commented Jun 23, 2023 •

edited

brettinternet commented Mar 9, 2024 •

edited

tjhorner commented Mar 9, 2024 •

edited

brettinternet commented Mar 10, 2024 •

edited

tjhorner commented Apr 22, 2024 •

edited

tjhorner commented Jun 6, 2024 •

edited