Bluetooth failing after upgrade to HAOS 10 on Raspberry Pi #2485
Comments
ghost
added
the
|
HAOS 10 uses the same firmware git hash as HAOS 9 is using:
So that would mean that the new BlueZ version (5.66) or the new Kernel 6.1 is causing this problem. @bdraco have you seen such regressions with newer BlueZ/Kernel versions? |
|
Hi. Inside HA I am using Awox community integration for Awox bluetooth lights and since yesterday update its completely unusable. Below screen from history of lights mesh and one light where you can see before 7PM (time of updating my system) the connection was stable. FYI: |
|
Same issue here after upgrading to 10.0 running on a Proxmox VM on a Intel Nuc from the 8th generation. |
|
Same issue, Odroid N2+, my switchbots are mostly not responding. |
|
I experience the same issue on an Intel NUC with HAOS 10, worked fine before the upgrade to HAOS 10. |
Downgrading HA OS to 9.5 fixed it for me. |
Is it the core you downgraded? Or how do you downgrade to 9.5? |
With the command: |
Thanks will try it when I get home |
|
I'm experiencing a similar issue on Raspberry Pi 3: logs below. |
agners
added
board/generic-x86-64
|
Same here with HAOS 10 on RPi3. Bluetooth (used for Xiaomi BLE LYWSD03MMC temperature) works for a few hours, then fails and doesn't come back. In 9.5 this sometimes also happened (once every few days), but after a few minutes it automatically came back. Now only a full system reboot helps |
|
Same here on 2 Intel NUC. on both HA is installed on the internal nvme config_entry-bluetooth-68b370091bd9558e25d68e230e7092cd.json.txt config_entry-bluetooth-394b387a8cee694d2e66bb4f34072697.json.txt |
|
Have reverted back to HAOS 9.5 now, and it is much more reliable but sometimes it keeps disconnecting. Getting this message and then have to reload the Bluetooth integration. 2023-04-20 19:21:15.348 ERROR (MainThread) [homeassistant.components.bluetooth.scanner] hci0 (XX:XX:XX:XX:XX:XX): Error stopping scanner: [org.bluez.Error.InProgress] Operation already in progress |
|
Same on RPi4. Reboot doesn't help. I'll try to rollback to 9.5 as it used to "kinda" work there - only failing every hour but recovering. |
|
Smells like BLE issue for RPi4: |
|
I had the same problem - 6 Xiaomi humidity/temp sensors not working on HA 10, fine on HA9.5. |
|
Same issue here, running on Intel NUC 7th gen with Proxmox (latest version). Based on the above feedback, it looks like something is very seriously broken with Bluetooth drivers in HAOS 10. |
|
Same with my Intel NUC (HA running on Proxmox). Reverting to version 9.5 fixed the problem. |
|
Issues here as well. Running HA in a VM in Proxmox. |
|
Same here my bluetooth dongle is not longer recognised |
|
I experience the same issue on an Intel NUC with HAOS 10, worked fine before the upgrade to HAOS 10. |
Try revert back to 9.5 it worked for me |
|
Yes we can revert to 9.5, but need that HA fix this issue. |
|
Can also confirm I updated to 10.3 last night, and all BT devices that had broken with previous updates are now working happily. Looks like it's been resolved in this release. |
|
Those are good indications. Can you turn back in 3 to 5 days and give a "long-term" report from your side? @thomasvochten @Rossaluss I'm still wondering which change might have fixed this... nothing specific/related in 10.3 release notes afaics. |
Will report back in two days 👌 [Update] After day 1: still working |
|
@bcutter Actually there was this change, as discussed few posts above:
Since many of the BT devices are reporting sensor data through BLE advertisements, there is significant chance most of the issues could be resolved with this patch. |
|
Update: Home Assistant 2023.6.2 Error I'm getting: |
|
Update: seems to be fixed! |
|
On my end too, the Raspberry Pi 3 where I've previously (see #2485 (comment)) was able to reproduce didn't show the problem in 24h+. There are still I consider this issue closed. If you have Bluetooth issues still, please open a new issue. |
|
Raspberry Pi 4 and Bluetooth dongle CSR8510 A10 after that, i have tons of IO on my SSD |
|
@almirus the logs are from Home Assistant Core, please report the issue in the Core repository. If the OS indeed has a problem with that particular dongle, please open a new issue since this issue is about the built-in Bluetooth support. |
agners
added
the
stable-kernel-regression
Thank you! I've been struggling with this for days and have only just come across this issue. To the HA team: This is NOT fixed in HA OS 10.3 or 10.4 for me:
I downgraded to 9.5.. works 👍 back to 10.4, it breaks 👎 This whole issue is yet another massive HA goof!... you should rename it Home 'it breaks weekly' Assistant. The reason I run HA in Proxmox is because of the high likelihood of breakages from updates and changes. I can just snap it and revert. One such goof that springs to mind was that database upgrade failure that caused boot loops requiring manual file deletion and loss of history when you made the initial startup faster. NOT fun! Perhaps the team need to take a step back from the new features and interface beautifying and look at what really matters: STABILITY. It's no damn fun when something that runs your entire house has constant issues after updates. April 19th this issue was opened and for me it's still not fixed. If HA wasn't the most compatible solution (ie. device support) I'd have ditched it long ago for OpenHAB. Sorry for the rant but this kind of stuff has to stop! I've recommended HA to a couple of people recently and they ditched it because of issues like this! Stop trying to add new features and release updates at the rate of Apple, and focus on making the damn thing not break! |
|
I believe this issue follows up on the problem for other board vendors and VMs #2612. At least the symptoms looks the same. |
|
I am very sorry for the troubles you have. Unfortunately, that regression is not a problem directly caused by changes on our side. Instead, this is a problem in the upstream Linux kernel. Between Linux 5.15 and 6.1 larger parts of the Bluetooth stack got reworked, and this seems to cause havoc especially on our use case. Large part of the regression we've been able to solve by reverting changes made in the latest version of Linux (see this mailing list post and the many reports who say thigns work now for them, in this issue as well as in #2535). FWIW, I've tried to understand and fix the issues but the Bluetooth stack is not trivial, and the problem seems to be related to a race condition, which are especially hard to find and fix. Currently we do live with the work around, and it seems to have fixed many installations, including the unit I had here where I was able to reproduce the problem. Unfortunately there seems more to it, and your installation has troubles still 😢 @bobmarley2021 BUT: It realllly doesn't help to comment on a issue, which is not related to your board (this is about RPi), and where the symptoms look different! If you want us to have a real look into your particular issue, please open a new issue, along with all the relevant information (such as type of Bluetooth hardware). Unfortunately, we work with very very diverse type of hardware. Heck we found bugs in x86-64 which even large distros like Ubuntu didn't catch, just because HAOS users run on very old x86 machines at times. We simply can't guarantee stability on every platform in the world. That is exactly why we have the option to downgrade the operating system, to bridge the time until the issue has been addressed.
I can pretty much guarantee you that if OpenHAB would have that Bluetooth support, you'd run into the same issue, since the issue is actually in the Linux kernel. @ellnic please share such generic rants in the community forum or somewhere else, it is really not helpful in an issue tracker. |
|
@agners My appologies - I've deleted my comment. FWIW though, I do feel bobmarley2021's pain. Issues with HA are not few and far between. |
|
The Linux kernel does not prioritize Bluetooth stability and likely only will if Linux makes significant headway into the desktop market. There need to be more kernel developers interested in or have use cases for Bluetooth to ensure it keeps working. https://esphome.github.io/bluetooth-proxies/ is a much better choice if you want a just-works solution, as working Bluetooth is critical to the Espressif product line. |
I didn't realise that. This comment and this one and this and many others talk of virtualisation and Intel hardware so I wrongly assumed that whilst OP was on a Pi that it was an arch free for all. I'm going to do as @bdraco suggests and give the proxy a go. Thanks for the explanation re upstream kernels. Finally, I would like to apologise for getting on my soap box, but I'm 2 years in and I often feel like the HA journey is a grind not a glide. |
|
@bdraco I do Arch Linux (which uses pretty recent Linux kernels) on my Desktop and listen to music with Bluetooth headphones daily and use a Bluetooth HID mouse on my Laptop with Linux. And I must say, both use cases work really reliable for hours, even with recent kernels. I think the bugs we hit on Linux are really related to how we use the Bluetooth stack. It seems that constant scanning or something triggers bugs in the stack 😢 Another problem seems to be the HCI protocol: It assumes a reliable transport underneath (as in, no bytes should get lost, ever). That is typically true, but when connected through serial (like on RPi), or virtualization is at play, things might get messy, and it seems to me that the HCI protocol is not well prepared for that. Again, this are environments/situations where typical desktops OSs are not running in... Also, at least a coule of years ago Bluetooth on Windows was really horrible as well: Every vendor shipped his own stack, and interoperability was quite bad. I remember I've had issues on Windows 10 with my Bluetooth mouse all the time. It might be better nowadays, but being a Desktop operating system seems not to guarantee high Bluetooth driver quality 😉 Using ESP32, capture just BLE and use our own TCP based "HCI" (which the API between ESPHome and HA is essentially) side steps the problem completely. It has much fewer features then a full featured Bluetooth stack, but It fits exactly our use case 👍 . FWIW, I think the changes introduced in the kernel were mainly driven by Bluetooth Mesh support. Not really used, but still caused changes throughout the stack. |
I also have zero trouble with my non-virtualized Feasycom FSC-BP119s. Its pretty clear experience varies across the board here and adding virtualization into the mix is a big wild card. The proxies are a great choice for anyone having trouble since trying to get it working if its already not reliable is usually an exercise in frustration that can be avoided with the proxies. I've given up trying to troubleshooting most bluetooth issues (sometimes the issue is obvious) and usually go strait to recommending the proxies as it almost always ends the same way, and it saves everyone involved hours/days of frustration |
|
After upgrading to Home Assistant 2023.9.1 this problem seems to be back. Upgraded yesterday, have had to fully reboot my RPi3 twice since then after bluetooth devices (temperature sensors) became unavailable. |
|
If upgrading to 2023.9.1 did cause it then it's a different bug. This bug was closed after fixing a bug in Linux's Bluetooth kernel drivers. Those are part of the operating system and shipped separately to the 2023.9.1 application update. Does downgrading/reverting fix it? (This is an important step to validate the root cause). |
|
It means the adapter is locking up/crashing, but there is a regression in the reset code that tries to recovery/power cycle the adapter in 2023.9.1 that is fixed in 2023.9.2 via home-assistant/core#99979 You can manually unplug/power cycle the adapter in the mean time or replace it with an adapter that doesn't lock up. |
|
H have exactly this issue since updating to HA OS 10.0. I use HP T530 PC with internal WIFI/bluetooth card and generic HA installation. I hope HA team can solve bluetooth issue soon. |
|
@randomizeusr proper fix from one of Bluetooth Linux stack maintainers fcf6156 was added to HAOS at 30. August... last version 10.5 is from 16. August so I would just wait for 10.6 and try it again |
|
problem still exists |
|
Yeah, I'm still on 9.5 as well. Every time a new release comes out and I move to it, BT fails and I move back to 9.5 |
and others
Describe the issue you are experiencing
I'm running a HA Yellow with a BT/WiFi CM4.
After a non consistant uptime the internal Bluetooth adapter is stopping to work and the integrations fail.
6 devices have been integrated, but even after reducing to 3 devices the problem occurs.
Unlike on HAOS 9.5, Bluetooth doesn't seem to recover from intermittend issues.
What operating system image do you use?
yellow (Home Assistant Yellow)
What version of Home Assistant Operating System is installed?
10
Did you upgrade the Operating System.
Yes
Steps to reproduce the issue
Anything in the Supervisor logs that might be useful for us?
Anything in the Host logs that might be useful for us?
System information
System Information
Home Assistant Community Store
Home Assistant Cloud
Home Assistant Supervisor
Dashboards
Recorder
Spotify
Additional information
Core Logs:
`Logger: bluetooth_auto_recovery.recover
Source: components/bluetooth/util.py:78
First occurred: 01:03:46 (2437 occurrences)
Last logged: 11:13:15
Bluetooth adapter hci0 [E4:5F:01:6F:FF:84] could not be reset due to timeout`
`Logger: homeassistant.components.bluetooth.scanner
Source: components/bluetooth/scanner.py:305
Integration: Bluetooth (documentation, issues)
First occurred: 01:03:47 (1219 occurrences)
Last logged: 11:13:15
hci0 (E4:5F:01:6F:FF:84): Failed to restart Bluetooth scanner: hci0 (E4:5F:01:6F:FF:84): Failed to start Bluetooth: [org.bluez.Error.InProgress] Operation already in progress
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/bluetooth/scanner.py", line 223, in _async_start
await self.scanner.start() # type: ignore[no-untyped-call]
File "/usr/local/lib/python3.10/site-packages/bleak/init.py", line 182, in start
await self._backend.start()
File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/scanner.py", line 191, in start
self._stop = await manager.active_scan(
File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/manager.py", line 368, in active_scan
assert_reply(reply)
File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/utils.py", line 22, in assert_reply
raise BleakDBusError(reply.error_name, reply.body)
bleak.exc.BleakDBusError: [org.bluez.Error.InProgress] Operation already in progress
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/bluetooth/scanner.py", line 350, in _async_restart_scanner
await self._async_start()
File "/usr/src/homeassistant/homeassistant/components/bluetooth/scanner.py", line 305, in _async_start
raise ScannerStartError(
homeassistant.components.bluetooth.scanner.ScannerStartError: hci0 (E4:5F:01:6F:FF:84): Failed to start Bluetooth: [org.bluez.Error.InProgress] Operation already in progress`
The text was updated successfully, but these errors were encountered: