Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RPi4 Home Assistant OS 6.4 won't boot if RFLink is plugged in a USB port #1544

Closed
2 of 8 tasks
antro31 opened this issue Sep 11, 2021 · 19 comments
Closed
2 of 8 tasks
Labels
board/raspberrypi Raspberry Pi Boards

Comments

@antro31
Copy link

antro31 commented Sep 11, 2021

Hardware Environment

  • Raspberry Pi [4]
  • ODROID [C2/C4/N2(+)/XU4]
  • ASUS Tinker [S]
  • Generic x86-64 (like Intel NUC)
  • OVA (Open Virtualization Appliance, on Intel NUC or any other hardware, please add the Hypervisor you are using)
  • Khadas VIM3

Home Assistant OS release:

  • Fresh installation of release x.y
  • Updated from version 6.2
  • Additional information (if accessible):
    Not available

Supervisor logs:
Not Available

Journal logs:
Not Available

Kernel logs:
Not available

Description of problem:
Using RPi4 with Homeassistant OS 6.4 (32 bits) with RFLink connected to RPi USB Port (no external power on RFLink)
When upgrading to 6.4, RPI does not boot anymore.
RFLink connected to RPi has been identified has being the issue since unplugging it allows RPi to boot.

Problem is reproducible.
Booting with 6.2 with RFLink plugged in USB worked before. So this is a regression.
This should have been corrected by #1539 in 6.3, looks similar to #1011 (but this works with 6.2).
Nothing is displayed on the screen (blank screen) and RPi keeps on trying to reboot.

Had to rever to 6.2 to have everything working.

@agners agners added the board/raspberrypi Raspberry Pi Boards label Sep 13, 2021
@agners
Copy link
Member

agners commented Sep 13, 2021

With RFLink, do you mean this device?
https://www.nodo-shop.nl/en/rflink-gateway/169-rflink-arduino-dipool-behuizing.html

It seems that U-Boot, our bootloader, gets stuck in your case.

In 6.2 and earlier versions U-Boot tried to initialize USB but failed (see #1203). With HAOS 6.3 I added a patch which fixes U-Boot: It successfully can start the USB bus now. However, unfortunately, this seems to have uncovered other bugs (like #1538, and this issue). I could just "break" USB in the bootloader again, but that seems a step backwards.

Can you try the following U-Boot build: u-boot-32-bit-with-screen.zip

Plug the SD card in your PC. The first partition should be a readable partition and contain a file named u-boot.bin. Rename that file to u-boot.bin.bak and replace it with the one from the zip archive.

This should enable the screen and show what goes wrong during boot.

@agners
Copy link
Member

agners commented Sep 14, 2021

Something must be really special with that device. I tried all three popular USB to UART bridges: CP2102N, FTDI and CH341 (as well as a Keyboard) connected to my Raspberry Pi 4, and the system still reboots fine with HAOS 6.4.

@antro31
Copy link
Author

antro31 commented Sep 14, 2021

Hi agners, yes this is exactly this kind of device. This is based on Arduino Mega.

I tried your u-boot with screen. Here is what is shows:
Capture d’écran 2021-09-14 à 18 40 43

Then it reboots... And loops on this.

Looks that a similar issue exists on balena-os that also uses u-boot. Check here for a potential fix.

Hope this can help you.

@agners
Copy link
Member

agners commented Sep 14, 2021

Hm, it looks very much related, but from what I can tell this fixes a null pointer dereference, and yours seems to be caused by a BUG_ON call. But maybe I don't understand it completely. Let's give this a try, this U-Boot is with that patch applied:

u-boot-32-bit-with-screen-and-fix.zip.zip

@agners
Copy link
Member

agners commented Sep 14, 2021

I found some more, probably related fix in the same patchset. Can you also give a test with this U-Boot, independent of the result above?
u-boot-32-bit-with-screen-and-more-fixes.zip

@antro31
Copy link
Author

antro31 commented Sep 15, 2021

Just tested the 2nd one, Not better, RPI does not reboot any more, but seems to loop on the initialization of the USB bus.
Here after a screen capture:
Capture d’écran 2021-09-15 à 16 33 02

And same for your 1st version. Here is the screenshot:
Capture d’écran 2021-09-15 à 18 22 35

@mcpfr
Copy link

mcpfr commented Sep 15, 2021

Hello,
Sorry for my English, but I'm French
I have a similar problem with a PI4, a USB HD with H.A. OS 6.4 and two ardunio connected via USB.
HA won't start or restart until I remove both arduino and reconnect after HD boot.
I'm interested in solving this case, if I can help you by re-reading some tests ?

@antro31
Copy link
Author

antro31 commented Sep 15, 2021

Hi larrychapard.
What you can do in a first step is

  • boot without your 2 arduinos connected
  • revert back to version 6.2. ssh to your Homeassistant and then use ha os upgrade --version 6.2

@mcpfr
Copy link

mcpfr commented Sep 16, 2021

Thanks for this anwer Antro31.

@agners
Copy link
Member

agners commented Sep 16, 2021

@antro31 thanks for your testing. Yeah its kinda what I feared, those patches do not address the underlying issue.

This is a bit of a wild guess, but maybe it helps U-Boot with disable USB keyboard. Can you test this version?
u-boot-32-bit-disabled-usb-keyboard.zip

@antro31
Copy link
Author

antro31 commented Sep 16, 2021

Hi Stefan, thanks for the try, but bad luck. Result is exactly the same as my post 2 days ago. here

@antro31
Copy link
Author

antro31 commented Sep 18, 2021

Hi Stefan,
Some further trials that I did (but still without success), but just in case it would help you:

  • I upgrade my RPI to latest firmware so that I am up to date.
  • I changed the usb ports the Arduino is connected to, trying all the 4 ports.

BTW, My RPI4 is a 2GB model.

@agners
Copy link
Member

agners commented Sep 20, 2021

I do have an Arduino here, but its FTDI based and FTDI seem not to be problematic. I see that Arduino have different chips which are used to connect to USB (ATmega8U2 or ATmega16U2). I try to get to the bottom of this, but I need the exact setup to replicate the issue here.

@antro31 you are using Arduino Mega is that correct? Which revision?

@larrychapard What exact Arduino version/revision are you using?

Exact variants, revisions or pictures help :)

@mcpfr
Copy link

mcpfr commented Sep 20, 2021

Hello Agners,

I have two arduino:

  • Noname UNO R3, with chipset MEGA328P
  • Original Ardunio MEGA R3 with chipset ATMEGA2560

When only UNO is connect, boot on Harddisk is OK,
When MEGA connect (+ UNO) or only Mega connect, boot on Harddisk is KO (the cycle seems to come to an end)

I have to disconnect MEGA, start Rapsberry and connect MEGA, after boot on hardisk.

For information, I have not FTDI on ardunio, I use only Firmataexpress on these arduino.

@antro31
Copy link
Author

antro31 commented Sep 20, 2021 via email

@agners
Copy link
Member

agners commented Sep 24, 2021

I did receive the an Arduino Mega R3 and can reproduce the issue here 🎉 On first sight it seems to be related with descriptor strings which can't be read from the device. I assume that this is a bug in the firmware of the ATmega16U2 which acts as USB to serial converter.

Specifically, the device announces three strings:

new device strings: Mfr=1, Product=2, SerialNumber=220

However, when trying to read the product string, it seem the device disappears from the bus. This brings U-boot to crash currently. There is a fix for the crash, but U-Boot will then still retry reading that string (and the serial number successively) which takes more than 10 seconds...

I am pretty sure this is also a problem on Linux, but it seems that Linux gracefully skips the Product string:

[95153.799450] usb 1-3.3.4: New USB device strings: Mfr=1, Product=2, SerialNumber=220                                                                                                                                                                                                                                        
[95153.799453] usb 1-3.3.4: Manufacturer: Arduino (www.arduino.cc)                                                                                                                                                                                                                                                            
[95153.799455] usb 1-3.3.4: SerialNumber: 85433333231351B0B2B1

Any other device which announces a Product strings does response to those requests:

[85566.824033] usb 1-3.3.3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3                                                                                                                                                                                                                                        
[85566.824034] usb 1-3.3.3.2: Product: CTRL Keyboard                                                                                                                                                                                                                                                                          
[85566.824035] usb 1-3.3.3.2: Manufacturer: Massdrop Inc.                                                                                                                                                                                                                                                                     
[85566.824036] usb 1-3.3.3.2: SerialNumber: 124232332

Long story short: This is a broken USB device, but it seems that other systems can deal with it. I'll check if I can make U-Boot behave more gracefully.

Btw, this has been reported previously for 64-bit HAOS in #1011. Only now with 32-bit USB being fixed in #1529, this started to show up on 32-bit HAOS too. I see if I can build a

agners added a commit to agners/operating-system that referenced this issue Sep 27, 2021
…t#1544)

Some USB devices cause the USB stack to get stuck with a TRB stall
error. This adds a patch which recovers from this situation.

This avoids an U-Boot crash when Arduino Mega R3 devices are connected,
which cause an USB stall when trying to read the product string.
agners added a commit to agners/operating-system that referenced this issue Sep 27, 2021
…t#1544)

Some USB devices cause the USB stack to get stuck with a stall error.
This adds a patch which recovers from this situation.

This avoids an U-Boot crash when Arduino Mega R3 devices are connected,
which cause an USB stall when trying to read the product string.
agners added a commit that referenced this issue Sep 27, 2021
Some USB devices cause the USB stack to get stuck with a stall error.
This adds a patch which recovers from this situation.

This avoids an U-Boot crash when Arduino Mega R3 devices are connected,
which cause an USB stall when trying to read the product string.
agners added a commit that referenced this issue Oct 4, 2021
Some USB devices cause the USB stack to get stuck with a stall error.
This adds a patch which recovers from this situation.

This avoids an U-Boot crash when Arduino Mega R3 devices are connected,
which cause an USB stall when trying to read the product string.
@agners
Copy link
Member

agners commented Oct 5, 2021

This is fixed with HAOS 6.5 (currently on beta channel).

@agners agners closed this as completed Oct 5, 2021
@antro31
Copy link
Author

antro31 commented Oct 5, 2021

I confirm this is OK in HAOS 6.5.
Thanks a lot agners.

@mcpfr
Copy link

mcpfr commented Oct 6, 2021

Hello,

I confirm the system starts up well with ardunio connected

@agners A big thank-you

Larry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
board/raspberrypi Raspberry Pi Boards
Projects
None yet
Development

No branches or pull requests

3 participants