Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RPi4 4GB] xHCI host controller not responding, assume dead #3404

Closed
JANogueira opened this issue Jan 13, 2020 · 77 comments
Closed

[RPi4 4GB] xHCI host controller not responding, assume dead #3404

JANogueira opened this issue Jan 13, 2020 · 77 comments

Comments

@JANogueira
Copy link

Describe the bug
After boot-up, an when activating a service that uses USB interface (Network UPS tools, as an example) xHCI interface crashs and the USB devices get disconnected. Recovery only possible after system reboot.

List of USB devices when the system boots up:

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 0463:ffff MGE UPS Systems UPS
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

After enabling a:

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

To reproduce
After system boot, start Network UPS tools (UPS connected through USB, having a poll frequency of 30 seconds), and the xHCI interface will crash shortly after, leading to all USB devices being disconnected.

System

Details:
Raspberry Pi 4 Model B Rev 1.1

# cat /etc/os-release | head -4
NAME=HassOS
VERSION="3.8 (RaspberryPi 4 64bit)"
ID=hassos
VERSION_ID=3.8
# uname -a
Linux hassio 4.19.93-v8 #1 SMP PREEMPT Sun Jan 12 18:33:10 UTC 2020 aarch64 Hassio/OS
# cat /proc/cpuinfo | tail -3
Revision        : c03111
Serial          : 1000000013df512e
Model           : Raspberry Pi 4 Model B Rev 1.1
#
# df                                                                                                                                                                                                                                     
Filesystem           1K-blocks      Used Available Use% Mounted on                                                                                                                                                                       
/dev/root                92032     92032         0 100% /                                                                                                                                                                                
devtmpfs               1915496         0   1915496   0% /dev                                                                                                                                                                             
tmpfs                  1948808         0   1948808   0% /dev/shm                                                                                                                                                                         
tmpfs                  1948808       708   1948100   0% /run                                                                                                                                                                             
tmpfs                  1948808         0   1948808   0% /sys/fs/cgroup                                                                                                                                                                   
tmpfs                  1948808       708   1948100   0% /etc/machine-id                                                                                                                                                                  
/dev/mmcblk0p7           91099     18656     65562  22% /mnt/overlay                                                                                                                                                                     
/dev/mmcblk0p7           91099     18656     65562  22% /root/.docker                                                                                                                                                                    
/dev/mmcblk0p7           91099     18656     65562  22% /etc/modprobe.d                                                                                                                                                                  
/dev/mmcblk0p7           91099     18656     65562  22% /etc/modules-load.d                                                                                                                                                              
/dev/mmcblk0p7           91099     18656     65562  22% /etc/docker                                                                                                                                                                      
/dev/mmcblk0p7           91099     18656     65562  22% /etc/dropbear                                                                                                                                                                    
/dev/mmcblk0p7           91099     18656     65562  22% /etc/udev/rules.d
/dev/mmcblk0p7           91099     18656     65562  22% /root/.ssh
/dev/mmcblk0p1           32686      3650     29036  11% /mnt/boot
/dev/mmcblk0p7           91099     18656     65562  22% /etc/hostname
/dev/mmcblk0p7           91099     18656     65562  22% /etc/systemd/timesyncd.conf
/dev/mmcblk0p7           91099     18656     65562  22% /etc/NetworkManager/system-connections
/dev/mmcblk0p7           91099     18656     65562  22% /etc/hosts
/dev/mmcblk0p8       122172044   6834280 109113732   6% /mnt/data
/dev/zram2               15856        40     14676   0% /tmp
/dev/zram1               31728       128     29312   0% /var
/dev/mmcblk0p7           91099     18656     65562  22% /var/lib/bluetooth
/dev/mmcblk0p8       122172044   6834280 109113732   6% /var/lib/docker
/dev/mmcblk0p7           91099     18656     65562  22% /var/log/journal
/dev/mmcblk0p7           91099     18656     65562  22% /var/lib/systemd
/dev/mmcblk0p7           91099     18656     65562  22% /var/lib/NetworkManager
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/c79789ef74f2520ce9bf8308f4f0a0a0c2e2d1453a2836027ab47f3c629d6263/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/c79789ef74f2520ce9bf8308f4f0a0a0c2e2d1453a2836027ab47f3c629d6263/merged
shm                      65536         0     65536   0% /mnt/data/docker/containers/4383e4a9b4bfd33f30377b389a5dd8e96f92522bfb5f5e75b5268393aca9af5f/mounts/shm
shm                      65536         0     65536   0% /var/lib/docker/containers/4383e4a9b4bfd33f30377b389a5dd8e96f92522bfb5f5e75b5268393aca9af5f/mounts/shm
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/22cdad6842ade8567501c09ce194d03f072af3910c18cb54da5175d19b0e3aa6/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/22cdad6842ade8567501c09ce194d03f072af3910c18cb54da5175d19b0e3aa6/merged
shm                      65536         0     65536   0% /mnt/data/docker/containers/d383f5286c098e038869aa411fbc087895e9852c34db2276b293d5493076e82f/mounts/shm
shm                      65536         0     65536   0% /var/lib/docker/containers/d383f5286c098e038869aa411fbc087895e9852c34db2276b293d5493076e82f/mounts/shm
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/8a1741030a19b420c34768b69144ab241b11b00ec67083b86228cee3844dd40a/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/8a1741030a19b420c34768b69144ab241b11b00ec67083b86228cee3844dd40a/merged
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/69458a8f8866e9605cd4a15109eb3d1b2a81e7b2b76c81b8f7a1d1f3f7241ec1/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/69458a8f8866e9605cd4a15109eb3d1b2a81e7b2b76c81b8f7a1d1f3f7241ec1/merged
shm                      65536        64     65472   0% /mnt/data/docker/containers/b9ddddacc42db28bed967a218c67d490d3b89e3bf3ac4bf980d0f027af739f41/mounts/shm
shm                      65536        64     65472   0% /var/lib/docker/containers/b9ddddacc42db28bed967a218c67d490d3b89e3bf3ac4bf980d0f027af739f41/mounts/shm
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/563028cbdc5e65467e4299d65e42050ca216a6252d1e144e8491723a1434ad1e/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/563028cbdc5e65467e4299d65e42050ca216a6252d1e144e8491723a1434ad1e/merged
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/a00df7501c5d7ef5be8316129829c2b57b7fef415914170006b3172bb43ebaaa/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/a00df7501c5d7ef5be8316129829c2b57b7fef415914170006b3172bb43ebaaa/merged
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/a72a5bc9321264232980140c9bc6686e89db9c4a83a1da17c8755811f160de6a/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/a72a5bc9321264232980140c9bc6686e89db9c4a83a1da17c8755811f160de6a/merged
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/7624b7f0f58cc8d32d9d8702d26ae71c27fcd0026f27a6623b15c0ec13fd499a/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/7624b7f0f58cc8d32d9d8702d26ae71c27fcd0026f27a6623b15c0ec13fd499a/merged
shm                      65536        48     65488   0% /mnt/data/docker/containers/376a3e6a98ad20b772ae7f38b2051750d0d38d2f876be82d1cc17a81ae801a05/mounts/shm
shm                      65536        48     65488   0% /var/lib/docker/containers/376a3e6a98ad20b772ae7f38b2051750d0d38d2f876be82d1cc17a81ae801a05/mounts/shm
shm                      65536        60     65476   0% /mnt/data/docker/containers/8043d49fda6b8c96b4cfd0fd77a8499b6f70e5a4ad7a66fc37eb417c1ac6a936/mounts/shm
shm                      65536        60     65476   0% /var/lib/docker/containers/8043d49fda6b8c96b4cfd0fd77a8499b6f70e5a4ad7a66fc37eb417c1ac6a936/mounts/shm
shm                      65536        68     65468   0% /mnt/data/docker/containers/ef4df3ae0e79c33285baec87cd55c0b9d28d150ffda2a19e799979073cd1297a/mounts/shm
shm                      65536        68     65468   0% /var/lib/docker/containers/ef4df3ae0e79c33285baec87cd55c0b9d28d150ffda2a19e799979073cd1297a/mounts/shm
shm                      65536        68     65468   0% /mnt/data/docker/containers/f73b6c980b13d97d27fb314c6224f8ed0ef66be8e8039b69436273dc328925f5/mounts/shm
shm                      65536        68     65468   0% /var/lib/docker/containers/f73b6c980b13d97d27fb314c6224f8ed0ef66be8e8039b69436273dc328925f5/mounts/shm
shm                      65536        56     65480   0% /mnt/data/docker/containers/6cbb56501e93036eebc86c0c4bfc38a133d2a62ab458290f5005c592a572e7e3/mounts/shm
shm                      65536        56     65480   0% /var/lib/docker/containers/6cbb56501e93036eebc86c0c4bfc38a133d2a62ab458290f5005c592a572e7e3/mounts/shm
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/9737f2fb6b35435b15a046da2583412dd66e3559ab0fde67c9fd7c2e67336aa4/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/9737f2fb6b35435b15a046da2583412dd66e3559ab0fde67c9fd7c2e67336aa4/merged
shm                      65536         0     65536   0% /mnt/data/docker/containers/05216e59fa3a97c660472ae072c417cab1743086a3bb61f40ff167f2caaa03ba/mounts/shm
shm                      65536         0     65536   0% /var/lib/docker/containers/05216e59fa3a97c660472ae072c417cab1743086a3bb61f40ff167f2caaa03ba/mounts/shm
overlay              122172044   6834280 109113732   6% /mnt/data/docker/overlay2/ce9f300e071672f49be4e085525ee1e19e381ff4e6bb869e12f5cad245018136/merged
overlay              122172044   6834280 109113732   6% /var/lib/docker/overlay2/ce9f300e071672f49be4e085525ee1e19e381ff4e6bb869e12f5cad245018136/merged
shm                      65536        68     65468   0% /mnt/data/docker/containers/f3c374354b066023eef0662df5fee6b5112d69eba4e2c18daff4223e387d00fa/mounts/shm
shm                      65536        68     65468   0% /var/lib/docker/containers/f3c374354b066023eef0662df5fee6b5112d69eba4e2c18daff4223e387d00fa/mounts/shm
# cat /proc/swaps
Filename                                Type            Size    Used    Priority
/dev/zram0                              partition       974400  0       -2
# 
  • Which model of Raspberry Pi? Raspberry Pi 4B 4GB
  • Which OS and version (cat /etc/rpi-issue)? HassOS 3.8
  • Which firmware version (vcgencmd version)?
  • Which kernel version (uname -a)? Linux hassio 4.19.93-v8 var->green.length may be left uninitialized #1 SMP PREEMPT Sun Jan 12 18:33:10 UTC 2020 aarch64 Hassio/OS

Logs
dmesg output:

[    0.197545] usbcore: registered new interface driver usbfs
[    0.197606] usbcore: registered new interface driver hub
[    0.197718] usbcore: registered new device driver usb
[    0.487955] usbcore: registered new interface driver r8152
[    0.488025] usbcore: registered new interface driver lan78xx
[    0.488474] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1
[    0.495448] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.19
[    0.495481] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.495507] usb usb1: Product: xHCI Host Controller
[    0.495528] usb usb1: Manufacturer: Linux 4.19.88-v8 xhci-hcd
[    0.495549] usb usb1: SerialNumber: 0000:01:00.0
[    0.496073] hub 1-0:1.0: USB hub found
[    0.496671] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2
[    0.496706] xhci_hcd 0000:01:00.0: Host supports USB 3.0 SuperSpeed
[    0.497145] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 4.19
[    0.497175] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.497200] usb usb2: Product: xHCI Host Controller
[    0.497220] usb usb2: Manufacturer: Linux 4.19.88-v8 xhci-hcd
[    0.497242] usb usb2: SerialNumber: 0000:01:00.0
[    0.497711] hub 2-0:1.0: USB hub found
[    0.499380] usbcore: registered new interface driver uas
[    0.499478] usbcore: registered new interface driver usb-storage
[    0.499601] usbcore: registered new interface driver usbserial_generic
[    0.499647] usbserial: USB Serial support registered for generic
[    0.507353] usbcore: registered new interface driver usbhid
[    0.507361] usbhid: USB HID core driver
[    0.830300] usb 1-1: new high-speed USB device number 2 using xhci_hcd
[    0.980982] usb 1-1: New USB device found, idVendor=2109, idProduct=3431, bcdDevice= 4.20
[    0.981020] usb 1-1: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[    0.981041] usb 1-1: Product: USB2.0 Hub
[    0.982796] hub 1-1:1.0: USB hub found
[    1.278306] usb 1-1.3: new full-speed USB device number 3 using xhci_hcd
[    1.615672] usbcore: registered new interface driver brcmfmac
[    1.999992] usb 1-1.3: New USB device found, idVendor=0463, idProduct=ffff, bcdDevice= 0.01
[    2.000016] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    2.000027] usb 1-1.3: Product: 5E
[    2.000036] usb 1-1.3: Manufacturer: EATON
[    3.967565] hid-generic 0003:0463:FFFF.0001: hiddev96,hidraw0: USB HID v1.10 Device [EATON 5E] on usb-0000:01:00.0-1.3/input0
[  172.256228] xhci_hcd 0000:01:00.0: xHCI host not responding to stop endpoint command.
[  172.272306] xhci_hcd 0000:01:00.0: Host halt failed, -110
[  172.272321] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead
[  172.272369] xhci_hcd 0000:01:00.0: HC died; cleaning up [  172.272442] usb 1-1: USB disconnect, device number 2
[  172.272467] usb 1-1.3: USB disconnect, device number 3

Additional context
This bug report was opened here because HassOS3.8 rus default raspbian Kernel.
This issue was initially reported here:
home-assistant/operating-system#526

USB cable was replaced to ensure that it was not faulty.

If more logs are needed, please let me know

@timg236
Copy link
Contributor

timg236 commented Jan 13, 2020

It might be worth trying different versions of the VL805 firmware. There’s a thread about it here
https://www.raspberrypi.org/forums/viewtopic.php?t=260879

@JANogueira
Copy link
Author

Thank you. I will give it a try and share the outcome.

@JANogueira
Copy link
Author

Well, on raspbian all went well, but not in hassio.
USB crash is still happening in hassio.

Before shutting down Raspbian I have ensured that the latest vl805 firmware was properly installed (using ´sudo vl805´ and have shown 0137ad).

Do you know if the EEPROM bin file deppends on any other optimization on the linux kernel side? or it should fix the USB behaviour by itself?

Thanks in advance.

@JANogueira
Copy link
Author

Please, disregard my last post.
I was doing it wrong while trying to flash the EEPROM.

Took additional actions and all good apparently. 3 hours have passed and all is stable.
I will observe this for a couple of days more just to be sure, and share the outcome.

Many thanks once again!

@JANogueira
Copy link
Author

Just to confirm that everything is still working perfectly.
0137ad solved the issue. USB devices are all stable and I had a small increase on the sdcard I/O.

@hardwareadictos
Copy link

Hey!! Having exactly the same issue on VL805 FW version: 000137ad

Are you still having issues? Can you provide more info about what you did to solve it?

@hardwareadictos
Copy link

Here some logs:

log_xhci_rpi4.txt

@JANogueira
Copy link
Author

Hi @hardwareadictos,

It worked for about 1 week with VL805 FW 0137ad and then I am facing the same issue again.
Not fully solved unfortunately...

@JANogueira JANogueira reopened this Jan 29, 2020
@hardwareadictos
Copy link

I opened an issue for that because I don't know if we have the same issue (I suspect we do):

#3438

Don't doubt on participating. Thanks for answering 😊

@hardwareadictos
Copy link

This could explain some things: c74b1b5

Dont know if that change is already implemented on last kernel...

@pelwell
Copy link
Contributor

pelwell commented Jan 30, 2020

That patch allows for the possibility of loading VL805 firmware from system RAM rather than EEPROM, which has to be re-enabled after a PCI reset. The VPU firmware is not making use of that facility, so when it comes to your problem that commit is a red herring.

@hardwareadictos
Copy link

That patch allows for the possibility of loading VL805 firmware from system RAM rather than EEPROM, which has to be re-enabled after a PCI reset. The VPU firmware is not making use of that facility, so when it comes to your problem that commit is a red herring.

Im not an expert, that's why i said "could" and not "actually". Was also the only recent commit referencing to xHCI.

That issue isnt happening on previous releases.

What's you opinion on that?

Thanks in advance :)

@pelwell
Copy link
Contributor

pelwell commented Jan 30, 2020

Perhaps the on-board voltage is a marginal - try adding over_voltage=1 to config.txt and rebooting.

@hardwareadictos
Copy link

Perhaps the on-board voltage is a marginal - try adding over_voltage=1 to config.txt and rebooting.

Thank you! Applied. Will test it some days and i will report back.

@JANogueira
Copy link
Author

thank you @pelwell
Also applied this to my hassio installation. I will test it as well for some days and report back.

Cheers

@hardwareadictos
Copy link

24 hours. No issues so far. Next report on Monday, but seems that was the issue...

@hardwareadictos
Copy link

One week without issues, i consider this issue fixed :)

@JANogueira
Copy link
Author

I also confirm that after 1 week, all is ok.
But not sure if over voltage should be the way to go. This is a workaround and not a fix imho.

@hardwareadictos
Copy link

Agree. It was working before that voltage related changes on kernel. Maybe someone can give more feedback about that. But for the moment all is working as always

@JamesH65
Copy link
Contributor

JamesH65 commented Feb 8, 2020

I also confirm that after 1 week, all is ok.
But not sure if over voltage should be the way to go. This is a workaround and not a fix imho.

Indeed, I expect a future firmware release will fix this, if the most recent one hasn't already done so.

@JANogueira
Copy link
Author

I am currently running on 0137ad

@timg236
Copy link
Contributor

timg236 commented Feb 8, 2020

The VideoCore firmware controls DVFS so the update would be there rather than the XHCI firmware. Might be worth trying rpi-update (assuming you are familiar with rpi-update / risks of using early firmware)

@DrJohnM61
Copy link

I am having the same issue after updating to the latest release. If I disconnect the [powered] USB3 hub and boot and then connect the hub, I can see the disks for a few minutes (they show in 'blkid' and 'df' and I can also 'ls' the mount points and see my files). They then they disconnect. Looking at dmesg, the same error as others is being produced:
[ 77.881799] xhci_hcd 0000:01:00.0: WARNING: Host System Error
[ 82.887053] xhci_hcd 0000:01:00.0: xHCI host not responding to stop endpoint command.
[ 82.887070] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead
[ 82.887236] xhci_hcd 0000:01:00.0: HC died; cleaning up
[ 82.887275] xhci_hcd 0000:01:00.0: xHCI host not responding to stop endpoint command.

I am currently running on 0137ad

I have tried over_voltage=1 (in fact I have tried a number of over_voltage numbers or 1,2 and 3)
I have performed a rpi-update
CPU/GPU is not overclocked

$ cat /proc/cpuinfo | tail -3
Revision : c03112
Serial : 10000000b9ae9a7d
Model : Raspberry Pi 4 Model B Rev 1.2

$ cat /etc/os-release | head -4
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"

$ uname -a
Linux PiHoleNAS 4.19.102-v7l+ #1295 SMP Thu Feb 6 15:49:36 GMT 2020 armv7l GNU/Linux

dmesg2.txt

I am an old DEC 10/TOPS-10, VAX/VMS and PDP11/RSX11M+ assembler programmer. Unix is rather new to me. I apologies in advance if I come over as a noob

@popcornmix
Copy link
Collaborator

@DrJohnM61 do you know what firmware/kernel you were on previously?
Does force_turbo=1 help?
Does switching to firmware from here help?

@DrJohnM61
Copy link

@DrJohnM61 do you know what firmware/kernel you were on previously?
Does force_turbo=1 help?
Does switching to firmware from here help?

IIRC the prior version was the 5th Feb release

I tried the force_turbo=1 but does not seem to make any difference.
[ 13.045392] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead
[ 13.045464] xhci_hcd 0000:01:00.0: HC died; cleaning up

Will try the firmware alternative in the morning (its late now here in London)

@DrJohnM61
Copy link

@DrJohnM61 do you know what firmware/kernel you were on previously?
Does force_turbo=1 help?
Does switching to firmware from here help?

I copied the files that your link pointed to into the /boot directory. Not sure if I was supposed to do anything after that (I am a unix noob). Rebooted and the problem with USB 3 connected hub and devices is still there. I note that the 'name -a' showed no difference in firmware version, so I suspect I am missing a step.

Went back to page one of the blog that you direct me to and executed the 'sudo rpi-update 4b2c270' command. Rebooted and then power cycled and the problem has gone away, so clearly (IMHO), this seems to be an issue that has been introduced in a later firmware version.

I will note that somewhere between the 4b2c270 version and the latest, I could not boot the RP4 with the connected USB hub/disks and see them but would have to boot without the hub connected and then plug it in after the boot had completed. This change happened after implementing the firmware change that allowed overclocking. I had for some time looked at forums to find a combination of voltage and wait times to try and fix the issue. However, the latest version of the firmware (4.19.102-v7l+ #1295 SMP) just did not allow the external powered USB hub or disks to stably connect (and even if the USB HUB was connected after boot, the USB ports would successively shut down, causing problems with my RAID sets, until the whole HUB disconnected).

Here is a snip from the log showing the USB disconnects (post force_turbo setting):
[ 14.305428] sd 2:0:0:0: [sdc] tag#20 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.305433] sd 2:0:0:0: [sdc] tag#20 CDB: opcode=0x41 41 00 89 e4 81 00 00 03 00 00
[ 14.335885] xhci_hcd 0000:01:00.0: WARN Can't disable streams for endpoint 0x81, streams are being disabled already
[ 14.337628] usb 2-1.4.3: USB disconnect, device number 6
[ 14.338786] sd 2:0:0:0: [sdc] Synchronizing SCSI cache
[ 14.355410] print_req_error: I/O error, dev sdc, sector 2313453824
[ 14.375412] sd 3:0:0:0: [sdd] tag#25 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.375420] sd 3:0:0:0: [sdd] tag#25 CDB: opcode=0x41 41 00 89 e4 80 00 00 01 00 00
[ 14.445403] sd 3:0:0:0: [sdd] tag#25 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.445412] sd 3:0:0:0: [sdd] tag#25 CDB: opcode=0x41 41 00 89 e4 80 00 00 01 00 00
[ 14.455411] print_req_error: I/O error, dev sdc, sector 2313454592
[ 14.455425] print_req_error: I/O error, dev sdc, sector 2313454848
[ 14.455434] print_req_error: I/O error, dev sdc, sector 2313455616
[ 14.525404] sd 3:0:0:0: [sdd] tag#25 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.525412] sd 3:0:0:0: [sdd] tag#25 CDB: opcode=0x41 41 00 89 e4 80 00 00 01 00 00
[ 14.595450] sd 3:0:0:0: [sdd] tag#25 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.595457] sd 3:0:0:0: [sdd] tag#25 CDB: opcode=0x41 41 00 89 e4 80 00 00 01 00 00
[ 14.675454] sd 3:0:0:0: [sdd] tag#25 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.675461] sd 3:0:0:0: [sdd] tag#25 CDB: opcode=0x41 41 00 89 e4 80 00 00 01 00 00
[ 14.745449] sd 3:0:0:0: [sdd] tag#25 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.745458] sd 3:0:0:0: [sdd] tag#25 CDB: opcode=0x41 41 00 89 e4 80 00 00 01 00 00
[ 14.805398] sd 3:0:0:0: [sdd] tag#25 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.805405] sd 3:0:0:0: [sdd] tag#25 CDB: opcode=0x41 41 00 89 e4 80 00 00 01 00 00
[ 14.855429] sd 2:0:0:0: [sdc] Synchronize Cache(10) failed: Result: hostbyte=0x07 driverbyte=0x00
[ 14.865412] sd 3:0:0:0: [sdd] tag#25 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.865419] sd 3:0:0:0: [sdd] tag#25 CDB: opcode=0x41 41 00 89 e4 80 00 00 01 00 00
[ 14.925448] sd 3:0:0:0: [sdd] tag#25 sense submit err -19 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
[ 14.925457] sd 3:0:0:0: [sdd] tag#25 CDB: opcode=0x41 41 00 89 e4 80 00 00 01 00 00
[ 14.975986] xhci_hcd 0000:01:00.0: WARN Can't disable streams for endpoint 0x81, streams are being disabled already
[ 14.977293] usb 2-1.4.4: USB disconnect, device number 7
[ 14.978401] sd 3:0:0:0: [sdd] Synchronizing SCSI cache
[ 14.978422] print_req_error: I/O error, dev sdd, sector 2313453568
[ 15.075452] print_req_error: I/O error, dev sdd, sector 2313453824
[ 15.485545] sd 3:0:0:0: [sdd] Synchronize Cache(10) failed: Result: hostbyte=0x07 driverbyte=0x00
[ 15.605939] xhci_hcd 0000:01:00.0: WARN Can't disable streams for endpoint 0x81, streams are being disabled already
[ 17.785614] Buffer I/O error on dev md0, logical block 0, lost sync page write
[ 17.785622] EXT4-fs (md0): I/O error while writing superblock
[ 17.866157] Buffer I/O error on dev md0, logical block 976690672, async page read
[ 17.887111] Buffer I/O error on dev md0, logical block 976690672, async page read
[ 18.037901] md0: detected capacity change from 4000525058048 to 0
[ 18.037951] md: md0 stopped.
[ 18.225997] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.

Happy to try different firmware if someone can explain how to install it after pulling from a file share.

@hardwareadictos
Copy link

CPU/GPU is not overclocked

Maybe your Power supply isnt giving you enough power then

@JANogueira
Copy link
Author

Guys, please share your current firmware version and all the details existing on the first post so you can help the devs with the right info on the first shot.

@malidong
Copy link

malidong commented Feb 13, 2022

I got a same issue here, and it happened on a WD 6TB HDD via USB3.

pi@raspberrypi:~ $ lsusb                                                                                                                                                                                                                                                                                             
Bus 002 Device 003: ID 152d:9561 JMicron Technology Corp. / JMicron USA Technology Corp. 
Bus 002 Device 002: ID 0411:0286 BUFFALO INC. (formerly MelCo., Inc.) 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
pi@raspberrypi:~ $ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
    |__ Port 2: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
pi@raspberrypi:~ $ uname -a                                                                                                                                                                                                                                                                                                  
Linux raspberrypi 5.10.63-v7l+ #1496 SMP Wed Dec 1 15:58:56 GMT 2021 armv7l GNU/Linux

The error logs are here. (I'm using btrfs.)

Feb 13 08:32:56 raspberrypi kernel: [74601.333202] xhci_hcd 0000:01:00.0: WARNING: Host System Error
Feb 13 08:33:01 raspberrypi kernel: [74606.408786] xhci_hcd 0000:01:00.0: xHCI host not responding to stop endpoint command.
Feb 13 08:33:01 raspberrypi kernel: [74606.408803] xhci_hcd 0000:01:00.0: USBSTS: HCHalted HSE EINT
Feb 13 08:33:01 raspberrypi kernel: [74606.408969] usb 1-1: USB disconnect, device number 2
Feb 13 08:33:01 raspberrypi kernel: [74606.411156] usb 2-1: USB disconnect, device number 2
...
Feb 13 08:33:01 raspberrypi kernel: [74606.411923] sd 1:0:0:1: Device offlined - not ready after error recovery
...
Feb 13 08:33:01 raspberrypi kernel: [74606.415336] sd 0:0:0:0: [sda] Synchronizing SCSI cache
Feb 13 08:33:01 raspberrypi kernel: [74606.415526] sd 0:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=0x01 driverbyte=0x00
Feb 13 08:33:01 raspberrypi kernel: [74606.432049] BTRFS info (device sda): forced readonly
Feb 13 08:33:01 raspberrypi kernel: [74606.461166] usb 2-2: USB disconnect, device number 3
Feb 13 08:33:01 raspberrypi kernel: [74606.463440] sd 1:0:0:0: [sdb] Synchronizing SCSI cache
Feb 13 08:33:02 raspberrypi kernel: [74606.768917] sd 1:0:0:0: [sdb] Synchronize Cache(10) failed: Result: hostbyte=0x07 driverbyte=0x00
Feb 13 08:33:02 raspberrypi kernel: [74606.950797] sd 1:0:0:1: [sdc] Synchronizing SCSI cache
Feb 13 08:33:02 raspberrypi kernel: [74607.198954] sd 1:0:0:1: [sdc] Synchronize Cache(10) failed: Result: hostbyte=0x07 driverbyte=0x00
Feb 13 08:33:02 raspberrypi kernel: [74607.269508] xhci_hcd 0000:01:00.0: WARN Can't disable streams for endpoint 0x82, streams are being disabled already
Feb 13 08:33:08 raspberrypi kernel: [74613.287026] btrfs_dev_stat_print_on_error: 4153 callbacks suppressed
Feb 13 08:33:13 raspberrypi kernel: [74618.331792] btrfs_dev_stat_print_on_error: 4871 callbacks suppressed
Feb 13 08:33:20 raspberrypi kernel: [74624.806919] btrfs_dev_stat_print_on_error: 5846 callbacks suppressed
Feb 13 08:33:25 raspberrypi kernel: [74629.820242] btrfs_dev_stat_print_on_error: 6337 callbacks suppressed
Feb 13 08:33:30 raspberrypi kernel: [74635.295587] btrfs_dev_stat_print_on_error: 4380 callbacks suppressed
...

Just FYI, this issue is no longer occurring after disabling uas.

But by following the keyword uas here, I found some information.
https://forums.raspberrypi.com/viewtopic.php?t=245931
and of course https://www.smartmontools.org/wiki/SAT-with-UAS-Linux

I changed the usb driver to usb-storage.quirks. Now waiting and see if the issue will happens again.

@wyuenho
Copy link

wyuenho commented Mar 29, 2022

Exact same issue.

Linux home 5.4.0-1056-raspi #63-Ubuntu SMP PREEMPT Mon Mar 14 07:48:06 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

sudo rpi-eeprom-config 
[all]
BOOT_UART=0
WAKE_ON_GPIO=0
POWER_OFF_ON_HALT=1
DHCP_TIMEOUT=45000
DHCP_REQ_TIMEOUT=4000
TFTP_FILE_TIMEOUT=30000
ENABLE_SELF_UPDATE=1
DISABLE_HDMI=0
BOOT_ORDER=0xf41
sudo rpi-eeprom-update 
BCM2711 detected
VL805 firmware in bootloader EEPROM
BOOTLOADER: up-to-date
CURRENT: Thu 03 Sep 2020 12:11:43 PM UTC (1599135103)
 LATEST: Thu 03 Sep 2020 12:11:43 PM UTC (1599135103)
 FW DIR: /lib/firmware/raspberrypi/bootloader/default
VL805: up-to-date
CURRENT: 000138a1
 LATEST: 000138a1
sudo lsusb -d 0bda:9210 -v

Bus 001 Device 003: ID 0bda:9210 Realtek Semiconductor Corp. 
Couldn't open device, some information will be missing
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.10
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x0bda Realtek Semiconductor Corp.
  idProduct          0x9210 
  bcdDevice           20.01
  iManufacturer           1 
  iProduct                2 
  iSerial                 3 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0020
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
sudo hdparm -I /dev/sda

/dev/sda:

ATA device, with non-removable media
	Model Number:       Samsung SSD 980 PRO 2TB                 
	Serial Number:      REDACTED     
	Firmware Revision:  5B2QGXA7
Standards:
	Likely used: 1
Configuration:
	soft sectored
	head switch time > 15us
	fixed drive
	disk xfer rate <= 5Mbs
	disk xfer rate > 5Mbs, <= 10Mbs
	data strobe offset option
	format speed tolerance gap reqd
	Logical		max	current
	cylinders	17218	0
	heads		0	0
	sectors/track	128	0
	--
	bytes/track: 512	bytes/sector: 0
	Logical/Physical Sector size:           512 bytes
	device size with M = 1024*1024:           0 MBytes
	device size with M = 1000*1000:           0 MBytes 
	cache/buffer size  = unknown
Capabilities:
	IORDY not likely
	Cannot perform double-word IO
	R/W multiple sector transfer: not supported
	DMA: not supported
	PIO: pio0
sudo smartctl -d scsi -a -x -i /dev/sda
smartctl 7.1 2019-12-30 r5022 [aarch64-linux-5.4.0-1056-raspi] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               Sabrent
Product:              
Revision:             1.00
Compliance:           SPC-4
User Capacity:        2,000,398,934,016 bytes [2.00 TB]
Logical block size:   512 bytes
LU is fully provisioned
Rotation Rate:        Solid State Device
Logical Unit id:      0x3001237923792379
Serial number:        0000000000000000
Device type:          disk
Local Time is:        Thu Mar 31 21:04:22 2022 BST
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Disabled or Not Supported
Read Cache is:        Enabled
Writeback Cache is:   Enabled

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK
Current Drive Temperature:     0 C
Drive Trip Temperature:        0 C

Error Counter logging not supported

Device does not support Self Test logging
Device does not support Background scan results logging

@nikita-fuchs
Copy link

This issue is almost 2 and a half years old now 🏝

After a short while, my external SSD also vanishes, RPi 4.

dmesg:

809.987218] xhci_hcd 0000:01:00.0: WARNING: Host System Error
[  810.019230] xhci_hcd 0000:01:00.0: Host halt failed, -110
[  815.069471] xhci_hcd 0000:01:00.0: xHCI host not responding to stop endpoint command.
[  815.069494] xhci_hcd 0000:01:00.0: USBSTS: HSE EINT
[  815.101508] xhci_hcd 0000:01:00.0: Host halt failed, -110
[  815.101521] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead
[  815.101661] xhci_hcd 0000:01:00.0: HC died; cleaning up
[  815.102216] sd 0:0:0:0: [sda] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD
[  815.102238] sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x28 28 00 00 09 43 1a 00 00 e6 00
[  815.102284] usb 1-1: USB disconnect, device number 2
[  815.102307] usb 1-1.3: USB disconnect, device number 3
[  815.102549] usb 2-1: USB disconnect, device number 2
[  815.103468] sd 0:0:0:0: [sda] tag#2 uas_zap_pending 0 uas-tag 3 inflight: CMD
[  815.103490] sd 0:0:0:0: [sda] tag#2 CDB: opcode=0x28 28 00 00 09 65 00 00 00 20 00
[  815.103504] sd 0:0:0:0: [sda] tag#3 uas_zap_pending 0 uas-tag 4 inflight: CMD
[  815.103515] sd 0:0:0:0: [sda] tag#3 CDB: opcode=0x28 28 00 00 09 43 03 00 00 01 00
[  815.103528] sd 0:0:0:0: [sda] tag#4 uas_zap_pending 0 uas-tag 5 inflight: CMD
[  815.103539] sd 0:0:0:0: [sda] tag#4 CDB: opcode=0x28 28 00 00 09 43 01 00 00 01 00
[  815.103551] sd 0:0:0:0: [sda] tag#5 uas_zap_pending 0 uas-tag 6 inflight: CMD
[  815.103561] sd 0:0:0:0: [sda] tag#5 CDB: opcode=0x28 28 00 00 09 43 00 00 00 01 00
[  815.103573] sd 0:0:0:0: [sda] tag#6 uas_zap_pending 0 uas-tag 7 inflight: CMD
[  815.103583] sd 0:0:0:0: [sda] tag#6 CDB: opcode=0x28 28 00 00 09 43 02 00 00 01 00
[  815.103595] sd 0:0:0:0: [sda] tag#7 uas_zap_pending 0 uas-tag 8 inflight: CMD
[  815.103604] sd 0:0:0:0: [sda] tag#7 CDB: opcode=0x28 28 00 00 09 43 19 00 00 01 00
[  815.103616] sd 0:0:0:0: [sda] tag#8 uas_zap_pending 0 uas-tag 9 inflight: CMD
[  815.103626] sd 0:0:0:0: [sda] tag#8 CDB: opcode=0x28 28 00 00 09 43 18 00 00 01 00
[  815.103640] sd 0:0:0:0: [sda] tag#11 uas_zap_pending 0 uas-tag 12 inflight: CMD
[  815.103650] sd 0:0:0:0: [sda] tag#11 CDB: opcode=0x28 28 00 00 09 43 15 00 00 01 00
[  815.103663] sd 0:0:0:0: [sda] tag#12 uas_zap_pending 0 uas-tag 13 inflight: CMD
[  815.103673] sd 0:0:0:0: [sda] tag#12 CDB: opcode=0x28 28 00 00 09 43 14 00 00 01 00
[  815.103686] sd 0:0:0:0: [sda] tag#13 uas_zap_pending 0 uas-tag 14 inflight: CMD
[  815.103695] sd 0:0:0:0: [sda] tag#13 CDB: opcode=0x28 28 00 00 09 43 13 00 00 01 00
[  815.103707] sd 0:0:0:0: [sda] tag#14 uas_zap_pending 0 uas-tag 15 inflight: CMD
[  815.103717] sd 0:0:0:0: [sda] tag#14 CDB: opcode=0x28 28 00 00 09 43 12 00 00 01 00
[  815.103730] sd 0:0:0:0: [sda] tag#15 uas_zap_pending 0 uas-tag 16 inflight: CMD
[  815.103739] sd 0:0:0:0: [sda] tag#15 CDB: opcode=0x28 28 00 00 09 43 11 00 00 01 00
[  815.103751] sd 0:0:0:0: [sda] tag#16 uas_zap_pending 0 uas-tag 17 inflight: CMD
[  815.103762] sd 0:0:0:0: [sda] tag#16 CDB: opcode=0x28 28 00 00 09 43 10 00 00 01 00
[  815.103774] sd 0:0:0:0: [sda] tag#17 uas_zap_pending 0 uas-tag 18 inflight: CMD
[  815.103784] sd 0:0:0:0: [sda] tag#17 CDB: opcode=0x28 28 00 00 09 43 0f 00 00 01 00
[  815.103796] sd 0:0:0:0: [sda] tag#18 uas_zap_pending 0 uas-tag 19 inflight: CMD
[  815.103806] sd 0:0:0:0: [sda] tag#18 CDB: opcode=0x28 28 00 00 09 43 0e 00 00 01 00
[  815.103819] sd 0:0:0:0: [sda] tag#19 uas_zap_pending 0 uas-tag 20 inflight: CMD
[  815.103828] sd 0:0:0:0: [sda] tag#19 CDB: opcode=0x28 28 00 00 09 43 0d 00 00 01 00
[  815.103841] sd 0:0:0:0: [sda] tag#20 uas_zap_pending 0 uas-tag 21 inflight: CMD
[  815.103851] sd 0:0:0:0: [sda] tag#20 CDB: opcode=0x28 28 00 00 09 43 0c 00 00 01 00
[  815.103863] sd 0:0:0:0: [sda] tag#21 uas_zap_pending 0 uas-tag 22 inflight: CMD
[  815.103873] sd 0:0:0:0: [sda] tag#21 CDB: opcode=0x28 28 00 00 09 43 0b 00 00 01 00
[  815.103885] sd 0:0:0:0: [sda] tag#22 uas_zap_pending 0 uas-tag 23 inflight: CMD
[  815.103895] sd 0:0:0:0: [sda] tag#22 CDB: opcode=0x28 28 00 00 09 43 0a 00 00 01 00
[  815.103908] sd 0:0:0:0: [sda] tag#23 uas_zap_pending 0 uas-tag 24 inflight: CMD
[  815.103917] sd 0:0:0:0: [sda] tag#23 CDB: opcode=0x28 28 00 00 09 43 08 00 00 01 00
[  815.103929] sd 0:0:0:0: [sda] tag#24 uas_zap_pending 0 uas-tag 25 inflight: CMD
[  815.103939] sd 0:0:0:0: [sda] tag#24 CDB: opcode=0x28 28 00 00 09 43 07 00 00 01 00
[  815.103952] sd 0:0:0:0: [sda] tag#25 uas_zap_pending 0 uas-tag 26 inflight: CMD
[  815.103961] sd 0:0:0:0: [sda] tag#25 CDB: opcode=0x28 28 00 00 09 43 06 00 00 01 00
[  815.103973] sd 0:0:0:0: [sda] tag#26 uas_zap_pending 0 uas-tag 27 inflight: CMD
[  815.103983] sd 0:0:0:0: [sda] tag#26 CDB: opcode=0x28 28 00 00 09 43 09 00 00 01 00
[  815.103994] sd 0:0:0:0: [sda] tag#27 uas_zap_pending 0 uas-tag 28 inflight: CMD
[  815.104004] sd 0:0:0:0: [sda] tag#27 CDB: opcode=0x28 28 00 00 09 43 05 00 00 01 00
[  815.104016] sd 0:0:0:0: [sda] tag#28 uas_zap_pending 0 uas-tag 29 inflight: CMD
[  815.104026] sd 0:0:0:0: [sda] tag#28 CDB: opcode=0x28 28 00 00 09 43 04 00 00 01 00
[  815.121722] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121754] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121768] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121781] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121794] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121807] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121820] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121833] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121845] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121857] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121870] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121882] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121894] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121906] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121919] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121931] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121944] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121956] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121968] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121981] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.121993] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.122005] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.122018] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.122031] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.122043] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.122056] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.122068] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.122080] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.122092] sd 0:0:0:0: Device offlined - not ready after error recovery
[  815.137587] blk_update_request: I/O error, dev sda, sector 607002 op 0x0:(READ) flags 0x80700 phys_seg 230 prio class 0
[  815.137954] blk_update_request: I/O error, dev sda, sector 647680 op 0x1:(WRITE) flags 0x104000 phys_seg 1024 prio class 0
[  815.137982] Buffer I/O error on dev sda2, logical block 236032, lost async page write
[  815.138046] Buffer I/O error on dev sda2, logical block 236033, lost async page write
[  815.138066] Buffer I/O error on dev sda2, logical block 236034, lost async page write
[  815.138085] Buffer I/O error on dev sda2, logical block 236035, lost async page write
[  815.138102] Buffer I/O error on dev sda2, logical block 236036, lost async page write
[  815.138120] Buffer I/O error on dev sda2, logical block 236037, lost async page write
[  815.138138] Buffer I/O error on dev sda2, logical block 236038, lost async page write
[  815.138155] Buffer I/O error on dev sda2, logical block 236039, lost async page write
[  815.138192] Buffer I/O error on dev sda2, logical block 236040, lost async page write
[  815.138211] Buffer I/O error on dev sda2, logical block 236041, lost async page write
[  815.144660] blk_update_request: I/O error, dev sda, sector 475136 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 0
[  815.144720] blk_update_request: I/O error, dev sda, sector 604672 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 0
[  815.144760] blk_update_request: I/O error, dev sda, sector 606980 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[  815.144790] blk_update_request: I/O error, dev sda, sector 606981 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[  815.144815] blk_update_request: I/O error, dev sda, sector 606985 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[  815.144840] blk_update_request: I/O error, dev sda, sector 606982 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[  815.144864] blk_update_request: I/O error, dev sda, sector 606983 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[  815.144887] blk_update_request: I/O error, dev sda, sector 606984 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[  815.179711] exFAT-fs (sda2): error, failed to access to FAT (entry 0x00000152, err:-5)
[  815.179737] exFAT-fs (sda2): Filesystem has been set read-only
[  815.185930] exFAT-fs (sda2): error, failed to access to FAT (entry 0x0000005a, err:-5)
[  815.190758] exFAT-fs (sda2): error, failed to access to FAT (entry 0x0000005a, err:-5)
[  815.206261] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[  815.445594] sd 0:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=0x07 driverbyte=DRIVER_OK

Other information as provided by OP:

$ cat /etc/os-release | head -4

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
$ uname -a
Linux raspberrypi 5.15.32-v8+ #1538 SMP PREEMPT Thu Mar 31 19:40:39 BST 2022 aarch64 GNU/Linux
$ cat /proc/cpuinfo | tail -3
Revision	: c03114
Serial		: 100000000f4e7811
Model		: Raspberry Pi 4 Model B Rev 1.4
(after the USB drive disappeared)
$ df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root       12532936 2535188   9338064  22% /
devtmpfs         1777048       0   1777048   0% /dev
tmpfs            1942776       0   1942776   0% /dev/shm
tmpfs             777112     864    776248   1% /run
tmpfs               5120       4      5116   1% /run/lock
/dev/mmcblk0p8    258094   31995    226100  13% /boot
tmpfs             388552       0    388552   0% /run/user/1000

@EliaTolin
Copy link

News?
I have same issue

@DerKleinePunk
Copy link

I have the same Problem!

@percysnoodle
Copy link

I'm seeing the same problem.

@Nazgile94
Copy link

Nazgile94 commented Dec 1, 2022

me ² external hdd enclosure

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
Linux raspberrypi 5.15.76-v8+ #1597 SMP PREEMPT Fri Nov 4 12:16:41 GMT 2022 aarch64 GNU/Linux

raspbian 64 bit

openmediavault 6
Version
6.0.46-5 (Shaitan)
Prozessor
BCM2835
Kernel
Linux 5.15.76-v8+

device disconnects, share disaspears - need to reset enclosure + pi
on a x86 linux machine , all working.


1.12.2022, 22:57:05
kernel: [ 509.452860] BTRFS warning (device sdb1): Skipping commit of aborted transaction.
1.12.2022, 22:57:05
kernel: [ 509.621599] sd 0:0:0:1: [sdb] Synchronize Cache(10) failed: Result: hostbyte=0x07 driverbyte=DRIVER_OK
1.12.2022, 22:57:05
kernel: [ 509.452828] BTRFS info (device sdb1): forced readonly
1.12.2022, 22:57:04
kernel: [ 508.913535] sd 0:0:0:1: [sdb] tag#16 uas_eh_abort_handler 0 uas-tag 1 inflight: IN
1.12.2022, 22:57:04
kernel: [ 508.933539] scsi host0: uas_eh_device_reset_handler start
1.12.2022, 22:57:04
kernel: [ 509.062290] usb 2-1: reset SuperSpeed USB device number 2 using xhci_hcd
1.12.2022, 22:57:04
kernel: [ 509.082480] usb 2-1: device firmware changed
1.12.2022, 22:57:04
kernel: [ 508.913555] sd 0:0:0:1: [sdb] tag#16 CDB: opcode=0x85 85 08 0e 00 00 00 01 00 00 00 00 00 00 00 ec 00
1.12.2022, 22:57:04
kernel: [ 509.090735] sd 0:0:0:1: Device offlined - not ready after error recovery
1.12.2022, 22:57:04
kernel: [ 509.090849] usb 2-1: USB disconnect, device number 2
1.12.2022, 22:57:04
kernel: [ 509.090717] scsi host0: uas_eh_device_reset_handler FAILED err -19
1.12.2022, 22:56:03
kernel: [ 447.196946] xhci_hcd 0000:01:00.0: Looking for event-dma 0000000441e40da0 trb-start 0000000441e40db0 trb-end 0000000441e40db0 seg-start 0000000441e40000 seg-end 0000000441e40ff0
1.12.2022, 22:56:03
kernel: [ 447.129915] xhci_hcd 0000:01:00.0: WARN Successful completion on short TX

@EliaTolin
Copy link

@Nazgile94 same problem.

Without rebooting not works if i umount and mount again.

@P33M
Copy link
Contributor

P33M commented Dec 1, 2022

duplicate of #5060

@P33M P33M closed this as completed Dec 1, 2022
@barart
Copy link

barart commented Sep 19, 2023

Seems closed but im new one with this problem

@cyberplant
Copy link

I been having this issue and it's now "fixed" by adding the quirks to the cmdline: 0634:5602:u (Crucial X8 1Tb)

It's a lot slower, but at least it doesn't hang everyday!!!

I was using an external HP SSD and it worked great, but it was veeery slow, so I replaced with an SSD with an external bay and my problems started. I blamed the cheap bay, replaced with a good one. Still the same. Replaced with an external well known SSD (Crucial) and the same!!

I've created a "watchdog" on my NAS, I connected my RPi to a Shelly switch, so when the HomeAssistant that's running there doesn't reply for a while, my script turns it off and on again after some seconds. This worked fine, but still had one or two reboots per day.

@dhjackal
Copy link

I take it that this issue is still occuring seeing as I appear to be having it

@dhjackal
Copy link

I been having this issue and it's now "fixed" by adding the quirks to the cmdline: 0634:5602:u (Crucial X8 1Tb)

It's a lot slower, but at least it doesn't hang everyday!!!

I was using an external HP SSD and it worked great, but it was veeery slow, so I replaced with an SSD with an external bay and my problems started. I blamed the cheap bay, replaced with a good one. Still the same. Replaced with an external well known SSD (Crucial) and the same!!

I've created a "watchdog" on my NAS, I connected my RPi to a Shelly switch, so when the HomeAssistant that's running there doesn't reply for a while, my script turns it off and on again after some seconds. This worked fine, but still had one or two reboots per day.

Would you mind sharing your solution.....step by step and the "watchdog" script with us. Would be terribly greatful 'ol chap.

@dhjackal
Copy link

Can I quantify / qualify.....is this a hardware, software, firmware / hardware combo or other issue completely???? Anyone? The reason I ask is a) it's not clear from the thread exactly where the problem lies....b) what the actual cause of the problem is (apart from A external USB connected to a Raspberry pi c) most importantly (and selfishly in my case) my Raspberry pi is still within it's 30 day retrurn period SO if it's a hardware issue i'll take the easier softer solution and send the thing back. Thanks

@nikita-fuchs
Copy link

nikita-fuchs commented May 28, 2024 via email

@dhjackal
Copy link

In my case IT helped to give my SSD an external Power supply. Declan Heerey @.> schrieb am Di., 28. Mai 2024, 13:51:

Can I quantify / qualify.....is this a hardware, software, firmware / hardware combo or other issue completely???? Anyone? The reason I ask is a) it's not clear from the thread exactly where the problem lies....b) what the actual cause of the problem is (apart from A external USB connected to a Raspberry pi c) most importantly (and selfishly in my case) my Raspberry pi is still within it's 30 day retrurn period SO if it's a hardware issue i'll take the easier softer solution and send the thing back. Thanks — Reply to this email directly, view it on GitHub <#3404 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZXRJBIB4Y54CIB6ANZJPDZERVSTAVCNFSM4KGFPQPKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJTGUYDENJRGMYA . You are receiving this because you commented.Message ID: @.
>

Cool. Thanks. Happy to try this. Easy enough to try and remedy with a spare powered USB hub I have hiding around here somewhere......now where did I leave it??? :o)

@dhjackal
Copy link

I'm having mixed results....as usual with Raspberry Pi's....I didn't enter into this project to tinker but well that's what I always end up doing. Still, learning. Anyway ;

Powering the USB SSD via a hub seems to work....BUT agonizingly not if I put any pressure (load) on the mount, filesystem / disk for long - so it probably isn't working at all....I see the following messages in dmesg

[ 561.128679] device offline error, dev sda, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 2 [ 561.128750] EXT4-fs (sda1): shut down requested (2) [ 561.128784] Aborting journal on device sda1-8. [ 561.128837] device offline error, dev sda, sector 247728128 op 0x1:(WRITE) flags 0x9800 phys_seg 1 prio class 2 [ 561.128857] device offline error, dev sda, sector 247728128 op 0x1:(WRITE) flags 0x9800 phys_seg 1 prio class 2 [ 561.128871] Buffer I/O error on dev sda1, logical block 30965760, lost sync page write [ 561.128902] JBD2: I/O error when updating journal superblock for sda1-8. [ 561.183827] sd 0:0:0:0: [sda] Synchronizing SCSI cache

@nikita-fuchs
Copy link

nikita-fuchs commented May 28, 2024 via email

@dhjackal
Copy link

Some info ;

uname -a
Linux Malta 6.6.28+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.28-1+rpt1 (2024-04-22) aarch64 GNU/Linux

`
pi@Malta:~ $ sudo rpi-eeprom-update
BOOTLOADER: up to date
CURRENT: Fri May 17 11:26:58 UTC 2024 (1715945218)
LATEST: Fri May 17 11:26:58 UTC 2024 (1715945218)
RELEASE: latest (/lib/firmware/raspberrypi/bootloader-2711/latest)
Use raspi-config to change the release.

VL805_FW: Using bootloader EEPROM
VL805: up to date
CURRENT: 000138c0
LATEST: 000138c0
`

pi@Malta:~ $ sudo rpi-eeprom-config [all] BOOT_UART=0 WAKE_ON_GPIO=1 POWER_OFF_ON_HALT=0

Official PSU, externally powered hub for SDD.

@dhjackal
Copy link

This reads more Like you should run a thorough checkt of your hard Drive. Declan Heerey @.> schrieb am Di., 28. Mai 2024, 18:00:

I'm having mixed results....as usual with Raspberry Pi's....I didn't enter into this project to tinker but well that's what I always end up doing. Still, learning. Anyway ; Powering the USB SSD via a hub seems to work....BUT agonizingly not if I put any pressure (load) on the mount, filesystem / disk for long - so it probably isn't working at all....I see the following messages in dmesg [ 561.128679] device offline error, dev sda, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 2 [ 561.128750] EXT4-fs (sda1): shut down requested (2) [ 561.128784] Aborting journal on device sda1-8. [ 561.128837] device offline error, dev sda, sector 247728128 op 0x1:(WRITE) flags 0x9800 phys_seg 1 prio class 2 [ 561.128857] device offline error, dev sda, sector 247728128 op 0x1:(WRITE) flags 0x9800 phys_seg 1 prio class 2 [ 561.128871] Buffer I/O error on dev sda1, logical block 30965760, lost sync page write [ 561.128902] JBD2: I/O error when updating journal superblock for sda1-8. [ 561.183827] sd 0:0:0:0: [sda] Synchronizing SCSI cache — Reply to this email directly, view it on GitHub <#3404 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZXRJCW4LKZ5OKUSHOBX23ZESS2VAVCNFSM4KGFPQPKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJTGU3DANZXG42Q . You are receiving this because you commented.Message ID: @.
>

Interesting. It was cheap, from Amazon....probably. You get what you pay for. I'll go down the hardware rabbit hole and see where that takes me. Thanks for the response.

@electropolis
Copy link

Is there any solution after all ?

@barart
Copy link

barart commented May 29, 2024

Is there any solution after all ?

No, just the walk-arounds mentioned above they makes the usb not to crash but makes it slower and affects the lifespan of the disks 🤷🏻‍♂️ i dont think that a realfix (that needs a kernel update) its going to happen anytime, rpi5 has similar issues too

@cyberplant
Copy link

Is there any solution after all ?

No, just the walk-arounds mentioned above they makes the usb not to crash but makes it slower and affects the lifespan of the disks 🤷🏻‍♂️ i dont think that a realfix (that needs a kernel update) its going to happen anytime, rpi5 has similar issues too

How is this affecting the lifespan of the disks? I see my system is a lot slower than before and takes much more time to boot. I think even more than the external HP disk I had before that was like 10x times slower than the Crucial!

But right now has been running since May 13th (when I rebooted it for an upgrade)!

@dhjackal
Copy link

Is there any solution after all ?

Changing the external SSD and USB connector "seems" to be working for me. For now at least. My setup now isn't ideal but it's only for testing atm.....I've reverted back to a second SD card and a adapter connected to a USB hub. I went through a multitude of swapping in and out and this worked so i moved on.

@electropolis
Copy link

Is there any solution after all ?

Changing the external SSD and USB connector "seems" to be working for me. For now at least. My setup now isn't ideal but it's only for testing atm.....I've reverted back to a second SD card and a adapter connected to a USB hub. I went through a multitude of swapping in and out and this worked so i moved on.

I also found a table that shows all reliable adapters that work on USB 3.0 with RPi4 and I figure it out that my wasn't actually working and had buy another one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests