root filesystem corruption on apt-get upgrade to Raspbian Jessie

[zerxy]

I am experiencing root filesystem corruption when I do a apt-get upgrade from Raspbian Wheezy (from 2013-09-25-wheezy-raspbian.img) to Jessie. Prior to this I used rpi-update to update to latest VideoCore libraries and SDK with no errors at this point in dmesg. Relevant dmesg output after performing apt-get:

[  376.642235] EXT4-fs error (device mmcblk0p2) in ext4_dirty_inode:5081: error 860660780
[  872.816130] EXT4-fs error (device mmcblk0p2) in ext4_dirty_inode:5081: error 863970348
[  975.020297] EXT4-fs error (device mmcblk0p2) in ext4_dirty_inode:5081: error 105042846
[ 1421.942344] mmc0: final write to SD card still running
[ 1430.475252] mmc0: Timeout waiting for hardware interrupt - cmd12.
[ 1430.476435] mmcblk0: error -110 sending stop command, original cmd response 0x900, card status 0x900
[ 2498.232562] EXT4-fs warning (device mmcblk0p2): empty_dir:2468: bad directory (dir #40530) - no `.' or `..'
[ 2645.218545] EXT4-fs error (device mmcblk0p2) in ext4_dirty_inode:5081: error 876389420

Relevant apt-get output:

Preparing to replace dictionaries-common 1.12.11 (using .../dictionaries-common_1.20.3_all.deb) ...
Leaving 'diversion of /usr/share/dict/words to /usr/share/dict/words.pre-dictionaries-common by dictionaries-common'
Unpacking replacement dictionaries-common ...
dpkg: error processing /var/cache/apt/archives/dictionaries-common_1.20.3_all.deb (--unpack):
 too-long line or missing newline in `/var/lib/dpkg/triggers/aspell-autobuildhash'

The file /var/lib/dpkg/triggers/aspell-autobuildhash appears to be corrupt.

After this I deleted .rpi-firmware/ and redid rpi-update. Got the following errors in dmesg:

[ 6002.822676] EXT4-fs error (device mmcblk0p2) in ext4_dirty_inode:5081: error 965993516
[ 6114.045070] EXT4-fs error (device mmcblk0p2) in ext4_dirty_inode:5081: error 102116663
[ 6131.757361] EXT4-fs error (device mmcblk0p2) in ext4_dirty_inode:5081: error 102116663

Booting from a 8 gig class 4 Sandisk SD card which was purchased locally from a reputable bricks and mortar retailer so I know it's genuine and using files from September 10 in /boot . No overclocking or overvolting.

[licaon-kter]

Do you use a real /tmp folder or a tmpfs one? IIRC, dpkg might fill up /tmp hence some errors.

[ghollingworth]

The real errors are:

[ 1421.942344] mmc0: final write to SD card still running
[ 1430.475252] mmc0: Timeout waiting for hardware interrupt - cmd12.
[ 1430.476435] mmcblk0: error -110 sending stop command, original cmd response 0x900, card status 0x900

This means that communication with the SDCard has failed in some way... This is the usual corruption we've seen in the part and is something I'm working on diagnosing now. I've written an SD protocol analyser expressly for this purpose and it's working, so it's really a matter of time. Can't be very helpful on timing yet but I'm hoping to nail this one down over the next two weeks

Gordon

[zerxy]

It's been two weeks. Any progress?

[ghollingworth]

Yes,

I've been able to identify a bug somewhere in the dma - FIFO interface in the Arasan module. Basically under very specific circumstances we fail to transmit data correctly to the sd card which results in corruption. But it was a very specific sd card that caused the problem so still don't understand what it is about the card that triggers the effect.

Stay tuned but I'm hoping to fully diagnose the issue in the next two weeks

Gordon

[lurch]

Looks like #413 is where these issues are now being discussed / referenced.

[popcornmix]

@zerxy
Have you tried latest rpi-update firmware? This has fixed sdcard corruption for many.

[zerxy]

Not yet. I have a card ready for testing, a class 10 Transcend. If that tests okay I'll try the new firmware with my Sandisk card.

[zerxy]

I've just finished testing. Upgraded from Wheezy to Jessie twice on two different cards, a class 4 Sandisk and a class 10 Transcend. No SD card errors were logged and no corruption has been detected.

[popcornmix]

Great! Okay to close?

[lightstriders]

This issue is now happening for the B+ with latest 2014-06-20 Raspbian.

See here for discussions:

http://www.raspberrypi.org/forums/viewtopic.php?f=28&t=85210&p=602257#p602257
http://www.raspberrypi.org/forums/viewtopic.php?f=28&t=75298