Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproducible filesystem corruption with too-low RAM buffer size #13

Closed
patrakov opened this issue Aug 27, 2017 · 9 comments

Comments

Projects
None yet
3 participants
@patrakov
Copy link

commented Aug 27, 2017

When given "-m something_too_low", fsremap corrupts data. The precise value of "too low" is dependent on the data to be remapped, I have seen a case when 64 MB is not enough.

Here is a test case that relies only on downloadable files.

wget https://downloads.lede-project.org/releases/17.01.2/targets/x86/64/lede-17.01.2-x86-64-rootfs-ext4.img.gz
zcat lede-17.01.2-x86-64-rootfs-ext4.img.gz > lede-17.01.2-x86-64-rootfs-ext4.img
losetup /dev/loop7 lede-17.01.2-x86-64-rootfs-ext4.img
fstransform --opts-fsremap="-m 262144" /dev/loop7 xfs
xfs_repair -n /dev/loop7
# lots of corruption found

@cosmos72 cosmos72 self-assigned this Sep 19, 2017

@cosmos72

This comment has been minimized.

Copy link
Owner

commented Sep 19, 2017

Thanks, I will look into it

@patrakov

This comment has been minimized.

Copy link
Author

commented Dec 16, 2017

Any news?

@cosmos72

This comment has been minimized.

Copy link
Owner

commented Dec 16, 2017

not yet, sorry.
Hopefully the winter holidays will give me enough spare time...

@cosmos72

This comment has been minimized.

Copy link
Owner

commented Jan 7, 2018

Bug confirmed. Working on it.

@cosmos72 cosmos72 added the bug label Jan 7, 2018

@patrakov

This comment has been minimized.

Copy link
Author

commented Jun 29, 2018

Still working?

@cosmos72

This comment has been minimized.

Copy link
Owner

commented Jun 30, 2018

Hello,
sorry for the delay.

I reproduced the bug, but in addition to the (very) little available time, I am having some difficulty in finding where the behaviour of the low-RAM case starts diverging from the normal case (ponder)

@giomasce

This comment has been minimized.

Copy link

commented Mar 30, 2019

It might be sensible to note here too that I added a patch in Debian to display a warning specifically for this bug: https://salsa.debian.org/debian/fstransform/blob/master/debian/patches/0001-Output-explicit-warning-about-data-corruption-with-l.patch

Of course a proper fix would be better, but at least users are warned.

@cosmos72

This comment has been minimized.

Copy link
Owner

commented Mar 30, 2019

Makes sense.
Thanks for reminding me of this issue, I restarted working on it.

@cosmos72 cosmos72 closed this in 1a93573 Mar 31, 2019

@cosmos72

This comment has been minimized.

Copy link
Owner

commented Mar 31, 2019

After two days rewriting a lot of code (in another branch), I finally found the cause:

the loop copying continuous segments larger than RAM buffer was bugged, it incremented only one iteration variable instead of all three.

Fixed in 1a93573

cosmos72 added a commit that referenced this issue Apr 3, 2019

backport fix #13 to version 0.9.3:
fix reproducible filesystem corruption with too-low RAM buffer size:
the loop copying continuous segments larger than RAM buffer was bugged,
it incremented only one iteration variable instead of all three
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.