file checksum mysteriously changed

[wayzard]

I am not sure if this is an error with the md5sum program or an error with a dom mangling the data when the device is added to loopback or... I don't know....

Scanning and/or copying a large file from an ext2 partition (from an external USB) shows wrong md5sum when summed in Qubes Dom0:

[work@dom0 ~]$ sudo mount /dev/sdb1 /tmp/mnt/
[work@dom0 ~]$ sudo cp /tmp/mnt/kali-linux-2.0-amd64.iso ./
[work@dom0 ~]$ md5sum /tmp/mnt/kali-linux-2.0-amd64.iso kali-linux-2.0-amd64.iso 
c693c9beecd77104718cede97d78ff08  /tmp/mnt/kali-linux-2.0-amd64.iso
c693c9beecd77104718cede97d78ff08  kali-linux-2.0-amd64.iso
[work@dom0 ~]$ sudo umount /tmp/mnt/
[work@dom0 ~]$ sudo mount /dev/sdb1 /tmp/mnt/
[work@dom0 ~]$ cat kali-linux-2.0-amd64.iso | md5sum 
c693c9beecd77104718cede97d78ff08  -
[work@dom0 ~]$ md5sum /tmp/mnt/kali-linux-2.0-amd64.iso 
c30f25e06560322d4e46bd5b40ff0e14  /tmp/mnt/kali-linux-2.0-amd64.iso
[work@dom0 ~]$ 

When viewed on the external USB in Dom0, it is already a different MD5 sum. Then, after copied and used in a VM/DomU/guest, it becomes another MD5 sum.

FYI, on Linux Mint and Fedora 23 (not Qubes), both before and after loading it in Qubes, it was consistently:
ef192433017c5d99a156eaef51fd389d

The files that I tried were over 3GB in size. The problem existed for all such files.

[cfcs]

Did the file size change? (ls -l)
This sounds very, very strange.

By the way, you should use sha256sum or sha224sum; using md5sum is like running crc32 on it.

[cfcs]

Did the file size change? (ls -l)
This sounds very, very strange.

By the way, you should use sha256sum or sha224sum; using md5sum is like running crc32 on it.

[wayzard]

The filesize did not change. When dding the first hundred(s of) MB(s), checksum looked the same as on another system. When I checksum the whole giant file, though, it would differ.

Unfortunately this problem is no longer reproducible (or at least not reliably) after the system has reset a few times. Checksums look good now (except for a strange episode where it dropped the device while checksumming a file on a large vfat partition. I/O error and stopped writing to or even recognizing the usb until I unbind/rebind the pci slot. This also is no longer reproducible reliably).

I don't know what else to say about that. Maybe the issue should be closed unless someone can reproduce it?

EDIT: I just use md5 to check if a file changed in the course of writing between my laptop and my thumbdrive. I'm more worried about quickly checking a write failure than whether some wizardry has inserted maliciousness into my file automagically with a collision attack.

[wayzard]

The filesize did not change. When dding the first hundred(s of) MB(s), checksum looked the same as on another system. When I checksum the whole giant file, though, it would differ.

Unfortunately this problem is no longer reproducible (or at least not reliably) after the system has reset a few times. Checksums look good now (except for a strange episode where it dropped the device while checksumming a file on a large vfat partition. I/O error and stopped writing to or even recognizing the usb until I unbind/rebind the pci slot. This also is no longer reproducible reliably).

I don't know what else to say about that. Maybe the issue should be closed unless someone can reproduce it?

EDIT: I just use md5 to check if a file changed in the course of writing between my laptop and my thumbdrive. I'm more worried about quickly checking a write failure than whether some wizardry has inserted maliciousness into my file automagically with a collision attack.

[andrewdavidwong]

I don't know what else to say about that. Maybe the issue should be closed unless someone can reproduce it?

Closing unless/until someone can reproduce this.

[andrewdavidwong]

I don't know what else to say about that. Maybe the issue should be closed unless someone can reproduce it?

Closing unless/until someone can reproduce this.