file copied using qvm-copy-to-vm probably ended up corrupted

[tezeb]

This probably will be hard to reproduce. When working on Qubes 3.2 and copying multiple files, it happened (so far once) that one of files being copied got corrupted. Files were copied between archlinux templates(if that matters).

Setup:

1. usb-vm, with connected SD-card(PCI device attached)
2. target-vm, to which file will be copied

Trying to copy 3 files using following command:

usb-vm $ ls -l
-rwxr-xr-x 1 root root 4294967295 img_all.tar.gz
-rwxr-xr-x 1 root root  626909320 img_all.tar.gz_2
-rwxr-xr-x 1 root root  165263360 other.tar

usb-vm $ qvm-copy-to-vm target-vm img_all.tar.gz* other.tar

Due to size limitation of SD-card filesystem(FAT32), file img_all.tar.gz was split into two chunks, so after copying to target-vm I reconstructed it using following command.

target-vm $ dd if=img_all.tar.gz_2 of=img_all.tar.gz conv=notrunc oflag=append bs=16M

File sizes were OK, but checking sha256sum img_all.tar.gz gives invalid result(also gzip errors-out with crc-error). At this point I verified merging point of files(using hexdump). I also checked sha256sum of original files, files on SD-card and copies in target-vm. The only file(or actually part of it), that didn't match was first part of img_all.tar.gz:

target-vm $ dd if=img_all.tar.gz bs=65537 count=65535 | sha256sum
65535+0 records in
65535+0 records out
4294967295 bytes (4.3 GB, 4.0 GiB) copied, 46.4987 s, 92.4 MB/s
9a9bd8e44fcc3cbb2aa4c45847aa622b7ed6d67870faf9d0dcaf22dbd2622159  -

target-vm $ rm img_all.tar.gz

Copied again using qvm-copy-to-vm target-vm img_all.tar.gz.

target-vm $ sha256sum img_all.tar.gz
6e4d3aa1c511988ab894755f69a047806b0c30c4c8718a0068a8b8fb7ffe2182  img_all.tar.gz

target-vm $ dd if=img_all.tar.gz_2 of=img_all.tar.gz conv=notrunc oflag=append bs=16M
37+1 records in
37+1 records out
626909320 bytes (627 MB, 598 MiB) copied, 13.8114 s, 45.4 MB/s

target-vm $ dd if=img_all.tar.gz bs=65537 count=65535 | sha256sum
65535+0 records in
65535+0 records out
4294967295 bytes (4.3 GB, 4.0 GiB) copied, 49.3768 s, 87.0 MB/s
6e4d3aa1c511988ab894755f69a047806b0c30c4c8718a0068a8b8fb7ffe2182  -

General notes:

I do understand that this sounds murky at best, but I've analyzed carefully possible issues on my('user') side. The only other possibility that comes to my mind is that dd corrupted file while appending it, as that's the step I am not able to prove.

[andrewdavidwong]

I appreciate the effort you put into careful checking, but I think that, in order for us to accept this as a Qubes bug, we're going to need a much simpler example that isolates qvm-copy-to-vm as the only variable.

The only other possibility that comes to my mind is that dd corrupted file while appending it, as that's the step I am not able to prove.

Have you tried using cat to merge the files instead?

[andrewdavidwong]

I appreciate the effort you put into careful checking, but I think that, in order for us to accept this as a Qubes bug, we're going to need a much simpler example that isolates qvm-copy-to-vm as the only variable.

The only other possibility that comes to my mind is that dd corrupted file while appending it, as that's the step I am not able to prove.

Have you tried using cat to merge the files instead?

[andrewdavidwong]

Closing due to lack of response.

[andrewdavidwong]

Closing due to lack of response.