Checksum problem in TPC with pathological file

[snafus]

Hi,
During some tests (on 5.3.0 and in 4.X) I noticed that for some very small files (4 bytes, all zero valued bytes), the checksums fail in TPC (with root://).
The checksum of the file is 00040001.

Looking at the logs, in non TPC mode:
xrdcp -f -d3 testfile_4B -C adler32:00040001 root://ceph-gw1.gridpp.rl.ac.uk:1094//dteam:test/jwalder/checksum_test
...
[2021-07-16 10:54:16.395552 +0100][Dump ][Utility ] Checksum for /dteam:test/jwalder/checksum_test checksum: adler32:40001
[2021-07-16 10:54:16.395586 +0100][Info ][Utility ] Checksum verification: succeeded.
...

It can be seen that the preceding "000" is not printed (although the stored checksum is correct).
In the non-TPC mode the verification is successful.

In TPC mode:
xrdcp -f -d3 --tpc delegate only -C adler32:00040001 root://golias100.farm.particle.cz:1094//dpm/farm.particle.cz/home/dteam/test/jwalder_ckstest root://ceph-gw8.gridpp.rl.ac.uk:1094//dteam:test/jwalder/checksum_test

[2021-07-16 10:58:47.215790 +0100][Dump ][Utility ] Checksum for /dteam:test/jwalder/checksum_test checksum: adler32:40001
Run: [ERROR] CheckSum error

This time there is a mismatch - I'm assuming because of the dropped 000 ?
Again, the stored checksum is still correct.

In the last case, if the checksum is requested, but the source not provided, then it works again:
xrdcp -f -d3 --tpc delegate only -C adler32 root://golias100.farm.particle.cz:1094//dpm/farm.particle.cz/home/dteam/test/jwalder_ckstest root://ceph-gw1.gridpp.rl.ac.uk:1094//dteam:test/jwalder/checksum_test

[2021-07-16 11:09:50.834772 +0100][Dump ][Utility ] Checksum for /dteam:test/jwalder/checksum_test checksum: adler32:40001
[2021-07-16 11:09:50.834798 +0100][Info ][Utility ] Checksum verification: succeeded.

I'm not sure where in the code to start looking though, but have (hopefully) attached the test file.
testfile_4B.txt

I'm sure this is a rather pathological case, but hopefully useful to report.
Thanks,
James

[abh3]

Hi James, Thanks! All bug reports are useful however pathological they may seem. Such failures usually point to an underlying problem that may surface in the most peculiar ways. So, again, thank you for reporting it. Andy

…

On Fri, 16 Jul 2021, snafus wrote: Hi, During some tests (on 5.3.0 and in 4.X) I noticed that for some very small files (4 bytes, all zero valued bytes), the checksums fail in TPC (with root://). The checksum of the file is 00040001. Looking at the logs, in non TPC mode: xrdcp -f -d3 testfile_4B -C adler32:00040001 root://ceph-gw1.gridpp.rl.ac.uk:1094//dteam:test/jwalder/checksum_test ... [2021-07-16 10:54:16.395552 +0100][Dump ][Utility ] Checksum for /dteam:test/jwalder/checksum_test checksum: adler32:40001 [2021-07-16 10:54:16.395586 +0100][Info ][Utility ] Checksum verification: succeeded. ... It can be seen that the preceding "000" is not printed (although the stored checksum is correct). In the non-TPC mode the verification is successful. In TPC mode: xrdcp -f -d3 --tpc delegate only -C adler32:00040001 root://golias100.farm.particle.cz:1094//dpm/farm.particle.cz/home/dteam/test/jwalder_ckstest root://ceph-gw8.gridpp.rl.ac.uk:1094//dteam:test/jwalder/checksum_test [2021-07-16 10:58:47.215790 +0100][Dump ][Utility ] Checksum for /dteam:test/jwalder/checksum_test checksum: adler32:40001 Run: [ERROR] CheckSum error This time there is a mismatch - I'm assuming because of the dropped 000 ? Again, the stored checksum is still correct. In the last case, if the checksum is requested, but the source not provided, then it works again: xrdcp -f -d3 --tpc delegate only -C adler32 root://golias100.farm.particle.cz:1094//dpm/farm.particle.cz/home/dteam/test/jwalder_ckstest root://ceph-gw1.gridpp.rl.ac.uk:1094//dteam:test/jwalder/checksum_test [2021-07-16 11:09:50.834772 +0100][Dump ][Utility ] Checksum for /dteam:test/jwalder/checksum_test checksum: adler32:40001 [2021-07-16 11:09:50.834798 +0100][Info ][Utility ] Checksum verification: succeeded. I'm not sure where in the code to start looking though, but have (hopefully) attached the test file. [testfile_4B.txt](https://github.com/xrootd/xrootd/files/6829843/testfile_4B.txt) I'm sure this is a rather pathological case, but hopefully useful to report. Thanks, James -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #1483

[simonmichal]

@snafus : thanks for reporting this problem! It should be fixed now by d5d490e.

[simonmichal]

@snafus : can we close this issue?