rdiff --paranoia produces (spurious?) critical errors

[dbaarda]

This was reported to Debian at;

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=435901

I've confirmed it still happens with v2.0.2 in Debian testing now.

This should not be happening. It's unlikely to cause any corruption (which would require a strong-sum collision), but it might be indicating a rollsum bug that could be causing delta calculation to miss some matches, resulting in a larger delta than it should.

I will investigate.

[dbaarda]

I figured it out, and it is indeed a spurious bug in how --paranoid is implemented.

The problem is when a miss is appended here;

librsync/src/delta.c

Line 151 in 7c21b88

result = rs_appendmiss(job, 1);

It can trigger an rs_appendflush() here;

librsync/src/delta.c

Line 307 in 7c21b88

result = rs_appendflush(job);

Which can block if the output "tube" gets too full here;

librsync/src/delta.c

Line 370 in 7c21b88

return rs_tube_catchup(job);

And while its blocked and until the "tube" catches up, the scoop_pos offset into the input data is invalid. However, the paranoid checking then uses it to calculate the full block rollsum here;

librsync/src/delta.c

Line 154 in 7c21b88

RollsumUpdate(&test, job->scoop_next + job->scoop_pos,

Resulting in the paranoia checking comparing the checksum calculated at the wrong offset.

This could be fixed by just removing the paranoia checking, which I'm really tempted to do. However, I'll also look into how hard it would be to ensure the scoop_pos offset is always valid even when the tube blocks.

[dbaarda]

After looking at the code in more detail I've decided that it's non-trivial to change the code to keep the scoop_pos offset valid while the tube is blocked.

The --paranoia feature is well past it's use-by date, and is now officially causing more bugs than it's uncovering. I'm just going to remove it.