rsync tests fail in 2038

[bmwiedemann]

While working on reproducible builds for openSUSE, I found that our rsync package fails tests in 2038.

Background:
As part of my work on reproducible builds for openSUSE, I check that software still gives identical build results in the future.
The usual offset is +16 years, because that is how long I expect some software will be used in some places.
This showed up failing tests in our package build.

The test log

 total: matches=0  hash_hits=0  false_alarms=0 data=0
 
 sent 707 bytes  received 835 bytes  3,084.00 bytes/sec
 total size is 129  speedup is 0.08
 -------------
 check how the directory listings compare with diff:
 
 --- /home/abuild/rpmbuild/BUILD/rsync-3.2.5/testtmp/exclude/ls-from	2038-09-21 22:30:28.628000000 +0000
 +++ /home/abuild/rpmbuild/BUILD/rsync-3.2.5/testtmp/exclude/ls-to	2038-09-21 22:30:28.632000000 +0000
 @@ -1,29 +1,29 @@
 -drwxr-xr-x               0    399.399         6 1902-08-16 16:02:10 .
 -drwxr-xr-x               0    399.399         3 1902-08-16 16:02:10 ./bar
 -drwxr-xr-x               0    399.399         3 1902-08-16 16:02:10 ./bar/down
 -drwxr-xr-x               0    399.399         4 1902-08-16 16:02:10 ./bar/down/to
 -drwxr-xr-x               0    399.399         3 1902-08-16 16:02:10 ./bar/down/to/bar
 -drwxr-xr-x               0    399.399         2 1902-08-16 16:02:10 ./bar/down/to/bar/baz
 +drwxr-xr-x               0    399.399         6 2038-09-21 22:30:26 .
 +drwxr-xr-x               0    399.399         3 2038-09-21 22:30:26 ./bar
 +drwxr-xr-x               0    399.399         3 2038-09-21 22:30:26 ./bar/down
 +drwxr-xr-x               0    399.399         4 2038-09-21 22:30:26 ./bar/down/to
 +drwxr-xr-x               0    399.399         3 2038-09-21 22:30:26 ./bar/down/to/bar
 +drwxr-xr-x               0    399.399         2 2038-09-21 22:30:26 ./bar/down/to/bar/baz
  -rw-r--r--               9    399.399         1 2038-09-21 22:30:28 ./bar/down/to/bar/baz/nodel.deep
 -drwxr-xr-x               0    399.399         3 1902-08-16 16:02:11 ./bar/down/to/foo
 +drwxr-xr-x               0    399.399         3 2038-09-21 22:30:27 ./bar/down/to/foo
  -rw-r--r--               6    399.399         1 2038-09-21 22:30:26 ./bar/down/to/foo/+ file3
  -rw-r--r--               7    399.399         1 2038-09-21 22:30:26 ./bar/down/to/foo/file1
  -rw-r--r--               5    399.399         1 2038-09-21 22:30:26 ./bar/down/to/foo/file4
  -rw-r--r--              18    399.399         1 2038-09-21 22:30:26 ./bar/down/to/foo/file4.junk
  l---------               0    399.399         1 2038-09-21 22:30:27 ./bar/down/to/foo/sym -> too
  -rw-r--r--               8    399.399         1 2038-09-21 22:30:26 ./bar/down/to/foo/to
 -drwxr-xr-x               0    399.399         2 1902-08-16 16:02:10 ./bar/down/to/foo/too
 -drwxr-xr-x               0    399.399         3 1902-08-16 16:02:10 ./foo
 +drwxr-xr-x               0    399.399         2 2038-09-21 22:30:26 ./bar/down/to/foo/too
 +drwxr-xr-x               0    399.399         3 2038-09-21 22:30:26 ./foo
  -rw-r--r--              23    399.399         1 2038-09-21 22:30:26 ./foo/.filt
 -drwxr-xr-x               0    399.399         2 1902-08-16 16:02:10 ./foo/sub
 +drwxr-xr-x               0    399.399         2 2038-09-21 22:30:26 ./foo/sub
  -rw-r--r--              15    399.399         1 2038-09-21 22:30:26 ./foo/sub/file1
 -drwxr-xr-x               0    399.399         3 1902-08-16 16:02:10 ./mid
 +drwxr-xr-x               0    399.399         3 2038-09-21 22:30:26 ./mid
  -rw-r--r--              15    399.399         1 2038-09-21 22:30:26 ./mid/.cvsignore
  -rw-r--r--               8    399.399         1 2038-09-21 22:30:26 ./mid/.filt2
 -drwxr-xr-x               0    399.399         3 1902-08-16 16:02:10 ./mid/for
 -drwxr-xr-x               0    399.399         2 1902-08-16 16:02:10 ./mid/for/foo
 +drwxr-xr-x               0    399.399         3 2038-09-21 22:30:26 ./mid/for
 +drwxr-xr-x               0    399.399         2 2038-09-21 22:30:26 ./mid/for/foo
  -rw-r--r--               9    399.399         1 2038-09-21 22:30:26 ./mid/for/foo/keep
  -rw-r--r--               6    399.399         1 2038-09-21 22:30:26 ./mid/for/one-in-one-out
  -rw-r--r--               6    399.399         1 2038-09-21 22:30:26 ./mid/one-for-all
 -drwxr-xr-x               0    399.399         3 1902-08-16 16:02:10 ./new
 -drwxr-xr-x               0    399.399         2 1902-08-16 16:02:10 ./new/keep
 +drwxr-xr-x               0    399.399         3 2038-09-21 22:30:26 ./new
 +drwxr-xr-x               0    399.399         2 2038-09-21 22:30:26 ./new/keep
 -------------
 check how the files compare with diff:
 
 -------------
 Failed:  dir-diff
 ----- exclude log ends
 FAIL    exclude
 PASS    executability
 PASS    files-from
 PASS    fuzzy
...

The 1902 timestamps indicate that somewhere a 32-bit time_t (UNIX epoch) value overflowed, even on x86_64.

[WayneD]

You'll note that all the bad timestamps are in the chk dir, not in the main copy. The exclude.test script was doing a protocol-28 compatibility test at one point in the test sequence, and protocol-28 can only handle 4-byte modtime values (so the high-bit mtime value gets interpreted as a negative number). I removed the protocol override from this dir-time touchup call and that fixes the test failure:

$RSYNC -av --protocol=28 --existing --include='*/' --exclude='*' "$fromdir/" "$chkdir/"                                         

Another change that would solve this issue when using an older protocol is to treat the arriving 4-byte mtime as unsigned. This would mean that the older protocol would not be able to convey a date prior to 1970 anymore, instead supporting upcoming dates through early 2106. I think this is probably a good compromise, given that (1) the older protocol is fairly rare (except on Apple), and (2) dates prior to 1970 are extremely rare (while newer dates are arriving in just 16 years). I'm considering this a bit more.

[WayneD]

I went ahead and made the 4-byte modtime change to unsigned since I think that this will better serve users who get forced into an old-protocol transfer in the coming years. I verified that "make check29" works just fine when run using a system time of Sept 2038.