os: TestChtimes fails on netbsd

[bradfitz]

The NetBSD builders are back up and running (now on GCE instead of dedicated machine singular) and are failing with:

ok  	net/url	0.009s
--- FAIL: TestChtimes (0.00s)
	os_test.go:1067: AccessTime didn't go backwards; was={486366670 63623730007 6480864}, after={486366670 63623730007 6480864}
--- FAIL: TestChtimesDir (0.00s)
	os_test.go:1067: AccessTime didn't go backwards; was={486603380 63623730007 6480864}, after={486603380 63623730007 6480864}
FAIL
FAIL	os	0.363s

(NetBSD 7.0)

/cc @bsiegert @mdempsky

[bradfitz]

NetBSD's stat(2) manpage says:

      st_atime   Time when file data was last accessed.  Changed by the
                 mknod(2), utimes(2), and read(2) system calls.

And the chtimes manpage says:

     The nanosecond fields for utimens(), lutimens(), futimens(), and
     utimensat() can be set to the special value UTIME_NOW to set the current
     time, or to UTIME_OMIT to let the time unchanged (this allows changing
     access time but not modification time, and vice-versa).

Those magic nanosecond values are:

https://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/sys/sys/stat.h

/*
 * Special values for utimensat and futimens
 */
#define UTIME_NOW	((1 << 30) - 1)
#define UTIME_OMIT	((1 << 30) - 2)
#endif

To change atime, do we need to do two system calls, once for modtime first, and then one for atime only with mtime set to UTIME_OMIT?

[gopherbot]

CL https://golang.org/cl/37492 mentions this issue.

[bradfitz]

I can no longer reproduce this with NetBSD 8-BETA, our new minimum version requirement for NetBSD.

So maybe it was fixed since we last saw it.

/cc @bsiegert

[bradfitz]

(If so, all that would remain here is removing the t.Skip in pkg os TestChtimes)

[gopherbot]

Change https://golang.org/cl/80875 mentions this issue: os: remove skipped test on netbsd

[bradfitz]

Reverting.

I was wrong. It still fails on the builders.

I don't know what's different about my VMWare VM, but on GCE it fails.

[bradfitz]

Oh, I figured it out. I tried the Timespec.Nsec UTIME_OMIT thing above, but this bug is actually because our builders run:

/sbin/mount -u -o async,noatime /

... at boot, to speed up builds. We do the same on almost all our Unix builders, with whatever spelling of the command they use.

On NetBSD, though, setting "noatime" seems to also disable explicit utimesat calls. Other BSDs seem to interpret "noatime" as only affecting the implicit atime updates from reading files.

I verified on my NetBSD VM:

localhost# go test -v -run=Chtimes
=== RUN   TestChtimes
--- PASS: TestChtimes (0.00s)
        os_test.go:1170: pre atime: 2017-11-30 19:19:15.960439676 -0800 PST
        os_test.go:1171: pre mtime: 2017-11-30 19:19:15.960610089 -0800 PST
        os_test.go:1186: post atime: 2017-11-30 19:19:14.960439676 -0800 PST (1512098354960439676)
        os_test.go:1187: post mtime: 2017-11-30 19:19:14.960610089 -0800 PST (1512098354960610089)
=== RUN   TestChtimesDir
--- PASS: TestChtimesDir (0.00s)
        os_test.go:1170: pre atime: 2017-11-30 19:19:15.961043733 -0800 PST
        os_test.go:1171: pre mtime: 2017-11-30 19:19:15.961043733 -0800 PST
        os_test.go:1186: post atime: 2017-11-30 19:19:14.961043733 -0800 PST (1512098354961043733)
        os_test.go:1187: post mtime: 2017-11-30 19:19:14.961043733 -0800 PST (1512098354961043733)
PASS
ok      os      0.004s
localhost# /sbin/mount -u -o async,noatime /
localhost# go test -v -run=Chtimes
=== RUN   TestChtimes
--- FAIL: TestChtimes (0.00s)
        os_test.go:1170: pre atime: 2017-11-30 19:19:22.214177347 -0800 PST
        os_test.go:1171: pre mtime: 2017-11-30 19:19:22.214352718 -0800 PST
        os_test.go:1186: post atime: 2017-11-30 19:19:22.214177347 -0800 PST (1512098362214177347)
        os_test.go:1187: post mtime: 2017-11-30 19:19:21.214352718 -0800 PST (1512098361214352718)
        os_test.go:1198: AccessTime didn't go backwards; was=2017-11-30 19:19:22.214177347 -0800 PST, after=2017-11-30 19:19:22.214177347 -0800 PST
=== RUN   TestChtimesDir
--- FAIL: TestChtimesDir (0.00s)
        os_test.go:1170: pre atime: 2017-11-30 19:19:22.214812902 -0800 PST
        os_test.go:1171: pre mtime: 2017-11-30 19:19:22.214812902 -0800 PST
        os_test.go:1186: post atime: 2017-11-30 19:19:22.214812902 -0800 PST (1512098362214812902)
        os_test.go:1187: post mtime: 2017-11-30 19:19:21.214812902 -0800 PST (1512098361214812902)
        os_test.go:1198: AccessTime didn't go backwards; was=2017-11-30 19:19:22.214812902 -0800 PST, after=2017-11-30 19:19:22.214812902 -0800 PST
FAIL
exit status 1
FAIL    os      0.004s

So I guess I'll just skip that test on NetBSD and close this bug.

@bsiegert, @krytarowski, @zoulasc, I assume that's expected behavior, but let me know if not.

[gopherbot]

Change https://golang.org/cl/81355 mentions this issue: os: ignore Chtimes test failure on NetBSD if fs mounted noatime

[krytarowski]

Hmm, From a quick look I don't see a relation between MNT_NOATIME and lack of precise time.

I think we should update to syscalls from NetBSD 8.0 (so switch to: __stat50). There were changes in time functions like extending to 64-bit time_t. At some point we have added POSIX interfaces. Some subtle bugs can go from mixing old syscalls (they appear to be from NetBSD 5.) and assume behavior from 8.0.

[bradfitz]

@krytarowski, this isn't about precise time. Ignore the nanoseconds part. The key part is:

On NetBSD, though, setting "noatime" seems to also disable explicit utime* calls. Other BSDs seem to interpret "noatime" as only affecting the implicit atime updates from reading files.