time: Now() faster than reported timestamps from filesystem

[taoyuanyuan]

What version of Go are you using (go version)?

$ go version go1.12.6 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

linux 3.10.0
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/lieyuan/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/lieyuan/gogo"
GOPROXY=""
GORACE=""
GOROOT="/home/lieyuan/go"
GOTMPDIR=""
GOTOOLDIR="/home/lieyuan/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build567923190=/tmp/go-build -gno-record-gcc-switches"

What did you do?

time.Now() faster than current system time

package main

import (
    "fmt"
    "os"
    "time"
)

func main() {
    for {
        time.Sleep(900*time.Millisecond)
        os.Remove("/tmp/1")
        var now time.Time
        for {
            now = time.Now()
            if now.Nanosecond() == 0 {
                break
            }
        }
        file, _ := os.OpenFile("/tmp/1", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
        stat, _ := file.Stat()
        if now.Second() > stat.ModTime().Second() {
            fmt.Println(now.Unix(), stat.ModTime().Unix(), time.Now().Unix())
            return
        }
    }
}

output:

1565151693 1565151692 1565151693

stat file:

$ stat /tmp/1
  File: ‘/tmp/1’
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 803h/2051d	Inode: 1703974     Links: 1
Access: (0644/-rw-r--r--)  Uid: (50051/ lieyuan)   Gid: (  100/   users)
Access: 2019-08-07 12:21:32.998566664 +0800
Modify: 2019-08-07 12:21:32.998566664 +0800
Change: 2019-08-07 12:21:32.998566664 +0800
 Birth: -

$ date -d @1565151693
Wed Aug  7 12:21:33 CST 2019

What did you expect to see?

time.Now() returns the current system time

What did you see instead?

time.Now() faster than current system time

[odeke-em]

Hello @taoyuanyuan, thank you for filing this issue!

In between the times reported, we need to account for those missing 1433.25 microseconds.
My biggest suspicion is that perhaps that difference arises because os.OpenFile invokes the "open" syscall which in its implementation uses nanoTime and doesn't account for monotonic time yet Go uses the monotonic time whenever it can.

I suggest that perhaps if such file time comparisons are tripping up your workflow, that perhaps you invoke .Round(time.Second) or after creating a file, you can invoke os.Chtimes https://golang.org/pkg/os/#Chtimes with the desired time rounded.

I can reproduce this too on my Linux box on the first try.

Kindly pinging @ianlancetaylor @ALTree @randall77 @aclements to help check over my hypothesis/suggestion and feel free to correct me.

[odeke-em]

@taoyuanyuan also it would be great to fill out the issue template as much and perhaps provide more information about your setup e.g. Linux version, distribution, what filesystem is being used etc. Thank you!

Never mind, it was just hidden in the HTML details, sorry for the false alarm asking you to fill out the template

[randall77]

I get the same behavior from a C program:

#include <stdio.h>
#include <time.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>

int main(int argc, char* argv[]) {
  while(1) {
    usleep(900000);
    remove("/tmp/1");
    struct timespec now;
    while(1) {
      clock_gettime(CLOCK_REALTIME, &now);
      if (now.tv_nsec == 0) { break; }
    }

    int fd = open("/tmp/1", O_RDWR|O_CREAT|O_APPEND, 0644);
    struct stat s;
    fstat(fd, &s);

    if (now.tv_sec > s.st_mtim.tv_sec) {
      printf("%d %d\n", now.tv_sec, s.st_mtim.tv_sec);
    }
  }
}

Prints after some time:

1565196323 1565196322

So I don't think there's any bug here. Or if there is, it is the kernel not Go.

[ianlancetaylor]

This does seem to be how the Linux kernel works. The implementation of clock_gettime(CLOCK_REALTIME, x), which is what the Go runtime calls for time.Now, makes sure to get a fully synchronized time. The implementation of the kernel internal current_fs_time, which is used to set the time in a newly created inode, doesn't bother. So on a multicore system the two times can be very slightly out of sync.

Closing because there is nothing we can do to fix this in the Go toolchain.