In CL 366176 (for #36108), I increased the timeout slop in net.TestWriteTimeoutFluctuation and net.TestReadTimeoutFluctuation to 33% even for tests with very generous (multiple-second) timeouts. At that scale, that margin of slop should be trivial for even a heavily-loaded builder to hit.
Unfortunately, the NetBSD and OpenBSD builders still do not reliably hit it, even on an n1 instance that does not appear to be affected by #49209.
Given the other issues we've had with NetBSD and OpenBSD, I suspect a kernel bug. I plan to further raise the slop for those two platforms, narrow it everywhere else, and call it at that without investigating further. However, I suggest that folks who care about these kernels (@bsiegert, @coypoop, @4a6f656c?) may want to look into whether the underlying system calls may be adding unnecessary slop in their timeout handling.
greplogs --dashboard -md -l -e 'FAIL: TestWriteTimeoutFluctuation' --since=2021-11-23
2021-12-13T16:09:22-d198a36/openbsd-amd64-70-n2d
2021-11-29T19:45:58-f598e29/netbsd-amd64-9_0-n1
2021-11-29T16:08:23-a59ab29/openbsd-amd64-68
2021-11-25T00:07:28-f7e34e7/openbsd-amd64-70-n1
2021-11-25T00:07:11-c58243a/openbsd-amd64-70-n1
In CL 366176 (for #36108), I increased the timeout slop in
net.TestWriteTimeoutFluctuationandnet.TestReadTimeoutFluctuationto 33% even for tests with very generous (multiple-second) timeouts. At that scale, that margin of slop should be trivial for even a heavily-loaded builder to hit.Unfortunately, the NetBSD and OpenBSD builders still do not reliably hit it, even on an
n1instance that does not appear to be affected by #49209.Given the other issues we've had with NetBSD and OpenBSD, I suspect a kernel bug. I plan to further raise the slop for those two platforms, narrow it everywhere else, and call it at that without investigating further. However, I suggest that folks who care about these kernels (@bsiegert, @coypoop, @4a6f656c?) may want to look into whether the underlying system calls may be adding unnecessary slop in their timeout handling.
greplogs --dashboard -md -l -e 'FAIL: TestWriteTimeoutFluctuation' --since=2021-11-232021-12-13T16:09:22-d198a36/openbsd-amd64-70-n2d
2021-11-29T19:45:58-f598e29/netbsd-amd64-9_0-n1
2021-11-29T16:08:23-a59ab29/openbsd-amd64-68
2021-11-25T00:07:28-f7e34e7/openbsd-amd64-70-n1
2021-11-25T00:07:11-c58243a/openbsd-amd64-70-n1