Repository navigation
testing: benchmark is using low resolution time on Windows #31160
Description
Activity
- changed the title
[-]testing: benchmark is using time.Now on Windows for measuring time.[/-][+]testing: benchmark is using low resolution time on Windows for measuring time.[/+]on Mar 30, 2019 I could make an easy patch like I did in https://github.com/rakyll/hey/blob/master/requester/now_windows.go#L24. However since
runtimealready contains definitions for QPC was wondering whether it can be organized better.As a more practical example:
func Benchmark(b *testing.B) { for i := 0; i < b.N; i++ { fmt.Sprintf("hello world") } }As we vary benchtime it shows quite a significant timing error:
f:\Go\src\sandbox\benchmark>go test -bench . -benchtime 1000x . goos: windows goarch: amd64 pkg: sandbox/benchmark Benchmark-32 1000 0.00 ns/op PASS ok sandbox/benchmark 0.261s f:\Go\src\sandbox\benchmark>go test -bench . -benchtime 100000x . goos: windows goarch: amd64 pkg: sandbox/benchmark Benchmark-32 100000 80.4 ns/op PASS ok sandbox/benchmark 0.285s f:\Go\src\sandbox\benchmark>go test -bench . -benchtime 1000000x . goos: windows goarch: amd64 pkg: sandbox/benchmark Benchmark-32 1000000 67.0 ns/op PASS ok sandbox/benchmark 0.342s- changed the title
[-]testing: benchmark is using low resolution time on Windows for measuring time.[/-][+]testing: benchmark is using low resolution time on Windows[/+]on Mar 30, 2019 Related: #29714
CC @mpvl @josharian
Ping.
I could make a PR and discuss a specific fix, if that's requires less effort from core maintainers.
I don't have much to say about this. A higher resolution benchmarking timer certainly seems desirable. What are the downsides?
cc @alexbrainman for windows
Some duplicated code for windows and handling for other platforms to be compatible with that.
Specifically the
qpcpart. Unless there's a nice way to share that code withLine 381 in 9b6e9f0
testingpackage.We could use //go:linkname to get access to it. Or an internal package, perhaps.
After poking it a bit it seems using
nanotimeQPCat the moment works only for Wine, since the implementation converts a division into multiplication that doesn't work in the general case.Usually the way to convert QPC value is
result = qpcCounter * 1e9 / qpcFrequency. Currently the implementation usesqpcMultiplier = 1e9 / qpcFrequencyand thenresult = qpcCounter * qpcMultiplier, which only works correctly when frequency is a power of 10.https://github.com/golang/go/blob/master/src/runtime/os_windows.go#L447
Ideas so far:
- add QPC funcs into
internal/syscall/windowsand use them rather than try to modify runtime to fit the needs - use division inside
nanotimeQPCto avoid the error and accept the cost of division - find a faster way to calculate division by qpcFrequency, if there exists one
1 seems the smallest and easiest change. 2 requires adjusting things inside runtime... also, I'm not sure exactly how big the cost of additional division would be in practice. 3 seems slightly overkill, and I don't have a really good idea at the moment.
Based on these thoughts, my instincts tell me to go with 1.
- add QPC funcs into
10 remaining items
Change https://go.dev/cl/557315 mentions this issue:
testing: use QueryPerformanceCounter on Windows- added a commit that references this issue
on Apr 26, 2024 I'm fine with reverting the CL to avoid blocking other work, but I don't think that's the underlying issue. If there's no immediate fix, then we can revert to get things building again.
I agree. Let's wait for others before we create a revert. We cannot submit the revert anyways without Googlers.
Anyways, let's continue the discussion in #67077.
SGTM.
Alex
Change https://go.dev/cl/582135 mentions this issue:
testing: use time.Since in highPrecisionTime- added a commit that references this issue
on Apr 27, 2024 - locked and limited conversation to collaborators
on Apr 27, 2025
time.Nowhas a large resolution on Windows, so the benchmark results end up with a timing error.As a demonstration here's code that measures
QueryPerformanceCountervs.time.Now: https://play.golang.org/p/FN4AtJ9b51m.f:\Go\src\sandbox\benchmark>go version go version devel +2bd767b102 Thu Mar 28 09:36:43 2019 +0000 windows/amd64 f:\Go\src\sandbox\benchmark>go run main.go 50% 90% 99% time.Now Z=9999935 1000000 1005100 1014600 QPC Z=0 100 200 200Which roughly says that my getting measurements at
1000000nsor1msgranularity.Fixing nanotime would be problematic (as seen in #8687), however benchmarks could use QueryPerformanceCounter instead.