Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
testing: benchmark is using low resolution time on Windows #31160
As a demonstration here's code that measures
f:\Go\src\sandbox\benchmark>go version go version devel +2bd767b102 Thu Mar 28 09:36:43 2019 +0000 windows/amd64 f:\Go\src\sandbox\benchmark>go run main.go 50% 90% 99% time.Now Z=9999935 1000000 1005100 1014600 QPC Z=0 100 200 200
Which roughly says that my getting measurements at
Fixing nanotime would be problematic (as seen in #8687), however benchmarks could use QueryPerformanceCounter instead.
I could make an easy patch like I did in https://github.com/rakyll/hey/blob/master/requester/now_windows.go#L24. However since
As a more practical example:
As we vary benchtime it shows quite a significant timing error:
After poking it a bit it seems using
Usually the way to convert QPC value is
Ideas so far:
1 seems the smallest and easiest change. 2 requires adjusting things inside runtime... also, I'm not sure exactly how big the cost of additional division would be in practice. 3 seems slightly overkill, and I don't have a really good idea at the moment.
Based on these thoughts, my instincts tell me to go with 1.
I agree. I also see
I don't see why we cannot change nanotime to use
and it seems to be working fine.
I would not use
I would (if I had free time) try 2 or 3 first. Maybe division is not expensive enough? Can we do division in every 100's nanotime call to adjust for the drift?
Making runtime.nanotime and time.Now more precise might be beneficial in other areas (not just for benchmarks).
I am also worried about creating too many functions that read time. They all use different approach to collect time. How do you know the time they return is all in sync? And different implementations might diverge.
As far as I recall QPC is unreliable for measuring long periods of times.
Best reference mentioning issues I was able to find https://stackoverflow.com/a/5297163/192220. More in depth info on timing issues https://github.com/chromium/chromium/blob/08dbb44e81454b6d67c3b6f4989e7e58e88f4b0b/base/time/time_win.cc#L14. MSDN also only suggests it for measuring short periods of time https://docs.microsoft.com/en-us/windows/desktop/SysInfo/acquiring-high-resolution-time-stamps.
Yeah, I wouldn't use that value either. This is to demonstrate the make the issue more obvious.