Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
cmd/go: 'go test' should dynamically compute best timeout #12446
I discovered that "go test" crashes after the timeout of 600s has passed. I was running all the benchmarks in
In situations like these, the
One possible timeout is:
The 5x increase was chosen since it seems the benchmark follow the following progression: 1x, 2x, 5x, 10x, 20x, 50x, etc. Thus, at worst, the last iteration of a Benchmark may run for 2.5x longer than targeted time. Also, since the summation of all the iterations of a given Benchmark resembles a geometric series where r=1/2, we get a another factor of 2x. Thus, 2*2.5 = 5x.
referenced this issue
Oct 4, 2016
I don't think this bug should be addressed. That is to say, I don't think we should change the go tool as this will enable people to continue to do the wrong thing.
To accurately benchmark you should have a before and after sample and run those both several times back to back to rule out thermal variation. This is what the -c flag gives you, a binary that you can run with whatever timeout and count flags you need on dedicated benchmarking hardware.
Working around this inside go test encourages people to benchmark on shared cloud Ci hardware, which is inaccurate.