testing: b.Loop gives bogus results in some situations #72971
Description
Go version
go version go1.24.0 linux/amd64
What did you do?
There are at least two user errors that testing.B.Loop could detect, but that currently lead to bogus benchmark results instead:
- Returning from a benchmark function before b.Loop() has returned false.
- Stopping the timer in the benchmark loop without starting it again.
What did you see happen?
Here are two examples:
package main
import "testing"
func BenchmarkBreak(b *testing.B) {
for i := 0; b.Loop(); i++ {
if i == 2 {
break
}
}
}
func BenchmarkStop(b *testing.B) {
for i := 0; b.Loop(); i++ {
b.StopTimer()
}
}With Go 1.24 (and tip as of this writing), these produce bogus results:
BenchmarkBreak-8 100 5.950 ns/op
BenchmarkStop-8 458912020 0.0000006 ns/op
With the fix for #72922, BenchmarkStop actually loops forever, until the whole test binary times out.
What did you expect to see?
I expected to see useful errors, and certainly not a timed out benchmark.