testing: benchmarks timing issue when using -count > 1 #25622
Description
@laboger reported in #25520 that 05ca340 introduced timing issues in the results of some benchmarks.
For example, the ChanNonblocking benchmark in runtime is much slower after the 1st iteration on ppc64le
BenchmarkChanNonblocking-16 1000000000 3.15 ns/op
BenchmarkChanNonblocking-16 1000000000 32.7 ns/op
The benchmark uses RunParallel. It seems the the change linked above may have introduced timing issues on benchmarks that use RunParallel. For example, using this:
package p_test
import (
"bytes"
"testing"
"text/template"
)
func BenchmarkF(b *testing.B) {
templ := template.Must(template.New("test").Parse("Hello, {{.}}!"))
b.RunParallel(func(pb *testing.PB) {
var buf bytes.Buffer
for pb.Next() {
buf.Reset()
templ.Execute(&buf, "World")
}
})
}
I see on go1.10:
BenchmarkF-4 10000000 188 ns/op
BenchmarkF-4 10000000 190 ns/op
BenchmarkF-4 10000000 189 ns/op
BenchmarkF-4 10000000 188 ns/op
BenchmarkF-4 10000000 189 ns/op
tip:
BenchmarkF-4 10000000 187 ns/op
BenchmarkF-4 10000000 198 ns/op
BenchmarkF-4 10000000 195 ns/op
BenchmarkF-4 10000000 195 ns/op
BenchmarkF-4 10000000 194 ns/op
notice how on tip every run after the 1st consistently reports an higher ns/op.