$ go test -bench=NewEmptyMap -benchtime=5us -run=NONE runtime
BenchmarkNewEmptyMap-8 100 69.9 ns/op 0 B/op 0 allocs/op
$ go test -bench=NewEmptyMap -benchtime=2us -run=NONE runtime
BenchmarkNewEmptyMap-8 1 3490 ns/op 266824 B/op 1168 allocs/op
Observe that it's not just a question of getting enough iterations for the division to work: 1168 allocs / 100 iterations = 11 allocs/op, but it is correctly reported as 0 in the first run. Something somewhere is not getting reset correctly. (The iterations does explain the variance in ns/op, but not allocs/op.)
I noticed this with tiny benchtimes and the runtime, but this could also impact benchmarks that run for longer than a second.
Reproduces with 1.8, although it is much less extreme:
$ go test -bench=NewEmptyMap -benchtime=6us -run=NONE runtime
BenchmarkNewEmptyMap-8 200 52.7 ns/op 0 B/op 0 allocs/op
$ go test -bench=NewEmptyMap -benchtime=2us -run=NONE runtime
BenchmarkNewEmptyMap-8 1 4804 ns/op 2112 B/op 4 allocs/op
Unclear whether this counts as a regression or not.
cc @mpvl
Observe that it's not just a question of getting enough iterations for the division to work: 1168 allocs / 100 iterations = 11 allocs/op, but it is correctly reported as 0 in the first run. Something somewhere is not getting reset correctly. (The iterations does explain the variance in ns/op, but not allocs/op.)
I noticed this with tiny benchtimes and the runtime, but this could also impact benchmarks that run for longer than a second.
Reproduces with 1.8, although it is much less extreme:
Unclear whether this counts as a regression or not.
cc @mpvl