testing: allocs not reset for single-cycle runs

[josharian]

$ go test -bench=NewEmptyMap -benchtime=5us -run=NONE runtime
BenchmarkNewEmptyMap-8   	     100	        69.9 ns/op	       0 B/op	       0 allocs/op
$ go test -bench=NewEmptyMap -benchtime=2us -run=NONE runtime
BenchmarkNewEmptyMap-8   	       1	      3490 ns/op	  266824 B/op	    1168 allocs/op

Observe that it's not just a question of getting enough iterations for the division to work: 1168 allocs / 100 iterations = 11 allocs/op, but it is correctly reported as 0 in the first run. Something somewhere is not getting reset correctly. (The iterations does explain the variance in ns/op, but not allocs/op.)

I noticed this with tiny benchtimes and the runtime, but this could also impact benchmarks that run for longer than a second.

Reproduces with 1.8, although it is much less extreme:

$ go test -bench=NewEmptyMap -benchtime=6us -run=NONE runtime
BenchmarkNewEmptyMap-8   	     200	        52.7 ns/op	       0 B/op	       0 allocs/op
$ go test -bench=NewEmptyMap -benchtime=2us -run=NONE runtime
BenchmarkNewEmptyMap-8   	       1	      4804 ns/op	    2112 B/op	       4 allocs/op

Unclear whether this counts as a regression or not.

cc @mpvl

[gopherbot]

CL https://golang.org/cl/46612 mentions this issue.

[bradfitz]

Has a CL, so promoting to non-maybe.