testing: weird AllocsPerRun return results

[go101]

What version of Go are you using (go version)?

$ go version
go version go1.16.6 linux/amd64
go version go1.17rc1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What did you do?

Moved from #20021 (comment)

package main

import t "testing"

const N = 1000 * 1000 * 1 // 513
var a [N]byte
var k int

func g(vs ...interface{}) {
	type _ int // avoid g being inlined
	_ = vs[k]
}

func main() {
	{
		x := t.AllocsPerRun(1, func() {
			g(a)
		})
		println(int(x))
	}
	{
		x := t.AllocsPerRun(1, func() {
			g(a, a)
		})
		println(int(x))
	}
	{
		x := t.AllocsPerRun(1, func() {
			g(a, a, a)
		})
		println(int(x))
	}
	{
		x := t.AllocsPerRun(1, func() {
			g(a, a, a, a)
		})
		println(int(x))
	}
	{
		x := t.AllocsPerRun(1, func() {
			g(a, a, a, a, a)
		})
		println(int(x))
	}
}

What did you expect to see?

1 2 3 4 5

What did you see instead?

1 4 3 4 5

If all are removed but the g(a, a, a) case, then it prints 5, which is also weird.

[go101]

The benchmark result of the following code is also weird in two points:

1. sometimes, it reports 8 B/op, sometimes 7 B/op.
2. it always reports 0 allocs/op, which is unreasonable (by considering B/op is not zero).

package main

import "testing"


var i interface{}
func foo(v int) {
	i = v
}

func Benchmark_Foo(b *testing.B) {
	for i := 0; i < b.N; i++ {
		foo(i)
	}
}

[cherrymui]

For the first case, I don't think 1 iteration is a good way to get the results you wanted. If you change it to (say) 1000 iterations, it gets 1 2 3 4 5 as expected.

For the second case it might have something to do with the tiny allocator. As you are allocating an int, it is tiny-allocated.

[randall77]

I think your second case has to do with the fact that we don't need to allocate when putting values 0-255 in an interface. See convT64 in the runtime.
Change your code to foo(i+256) and I always get 8 B/op and 1 alloc/op.
(Truncating to an integral number of bytes and allocs makes this look worse than it actually is.)

[cherrymui]

@go101 as you thumb-up'd the replies it seems your questions are answered. Anything else needs to be addressed on this issue? Thanks.

[go101]

For the 2nd case, if what @randall77 described is true, it would be better to use math.Round than math.Floor for the allocs/op calculation. (edit: or a floating point number with at most two decimal places)

[randall77]

We decided against rounding to nearest in #24631

[go101]

Okay. So I will close this issue soon.