Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: weird AllocsPerRun return results #47199

Closed
go101 opened this issue Jul 14, 2021 · 7 comments
Closed

testing: weird AllocsPerRun return results #47199

go101 opened this issue Jul 14, 2021 · 7 comments

Comments

@go101
Copy link

@go101 go101 commented Jul 14, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16.6 linux/amd64
go version go1.17rc1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What did you do?

Moved from #20021 (comment)

package main

import t "testing"

const N = 1000 * 1000 * 1 // 513
var a [N]byte
var k int

func g(vs ...interface{}) {
	type _ int // avoid g being inlined
	_ = vs[k]
}

func main() {
	{
		x := t.AllocsPerRun(1, func() {
			g(a)
		})
		println(int(x))
	}
	{
		x := t.AllocsPerRun(1, func() {
			g(a, a)
		})
		println(int(x))
	}
	{
		x := t.AllocsPerRun(1, func() {
			g(a, a, a)
		})
		println(int(x))
	}
	{
		x := t.AllocsPerRun(1, func() {
			g(a, a, a, a)
		})
		println(int(x))
	}
	{
		x := t.AllocsPerRun(1, func() {
			g(a, a, a, a, a)
		})
		println(int(x))
	}
}

What did you expect to see?

1 2 3 4 5

What did you see instead?

1 4 3 4 5

If all are removed but the g(a, a, a) case, then it prints 5, which is also weird.

@go101
Copy link
Author

@go101 go101 commented Jul 14, 2021

The benchmark result of the following code is also weird in two points:

  1. sometimes, it reports 8 B/op, sometimes 7 B/op.
  2. it always reports 0 allocs/op, which is unreasonable (by considering B/op is not zero).
package main

import "testing"


var i interface{}
func foo(v int) {
	i = v
}

func Benchmark_Foo(b *testing.B) {
	for i := 0; i < b.N; i++ {
		foo(i)
	}
}

Loading

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jul 14, 2021

For the first case, I don't think 1 iteration is a good way to get the results you wanted. If you change it to (say) 1000 iterations, it gets 1 2 3 4 5 as expected.

For the second case it might have something to do with the tiny allocator. As you are allocating an int, it is tiny-allocated.

Loading

@randall77
Copy link
Contributor

@randall77 randall77 commented Jul 14, 2021

I think your second case has to do with the fact that we don't need to allocate when putting values 0-255 in an interface. See convT64 in the runtime.
Change your code to foo(i+256) and I always get 8 B/op and 1 alloc/op.
(Truncating to an integral number of bytes and allocs makes this look worse than it actually is.)

Loading

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jul 15, 2021

@go101 as you thumb-up'd the replies it seems your questions are answered. Anything else needs to be addressed on this issue? Thanks.

Loading

@go101
Copy link
Author

@go101 go101 commented Jul 16, 2021

For the 2nd case, if what @randall77 described is true, it would be better to use math.Round than math.Floor for the allocs/op calculation. (edit: or a floating point number with at most two decimal places)

Loading

@randall77
Copy link
Contributor

@randall77 randall77 commented Jul 16, 2021

We decided against rounding to nearest in #24631

Loading

@go101
Copy link
Author

@go101 go101 commented Jul 16, 2021

Okay. So I will close this issue soon.

Loading

@go101 go101 closed this Jul 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants