proposal: testing: panic if benchmark function tampers with b.N?

[minux]

If the benchmark function changes b.N, the testing package will get very confused.

As an example, this (contrived) function:

func Benchmark(b *testing.B) {
    b.N = 1000000000
    time.Sleep(time.Second)
}

will lead to this:

Benchmark-4    1000000000               1.00 ns/op

I've seen this in the wild, for example:
https://groups.google.com/forum/#!topic/golang-nuts/DjKNjpSHi3M

[cznic]

There are probably other places in the stdlib, as well as in 3rd party packages, where mutating a struct field/package level variable, intended only for reading, will create mess. That's why I think some //go:readonly or similar annotation considered by go vet might be of more general use.

[dsymonds]

On the surface, a vet check is plausible. On the other hand, if you're new enough to Go to consider setting b.N then you are likely to not even know about vet. I think the best vet checks are those that will catch things that even experienced Go programmers can make.

[adg]

These seems of extremely marginal utility, and once we go down this road we will spend more time being defensive than being productive.