Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: testing: panic if benchmark function tampers with b.N? #14479

Closed
minux opened this issue Feb 23, 2016 · 3 comments

Comments

Projects
None yet
6 participants
@minux
Copy link
Member

commented Feb 23, 2016

If the benchmark function changes b.N, the testing package will get very confused.

As an example, this (contrived) function:

func Benchmark(b *testing.B) {
    b.N = 1000000000
    time.Sleep(time.Second)
}

will lead to this:

Benchmark-4    1000000000               1.00 ns/op

I've seen this in the wild, for example:
https://groups.google.com/forum/#!topic/golang-nuts/DjKNjpSHi3M

@minux minux added the Proposal label Feb 23, 2016

@minux minux changed the title proposal: testing: detect if benchmark function tampers with b.N? proposal: testing: panic if benchmark function tampers with b.N? Feb 23, 2016

@cznic

This comment has been minimized.

Copy link
Contributor

commented Feb 23, 2016

There are probably other places in the stdlib, as well as in 3rd party packages, where mutating a struct field/package level variable, intended only for reading, will create mess. That's why I think some //go:readonly or similar annotation considered by go vet might be of more general use.

@dsymonds

This comment has been minimized.

Copy link
Member

commented Feb 24, 2016

On the surface, a vet check is plausible. On the other hand, if you're new enough to Go to consider setting b.N then you are likely to not even know about vet. I think the best vet checks are those that will catch things that even experienced Go programmers can make.

@bradfitz bradfitz modified the milestone: Unplanned Apr 7, 2016

@adg

This comment has been minimized.

Copy link
Contributor

commented Aug 15, 2016

These seems of extremely marginal utility, and once we go down this road we will spend more time being defensive than being productive.

@adg adg closed this Aug 15, 2016

@golang golang locked and limited conversation to collaborators Aug 15, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.