Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: limit the number of errors #12276

Closed
josharian opened this issue Aug 23, 2015 · 6 comments

Comments

Projects
None yet
3 participants
@josharian
Copy link
Contributor

commented Aug 23, 2015

Exhaustive tests (such as found in math/big and strconv) can generate a phenomenal number of errors when they fail. At some point, there is no value in additional errors.

I think that we should maintain an error counter in testing.T and call t.Fatal("too many errors") once we hit (say) 100 errors in a test.

cc @adg and @robpike for feedback. I'll implement if you're ok with it, since this is a pain point for SSA development.

@robpike

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2015

go test | sed 100q seems good enough to me. If you add features like this, you need features to control the features, new features are justified because they are in the same style, and they need more features to control them, and it never stops.

@josharian

This comment has been minimized.

Copy link
Contributor Author

commented Aug 23, 2015

Ok. FTR, on digging, a large part of the pain is that I'm running with GOGC=off (by necessity) and all the output is buffered (not fixed by sed), which ends up causing massive memory usage.

@josharian josharian closed this Aug 23, 2015

@robpike

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2015

The place to fix that is certainly not with new flags in the testing package.

@josharian

This comment has been minimized.

Copy link
Contributor Author

commented Aug 23, 2015

I didn't have in mind new flags, FWIW, just an unexported counter field in testing.T and something at the beginning of Error(f) like

t.nerr++
if t.nerr > 1000 {
  t.Fatal("too many errors")
}

But it's not really worth more words.

@robpike

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2015

But you know someone will then ask to have a flag to change that 1000. Where to draw the line?

@josharian

This comment has been minimized.

Copy link
Contributor Author

commented Aug 23, 2015

Yep. :/

@golang golang locked and limited conversation to collaborators Aug 22, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.