New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/go: add go test -skip to skip specific tests #41583
Comments
go test
CLIgo test
CLI
go test
CLIgo test
CLI
Do you have a specific suggestion? To run specific tests you can already write |
go test
CLIgo test
CLI
I didn't want to prescribe a solution but here's some off the top of my head
This doesn't scale if I have O(10s) of tests in a package and want to skip one or two |
hey @ianlancetaylor - do you have any additional questions or want me to clarify anything above? |
No, I'm good. This issue is in the proposal process now and will get into the proposal committee in due course. The committee looks at a lot of issues (see minutes at #33502), so it won't be immediately, but it shouldn't take too long. Thanks. |
Almost no one on earth understands how to write negation regexps correctly, so I would lean toward -skip if we do this. |
I can see only two reasonable definitions.
I have a slight preference for (1), because it treats the default value of the |
Does precedence mean the order the filters are applied in or which is used if both are given? If it's the order, then |
Yeah, it seems especially useful in the context of subtests:
|
The latter in my opinion |
I have almost moved some subtests into their own top level test function on a few occasions exactly because there was not a good way to skip some subset of them. I have also had to run |
It sounds like people are generally in favor of adding Do I have that right? Does anyone object to adding |
go test
CLI
Hmm. It would be unfortunate to have an easy way to skip specific tests but not specific benchmarks. I think it would make sense to either have |
Sounds good to me
Expanding the scope of |
A way to skip benchmarks would be welcome as well. The |
Instead of introducing a new flag, could we add some syntax to the existing flag? Let a pattern starting with This avoids introducing new flags ( |
Truthfully I was tripped up by this when I first encountered it |
@magical, further complications in the syntax for the |
@ChrisHines, I think the question is, would someone running I would argue that pretty much everyone would expect that to skip the sub-benchmark, so I suspect that a unified |
FWIW this is usually a bad idea, at least interactively, since you inevitably end up benchmarking broken code. (And boy can you make broken code run fast!) |
@mvdan This is in the Go 1.18 milestone. Is it likely to happen for 1.18? Thanks. |
@ianlancetaylor thanks for the ping; I intend to pick up the patch again, but the change is too invasive for 1.18 at this point even if I finished the change this week. |
Would love to see this land! |
Would love to be able to do this to disable flaky tests in CI. |
I still intend to get to this for 1.19. There's no more need for more comments to just say it would be nice to have :) See https://github.com/golang/go/wiki/NoPlusOne. |
Can you make it support skipping multiple tests from different packages? Like:
It would be useful to disable flaky tests in different packages on CI |
Currently, there's no good way for any other project consuming these tests to selectively skip any of them. Maybe there will be [as of go 1.19](golang/go#41583), but until then, this introduces a new "skipTests" argument that allows users to pass in test names to skip.
@linzhp that sounds like it should be a separate issue or proposal, because it should also affect the |
Any plan to implement this? It seems not in Go 1.19 yet |
I clearly did not get to this for 1.19, so I'm unassigning myself for now. There is no plan or estimate right now; the issue is in the backlog milestone. |
Change https://go.dev/cl/421439 mentions this issue: |
Not entirely sure whether that was a coincidence :) |
Woah, next release can I finally stop |
Change https://go.dev/cl/449075 mentions this issue: |
Updates #41696. Updates #50332. Updates #41583. Change-Id: I99e96a2996f14da262570a5cb5273dcdce45df2b Reviewed-on: https://go-review.googlesource.com/c/go/+/449075 Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com>
t.Skip()
is an option but it requires changing the test code.It's hard to write a correct regex to pass to the
-run
flag in order to skip specific tests. This is due to the lack of lookaheads in the regexp library. Previous discussion here: https://groups.google.com/g/golang-nuts/c/7qgSDWPIh_E?pli=1The text was updated successfully, but these errors were encountered: