cmd/go: add go test -skip to skip specific tests

[dprotaso]

t.Skip() is an option but it requires changing the test code.

It's hard to write a correct regex to pass to the -run flag in order to skip specific tests. This is due to the lack of lookaheads in the regexp library. Previous discussion here: https://groups.google.com/g/golang-nuts/c/7qgSDWPIh_E?pli=1

[ianlancetaylor]

Do you have a specific suggestion?

To run specific tests you can already write go test -run="TestOne|TestTwo|TestThree".

[dprotaso]

Do you have a specific suggestion?

I didn't want to prescribe a solution but here's some off the top of my head

1. expand run flag functionality - -run [regex] to allow look aheads - thus negation

   • pros
     • most people are familiar with this flag
   • cons
     • requires expanded regex implementation

2. new flag -skip [regex]

   • pros
     • mirrors run flag
     • may be easier to use than a negative regex
   • cons
     • requires definition of precedence between -skip& -run
     • it'd be nice to see skipped test output - that might require more work to implement

To run specific tests you can already write go test -run="TestOne|TestTwo|TestThree".

This doesn't scale if I have O(10s) of tests in a package and want to skip one or two

[dprotaso]

hey @ianlancetaylor - do you have any additional questions or want me to clarify anything above?

[ianlancetaylor]

No, I'm good.

This issue is in the proposal process now and will get into the proposal committee in due course. The committee looks at a lot of issues (see minutes at #33502), so it won't be immediately, but it shouldn't take too long.

Thanks.

[rsc]

Almost no one on earth understands how to write negation regexps correctly, so I would lean toward -skip if we do this.

[bcmills]

requires definition of precedence between -skip & -run

I can see only two reasonable definitions.

1. Define that the default value of -run matches everything, and therefore -skip takes precedence over -run.
2. Define that the default value of -run imposes no constraints, and therefore it is an error to specify values of -run and -skip that both match the same test.

I have a slight preference for (1), because it treats the default value of the -run flag the same as other values.

[jimmyfrasche]

Does precedence mean the order the filters are applied in or which is used if both are given?

If it's the order, then -run followed by -skip makes the most sense to me and seems more useful than disallowing the combination.

[bcmills]

Yeah, it seems especially useful in the context of subtests:

go test foo -run=TestBar -skip=TestBar/flaky_subtest