-
Notifications
You must be signed in to change notification settings - Fork 18.5k
Description
When watchflakes encounters a new failure mode that none of the existing issues in the Test Flakes project match, it creates a new issue for it. When it creates the issue, it creates a pattern for that package path and test name, to match future instances of the same failure.
It seems that fairly often there may be a common failure that ends up spanning across multiple subtests, and sometimes also across package:variant, and perhaps it'd be a better default for watchflakes to create a single new issue to match all of them.
Some recent examples where watchflakes ends up filing many similar reports across subtests as separate issues:
# new issue: cmd/go/internal/modfetch/codehost: TestReadZip/hgrepo1/v3/ failures
# new issue: cmd/go/internal/modfetch/codehost: TestReadZip/hgrepo1/v2/ failures
# new issue: cmd/go/internal/modfetch/codehost: TestReadZip/hgrepo1/v3/v3/sub failures
# new issue: cmd/go/internal/modfetch/codehost: TestReadZip/hgrepo1/v3/v3/sub/dir failures
# new issue: x/tools/gopls/internal/test/marker: Test/completion/unimported.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/rename/shadow.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/mcptools/context.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/completion/func_value.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/completion/deep.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/workspacesymbol/allscope.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/codeaction/functionextraction_issue73972.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/mcptools/file_diagnostics.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/configuration/static.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/completion/address.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/codeaction/moveparam.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/workfile/godebug.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/definition/asm.txt failures
A recent example where package:variant should likely also be grouped together, instead of being separate per-variant issues:
# new issue: runtime: TestAppendByteCapInLoop failures
# new issue: runtime:cpu1: TestAppendByteCapInLoop failures
# new issue: runtime:cpu2: TestAppendByteCapInLoop failures
In cases where it does make sense to separate reports by specific subtest or specific failure modes, humans can narrow down the pattern as needed. This is only about what watchflakes files by default.
CC @golang/release.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status