Skip to content

x/build/cmd/watchflakes: default to creating new issues per test (instead of per subtest) and per package (instead of per package:variant) #76587

@dmitshur

Description

@dmitshur

When watchflakes encounters a new failure mode that none of the existing issues in the Test Flakes project match, it creates a new issue for it. When it creates the issue, it creates a pattern for that package path and test name, to match future instances of the same failure.

It seems that fairly often there may be a common failure that ends up spanning across multiple subtests, and sometimes also across package:variant, and perhaps it'd be a better default for watchflakes to create a single new issue to match all of them.

Some recent examples where watchflakes ends up filing many similar reports across subtests as separate issues:

# new issue: cmd/go/internal/modfetch/codehost: TestReadZip/hgrepo1/v3/ failures
# new issue: cmd/go/internal/modfetch/codehost: TestReadZip/hgrepo1/v2/ failures
# new issue: cmd/go/internal/modfetch/codehost: TestReadZip/hgrepo1/v3/v3/sub failures
# new issue: cmd/go/internal/modfetch/codehost: TestReadZip/hgrepo1/v3/v3/sub/dir failures

# new issue: x/tools/gopls/internal/test/marker: Test/completion/unimported.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/rename/shadow.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/mcptools/context.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/completion/func_value.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/completion/deep.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/workspacesymbol/allscope.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/codeaction/functionextraction_issue73972.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/mcptools/file_diagnostics.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/configuration/static.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/completion/address.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/codeaction/moveparam.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/workfile/godebug.txt failures
# new issue: x/tools/gopls/internal/test/marker: Test/definition/asm.txt failures

A recent example where package:variant should likely also be grouped together, instead of being separate per-variant issues:

# new issue: runtime: TestAppendByteCapInLoop failures
# new issue: runtime:cpu1: TestAppendByteCapInLoop failures
# new issue: runtime:cpu2: TestAppendByteCapInLoop failures

In cases where it does make sense to separate reports by specific subtest or specific failure modes, humans can narrow down the pattern as needed. This is only about what watchflakes files by default.

CC @golang/release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Buildersx/build issues (builders, bots, dashboards)FeatureRequestIssues asking for a new feature that does not need a proposal.FrictionNuisances that make good candidates for our "friction" fix-it weeksNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.

    Type

    No type

    Projects

    Status

    No status

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions