Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Cop: group issues for related tests when it makes sense #258

Closed
tbpg opened this issue Jan 30, 2020 · 5 comments
Closed

Build Cop: group issues for related tests when it makes sense #258

tbpg opened this issue Jan 30, 2020 · 5 comments
Assignees
Labels
bot: flakybot type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@tbpg
Copy link
Contributor

tbpg commented Jan 30, 2020

Is your feature request related to a problem? Please describe.
If 10 Go subtests of a single test all fail, you get 10 separate issues filed at the same time.

Describe the solution you'd like
Group subtests into a single issue. We could do this by splitting the test name on "/" and only keying on the first part. Not sure how this would affect other languages.

Describe alternatives you've considered
Keep all of the issues separate. This is very noisy.

Additional context
https://github.com/GoogleCloudPlatform/golang-samples/issues?utf8=%E2%9C%93&q=is%3Aissue+%22bigquery%2Fsnippets%2Fquerying%3A+TestQueries%22+label%3A%22buildcop%3Aissue%22

@tbpg tbpg added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Jan 30, 2020
@tbpg tbpg self-assigned this Jan 30, 2020
@tbpg tbpg changed the title Build Cop: group issues for Go subtests Build Cop: group issues for related tests when it makes sense Feb 10, 2020
@tbpg
Copy link
Contributor Author

tbpg commented Feb 13, 2020

cc @fhinkel

@tbpg
Copy link
Contributor Author

tbpg commented Mar 23, 2020

I queried to get all issues for test cases with a slash in the name:

title,test_case,repo_url
"""bigquery/snippets/querying: TestQueries/group/createTableAndWidenQuery failed""",TestQueries/group/createTableAndWidenQuery,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryBatch failed""",TestQueries/group/queryBatch,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryClusteredTable failed""",TestQueries/group/queryClusteredTable,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryDisableCache failed""",TestQueries/group/queryDisableCache,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""profiler: TestAgentIntegration/profiler-test-go11113 failed""",TestAgentIntegration/profiler-test-go11113,"""https://api.github.com/repos/googleapis/google-cloud-go"""
"""bigquery/snippets/querying: TestQueries/group/queryWithNamedParams failed""",TestQueries/group/queryWithNamedParams,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryWithStructParam failed""",TestQueries/group/queryWithStructParam,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryDryRun failed""",TestQueries/group/queryDryRun,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryWithDestinationCMEK failed""",TestQueries/group/queryWithDestinationCMEK,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryWithTimestampParam failed""",TestQueries/group/queryWithTimestampParam,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""profiler: TestAgentIntegration/profiler-test-gomaster failed""",TestAgentIntegration/profiler-test-gomaster,"""https://api.github.com/repos/googleapis/google-cloud-go"""
"""profiler: TestAgentIntegration/profiler-test-go1138 failed""",TestAgentIntegration/profiler-test-go1138,"""https://api.github.com/repos/googleapis/google-cloud-go"""
"""bigquery/snippets/querying: TestQueries/group/queryLegacyLargeResults failed""",TestQueries/group/queryLegacyLargeResults,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryWithDestination failed""",TestQueries/group/queryWithDestination,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryLegacy failed""",TestQueries/group/queryLegacy,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryWithPositionalParams failed""",TestQueries/group/queryWithPositionalParams,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""profiler: TestAgentIntegration/profiler-test-go114 failed""",TestAgentIntegration/profiler-test-go114,"""https://api.github.com/repos/googleapis/google-cloud-go"""
"""bigquery/snippets/querying: TestQueries/group/queryBasic failed""",TestQueries/group/queryBasic,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group failed""",TestQueries/group,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""profiler: TestAgentIntegration/profiler-test-go11217 failed""",TestAgentIntegration/profiler-test-go11217,"""https://api.github.com/repos/googleapis/google-cloud-go"""
"""bigquery/snippets/querying: TestQueries/group/queryPartitionedTable failed""",TestQueries/group/queryPartitionedTable,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""
"""bigquery/snippets/querying: TestQueries/group/queryWithArrayParams failed""",TestQueries/group/queryWithArrayParams,"""https://api.github.com/repos/GoogleCloudPlatform/golang-samples"""

Going to write a shortener for this case. This will not cover the python-docs-samples case (updated link: https://github.com/GoogleCloudPlatform/python-docs-samples/issues?q=is%3Aissue+label%3A%22buildcop%3A+issue%22+dlp.inspect_content_test+) since that does not need to be grouped by the test case name -- the package name somehow needs to be pulled apart.

@tbpg
Copy link
Contributor Author

tbpg commented Apr 30, 2020

@tmatsuo, do you think this is something we should change in the Build Cop Bot? Or are we good to close this for now?

@tbpg
Copy link
Contributor Author

tbpg commented May 15, 2020

Closing this for now. Filed #550 for a more specific case.

@tbpg tbpg closed this as completed May 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bot: flakybot type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

1 participant