Skip to content

ci: only run bulk units for material changes #12332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
blakeli0 merged 4 commits into from
Apr 2, 2026
Merged

ci: only run bulk units for material changes #12332

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4d41e8e
ci: only run bulk units if stuff has materially changed
chingor13 Mar 31, 2026
e64069b
put skip in steps
chingor13 Mar 31, 2026
e115757
also run tests if ci is touched
chingor13 Mar 31, 2026
b55fca6
add comments
chingor13 Mar 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,33 @@ on:
pull_request:
name: ci
jobs:
# detect whether or note we should run "bulk" (non-handwritten) unit tests
Copy link
Copy Markdown
Member

@lqiu96 lqiu96 Mar 31, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, is bulk the same as all the generated unit tests for all the GAPICs? If so, do you think it would it be better for us to just run the gapic-showcase tests instead (theoretically it should cover all the GAPICs)?

Copy link
Copy Markdown
Contributor Author

@chingor13 chingor13 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"bulk" is everything previously in the monorepo prior to the migration effort (so effectively gapic generated clients). There are some other libraries in here that may have handwritten code or have separate testing (dns, compute?, containeranalysis). We could also opt to treat those few special cases like we do the split repos that we migrated in.

Copy link
Copy Markdown
Member

@lqiu96 lqiu96 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer this to you then. I think it could (?) be something like these categories:

  1. Showcase for all GAPIC coverage
  2. Handwrittens
  3. Special cases in the monorepo (DNS, Compute, etc)

Copy link
Copy Markdown
Contributor Author

@chingor13 chingor13 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these categories make sense. We can treat the special case monorepo ones the same way as handwrittens

bulk-filter:
runs-on: ubuntu-latest
permissions:
pull-requests: read
# Set job outputs to values from filter step
outputs:
src: ${{ steps.filter.outputs.src }}
ci: ${{ steps.filter.outputs.ci }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
id: filter
with:
# we want to run tests if source code is changed or the scripts
# used to run the unit tests
filters: |
src:
- '**/*.java'
Copy link
Copy Markdown
Contributor

@blakeli0 blakeli0 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So a bigquery change would trigger bulk units too?

Copy link
Copy Markdown
Contributor Author

@chingor13 chingor13 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So a bigquery change would trigger bulk units too?

Potentially, another library depends on bigquery. We could possibly get more explicit here on paths

- '**/pom.xml'
ci:
- '.github/workflows/ci.yaml'
- '.kokoro/**'
Copy link
Copy Markdown
Contributor

@blakeli0 blakeli0 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this filter is only for units, why do we want kokoro changes to trigger it?

Copy link
Copy Markdown
Contributor

@blakeli0 blakeli0 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I see that units and integration tests share the same scripts in .kokoro. Maybe we can split them, but that's for a different PR.

Copy link
Copy Markdown
Contributor Author

@chingor13 chingor13 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this filter is only for units, why do we want kokoro changes to trigger it?

We run .kokoro/build.sh and use helpers in .kokoro/ folder

# these unit tests are "bulk" (non-handwritten) libraries
units:
runs-on: ubuntu-latest
needs: bulk-filter
strategy:
fail-fast: false
matrix:
Expand All @@ -42,11 +67,13 @@ jobs:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-unified-${{ steps.date.outputs.week_of_year }}
- run: .kokoro/build.sh
if: ${{ needs.bulk-filter.outputs.src == 'true' || needs.bulk-filter.outputs.ci == 'true' }}
env:
JOB_TYPE: test
JOB_NAME: units-${{matrix.java}}
units-8-runtime:
runs-on: ubuntu-latest
needs: bulk-filter
name: "units (8)"
steps:
- name: Get current week within the year
Expand All @@ -73,6 +100,7 @@ jobs:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-unified-${{ steps.date.outputs.week_of_year }}
- run: .kokoro/build.sh
if: ${{ needs.bulk-filter.outputs.src == 'true' || needs.bulk-filter.outputs.ci == 'true' }}
shell: bash
env:
JOB_TYPE: test
Expand Down
Loading