Only force the full test matrix on API changes when the contract changes#68060
Merged
shahar1 merged 1 commit intoJun 5, 2026
Merged
Conversation
Any change under the API directory (airflow-core/src/airflow/api_fastapi/, the legacy api/, or their test dirs) forced full-tests-needed=true — the whole ~135-job matrix including every provider's tests. The API tree is large and churns constantly, so this fired on ~1 in 8 PRs and was, by measurement, the single largest source of unnecessary full-matrix runs. But only the API *contract* changing ripples broadly: the generated OpenAPI spec (consumed by the UI codegen and the generated clients). Plain API source/test edits that leave the committed spec untouched do not — and a prek hook regenerates and verifies the spec, so an unchanged spec reliably means an unchanged contract. Narrow the full-tests trigger from API_FILES to API_CODEGEN_FILES (the generated spec / client generator). API source edits still run the `API` test type and the `fab` provider via run_api_tests; they just no longer drag in the full provider matrix. Spec changes still force the full matrix. In a 30-day sample (750 merged PRs) this spares ~63 full-matrix runs while preserving full coverage on the 12 PRs that changed the contract. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
shahar1
requested review from
amoghrajesh,
ashb,
bugraoz93,
choo121600,
ephraimbuddy,
gopidesupavan,
jason810496,
jedcunningham,
jscheffl,
potiuk and
vatsrahul1001
as code owners
boring-cyborg
Bot
added
area:dev-tools
backport-to-v3-2-test
Contributor
Backport successfully created: v3-2-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
potiuk
added a commit
that referenced
this pull request
Jun 6, 2026
…ontract changes (#68060) (#68084) * [v3-2-test] Only force the full test matrix on API changes when the contract changes (#68060) Any change under the API directory (airflow-core/src/airflow/api_fastapi/, the legacy api/, or their test dirs) forced full-tests-needed=true — the whole ~135-job matrix including every provider's tests. The API tree is large and churns constantly, so this fired on ~1 in 8 PRs and was, by measurement, the single largest source of unnecessary full-matrix runs. But only the API *contract* changing ripples broadly: the generated OpenAPI spec (consumed by the UI codegen and the generated clients). Plain API source/test edits that leave the committed spec untouched do not — and a prek hook regenerates and verifies the spec, so an unchanged spec reliably means an unchanged contract. Narrow the full-tests trigger from API_FILES to API_CODEGEN_FILES (the generated spec / client generator). API source edits still run the `API` test type and the `fab` provider via run_api_tests; they just no longer drag in the full provider matrix. Spec changes still force the full matrix. In a 30-day sample (750 merged PRs) this spares ~63 full-matrix runs while preserving full coverage on the 12 PRs that changed the contract. (cherry picked from commit d452d27) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * Fix selective-checks test expectations for v3-2-test (no shared/state) The backported tests expected mypy-shared-state in the skip-prek-hooks output, but the shared/state distribution does not exist on v3-2-test, so that hook is never generated. Drop mypy-shared-state from the two expected skip-prek-hooks strings so the API-matrix selective-check tests pass on this branch. --------- Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Member
|
Nice |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Any change under the API directory (
airflow-core/src/airflow/api_fastapi/, thelegacy
api/, or their test dirs) currently setsfull-tests-needed=true— thewhole ~135-job matrix, including every provider's tests. The API tree is large
and churns constantly, so this is the single largest source of unnecessary
full-matrix runs.
But only the API contract changing ripples broadly: the generated OpenAPI
spec, consumed by the UI codegen and the generated clients. Plain API
source/test edits that leave the committed spec untouched do not — and a prek
hook (
Generate the FastAPI API spec) regenerates and verifies the spec, so anunchanged committed spec reliably means an unchanged contract.
This narrows the full-tests trigger from
API_FILEStoAPI_CODEGEN_FILES(thegenerated spec / client generator):
the
APIcore test type and thefabprovider (viarun_api_tests); just nolonger drags in the entire provider matrix.
Sizing
Measured by replaying the
full_tests_neededgates over 750 merged PRs tomain(30-day window) from their changed-file lists:
full-tests-neededfires on 31.6% of PRs.changed the OpenAPI spec.
full coverage on the 12 contract-changing PRs.
Part of the CI-capacity work to reduce GitHub Actions runner load (fork PRs run
on public runners and are the constrained resource). Sibling over-trigger fix
for the file-count gate: #68059.
Tests
Updated the four API selective-check cases to assert the new lean selection
(
API+Always+Providers[common.compat,fab]), and locked in that anOpenAPI spec change still forces the full matrix. Full suite green (172 passed).
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines