Sync v3-3-stable with v3-3-test to release 3.3.0rc1#69126
Merged
Conversation
…onAPI` (#68980) (#68982) PR #68840 fixed an issue where exceptions thrown by the FastAPI lifecycle hook were swallowed. In doing so, it exposed a pre-existing problem where the lifecycle hook couldn't run when the JWT secret was not provided. As the `InProcessExecutionAPI` overrides auth, it doesn't need a JWT secret, and we certainly don't want to start crashing processes that previously ran fine as a result of a missing secret that we don't need. This commit stubs out the registered `JWTValidator` in the FastAPI app created for the `InProcessExecutionAPI`. This is done in an isolated services registry to ensure we don't leak this into any real app instantiations. (cherry picked from commit e886dfd) Co-authored-by: Nick Stenning <nick@whiteink.com>
…connections (#68960) (#68973) ## Why On macOS the task JVM's dual-stack loopback connection is reported by psutil in the IPv4-compatible form (`::127.0.0.1`), which the coordinator's ownership check did not normalize (only the IPv4-mapped `::ffff:127.0.0.1` form was), so every Java task was rejected with "process exited with 1 before connecting". ## What - Canonicalize IPv4-compatible IPv6 (`::a.b.c.d`) to plain IPv4 in `_socket_address`, excluding `::` and `::1`, alongside the existing IPv4-mapped handling. - Add `test_matches_dual_stack_ipv4_compatible_connection` regression test. (cherry picked from commit a2f8a7a) --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes, with help of Claude Code Opus 4.8 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) Signed-off-by: LIU ZHE YOU <zhu424.dev@gmail.com> Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
…er path (#69016) (#69029) The Task SDK supervisor imports the execution-time schema package on every worker, and that package imported cadwyn at module load. cadwyn pulls in FastAPI, Starlette and Jinja2, which are only needed on the foreign-language-SDK schema-migration path and never for a pure-Python worker. Because the supervisor is imported lazily after the Celery prefork, each pool worker paid this import in private (non copy-on-write) pages, raising worker memory on Python 3.14. Defer the cadwyn import behind a cached bundle accessor and into _versioned_class so importing the schema package no longer loads FastAPI. The foreign-SDK path is unchanged; a regression test asserts the worker import graph stays cadwyn-free. (cherry picked from commit 85aa9b6)
…only changes (#68802) (#68999) ENVIRONMENT_FILES forced the full test matrix on any .github/workflows change and on any scripts/ci/prek change. Those over-match: non-test workflows (security scans, doc publishing, notifications, backporting, stale/calendar bots) cannot affect test outcomes, and prek hooks drive static checks rather than the test matrix. Exclude non-test workflows from ENVIRONMENT_FILES (test/CI-config workflows like ci-amd, run-unit-tests, k8s-tests still force full), and move scripts/ci/prek to a new PREK_FILES group that triggers only the CI image build (so mypy-scripts and image-based static checks still run for a prek-only change) without forcing the full matrix. Static checks still run on every PR regardless; this only stops the unnecessary full test matrix for changes that cannot affect tests. (cherry picked from commit a4c269a) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…es (#68811) Bumps the github-actions-updates group with 3 updates in the / directory: [actions/checkout](https://github.com/actions/checkout), [actions/setup-java](https://github.com/actions/setup-java) and [pnpm/action-setup](https://github.com/pnpm/action-setup). Updates `actions/checkout` from 6.0.2 to 7.0.0 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v6.0.2...9c091bb) Updates `actions/setup-java` from 5.2.0 to 5.3.0 - [Release notes](https://github.com/actions/setup-java/releases) - [Commits](actions/setup-java@be666c2...ad2b381) Updates `pnpm/action-setup` from 6.0.8 to 6.0.9 - [Release notes](https://github.com/pnpm/action-setup/releases) - [Commits](pnpm/action-setup@0e279bb...0ebf471) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates - dependency-name: actions/setup-java dependency-version: 5.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates - dependency-name: pnpm/action-setup dependency-version: 6.0.9 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…runners (#68963) (#69054) The mongo provider tests boot a real mongo:8.0 container via testcontainers in the setup phase of the first test that uses the session fixture. That bring-up -- image pull, container start, readiness ping -- has to fit inside the per-test pytest setup-timeout (60s). On the GitHub-hosted ARM canary, where the image cache is cold and registry/disk throughput is lower, the pull alone can eat most of that budget, so setup times out and cascades into "ERROR at setup" across every TestMongoHook test. AMD does not hit this because it runs on every PR with warm caches; ARM only runs on the cron canary. Two complementary changes: - Pre-pull the testcontainers images (mongo:8.0 + ryuk) on the shared host daemon before the timed run, but only in CI and only when the owning provider's tests are actually in the selection -- so the slow cold-cache pull happens outside the per-test setup timeout. Locally it stays a no-op. - Give pytest setup/teardown timeouts a higher floor (180s) for container- backed fixtures, while keeping execution-timeout at the per-test value so genuinely hung tests are still caught quickly. (cherry picked from commit 9b0fe28) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…68968) (#69053) The test asserted the best of 3 "airflow --help" startup samples was under a tight 5s bound. On a loaded or cold runner (e.g. the Pendulum2 special-test job, which runs in a freshly rebuilt env) the whole-provider-load startup consistently sits just over it -- the best of 3 was 5.49s -- even though nothing regressed. Take the min of more samples and loosen the threshold so the test still guards against gross startup regressions without flaking on small fluctuations. (cherry picked from commit b24194f) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…day) (#68821) (#69039) The scheduled canary runs the full matrix on all versions and is by far the most expensive run type. AMD ran 4x/day and ARM 8x/day; halving AMD alone saves ~1,900 compute-h / 2 weeks (~6% of AMD CI), and cutting ARM 8->2/day removes 75% of the ARM canary schedule. The two crons are interleaved (AMD 01:58 & 13:58, ARM 07:28 & 19:28) so a full-matrix canary still runs roughly every ~6h, alternating architecture - so the 'trust earned on main' backstop keeps a ~6h effective cadence for arch-agnostic regressions (12h for arch-specific ones) while the scheduled compute drops substantially. It also relieves runner-pool queuing, since each canary is the heaviest job-flood. check_ci_workflows_in_sync.py expected blocks updated to match. (cherry picked from commit 5226a1b) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
… calculation (#68647) (#68949) DeadlineReference.AVERAGE_RUNTIME computes a deadline from the average duration of past DAG runs, but the query filtered only on dag_id + start/end-date present — with no DagRun.state filter. Failed runs (which may have died fast or hung before failing) were folded into the average, skewing the computed deadline: a fast-failing history makes it too short (spurious misses), a slow-then-failed history makes it too long (real slowness never trips it). Filter the duration query to successful runs only. Add tests asserting failed runs are excluded from the average and that the deadline is skipped when too few successful runs exist. --------- (cherry picked from commit 88e6110) Co-authored-by: Sean Ghaeli <58916776+seanghaeli@users.noreply.github.com> Co-authored-by: Sean Ghaeli <ghaeli@amazon.com> Co-authored-by: Ramit Kataria <ramitkat@amazon.com>
…ts (#69018) (#69055) The scheduled "breeze ci upgrade" job force-pushes a stable branch (ci-upgrade-<branch>) and then looks for an existing PR before creating one. The lookup passed the "owner:branch" head label to "gh pr list --head", but that flag filters by the bare branch name only and does not support the "owner:branch" syntax, so the existing PR was never found and the job fell through to "gh pr create" — which rejected the duplicate and failed the run. Use the bare branch name for the "gh pr list" and "gh pr ready" lookups (only "gh pr create --head" needs the cross-fork "owner:branch" label), and treat an "already exists" creation error as success since the branch has already been force-pushed. (cherry picked from commit 419ac06) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
…tion, not inline credentials (#68373) (#69105) * Document that Dag bundle kwargs should reference a Connection, not inline credentials Generated-by: Claude Opus 4.8 (1M context) following the guidelines at https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions * Fix docs spelling: reword 'inlined' (sphinx spellcheck) (cherry picked from commit 8c74f8e) Generated-by: Claude Opus 4.8 (1M context) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…) (#69066) Airflow 2.11 has reached end of life, so dependabot no longer needs to raise dependency-update PRs against v2-11-test. Drop the three update blocks targeting that branch (github-actions, pip, npm) and their "2.11 branch as well" section. The v3-3-test and main update blocks are unchanged. (cherry picked from commit a341562) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…nstead of hanging (#68492) (#69104) * Make airflow dags test wait for Human-in-the-loop input instead of hanging Previously a HITL task that parked in awaiting_input made `airflow dags test` loop "No tasks to run. unrunnable tasks: ..." once per second forever, with no way to make progress (the in-process runner also swallows SIGTERM, so even `timeout` could not stop it). dag.test() now treats parked HITL tasks as waiting rather than unrunnable, and never resolves them itself: the run stays alive, logging which tasks await input, until a response recorded from outside flips them back to SCHEDULED -- at which point the existing loop resumes them. This matches how a parked task behaves on a real deployment, and the existing response channels work unchanged: the Required Actions UI or the HITL REST API (PATCH .../hitlDetails) of an api-server sharing the metadata database (e.g. airflow standalone). Humans and AI agents can drive HITL pipelines locally by running dags test and submitting the response through that API. Interactive console prompting in the dags test CLI is left for a follow-up. * Document the HITL REST API calls for responding during dags test (cherry picked from commit 63cee22) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
…nerated/ (#69057) (#69060) The check-no-new-airflow-exceptions hook keeps its allowlist baseline in scripts/ci/prek/known_airflow_exceptions.txt. That file is touched by any PR that legitimately adds, removes, or moves a raise AirflowException -- typically provider or core changes that have nothing to do with dev tooling. Because it lived under scripts/, boring-cyborg matched it against the scripts/**/* glob and auto-applied area:dev-tools and, worse, backport-to-v3-3-test, triggering wrong automated backports of unrelated provider PRs. Move the generated allowlist to generated/known_airflow_exceptions.txt, which no boring-cyborg rule matches, so editing it no longer looks like a dev-tools change. The hook script reads/writes it from the repo-root generated/ directory; .txt files carry no license header, so nothing else changes. (cherry picked from commit 78263d0) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
…ith 2 updates (#69044) (#69059) Bumps the uv-dependency-updates group with 2 updates in the /dev/breeze directory: [pytest](https://github.com/pytest-dev/pytest) and [boto3](https://github.com/boto/boto3). Updates `pytest` from 9.1.0 to 9.1.1 - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst) - [Commits](pytest-dev/pytest@9.1.0...9.1.1) Updates `boto3` from 1.43.32 to 1.43.34 - [Release notes](https://github.com/boto/boto3/releases) - [Commits](boto/boto3@1.43.32...1.43.34) (cherry picked from commit 9517b49) --- updated-dependencies: - dependency-name: pytest dependency-version: 9.1.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: uv-dependency-updates - dependency-name: boto3 dependency-version: 1.43.34 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: uv-dependency-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
boring-cyborg
Bot
added
area:UI
Contributor
|
We should protect the stable branch since we are now on RCs |
vatsrahul1001
force-pushed
the
changes-3.3.0rc1
branch
from
9c971f3 to
5a0a3d4
Compare
Contributor
Author
|
ephraimbuddy
approved these changes
Jun 29, 2026
vatsrahul1001
force-pushed
the
changes-3.3.0rc1
branch
from
5a0a3d4 to
5e23a69
Compare
amoghrajesh
approved these changes
Jun 30, 2026
vatsrahul1001
force-pushed
the
changes-3.3.0rc1
branch
2 times, most recently
from
ccc6dc0 to
20f141e
Compare
Lee-W
reviewed
Jun 30, 2026
vatsrahul1001
force-pushed
the
changes-3.3.0rc1
branch
from
20f141e to
fc6cd25
Compare
Lee-W
reviewed
Jun 30, 2026
vatsrahul1001
force-pushed
the
changes-3.3.0rc1
branch
from
fc6cd25 to
51638b6
Compare
Lee-W
approved these changes
Jun 30, 2026
Lee-W
left a comment
Lee-W
left a comment
Member
There was a problem hiding this comment.
nitpicks. Major thing addressed. Thanks!
vatsrahul1001
force-pushed
the
changes-3.3.0rc1
branch
from
51638b6 to
fc7faf2
Compare
Lee-W
approved these changes
Jun 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Time for 3.3.0rc1
Release Notes
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.