Sync v3-2-stable with v3-2-test to release 3.2.2rc1#67233
Merged
Conversation
… updates (#65809) * Bump the 3-2-auth-ui-package-updates group across 1 directory with 22 updates Bumps the 3-2-auth-ui-package-updates group with 22 updates in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: | Package | From | To | | --- | --- | --- | | [@hey-api/openapi-ts](https://github.com/hey-api/openapi-ts) | `0.94.0` | `0.96.1` | | [@tanstack/react-query](https://github.com/TanStack/query/tree/HEAD/packages/react-query) | `5.90.21` | `5.99.2` | | [axios](https://github.com/axios/axios) | `1.15.0` | `1.15.1` | | [react](https://github.com/facebook/react/tree/HEAD/packages/react) | `19.2.4` | `19.2.5` | | [react-cookie](https://github.com/ItsBenCodes/cookies) | `8.0.1` | `8.1.0` | | [react-dom](https://github.com/facebook/react/tree/HEAD/packages/react-dom) | `19.2.4` | `19.2.5` | | [react-hook-form](https://github.com/react-hook-form/react-hook-form) | `7.71.2` | `7.72.1` | | [react-router-dom](https://github.com/remix-run/react-router/tree/HEAD/packages/react-router-dom) | `7.13.1` | `7.14.1` | | [@7nohe/openapi-react-query-codegen](https://github.com/7nohe/openapi-react-query-codegen) | `2.0.0` | `2.1.0` | | [@eslint/compat](https://github.com/eslint/rewrite/tree/HEAD/packages/compat) | `2.0.3` | `2.0.5` | | [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) | `8.57.0` | `8.58.2` | | [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) | `8.57.0` | `8.58.2` | | [@typescript-eslint/utils](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/utils) | `8.57.0` | `8.58.2` | | [@vitest/coverage-v8](https://github.com/vitest-dev/vitest/tree/HEAD/packages/coverage-v8) | `4.1.0` | `4.1.4` | | [eslint](https://github.com/eslint/eslint) | `10.0.3` | `10.2.1` | | [eslint-plugin-perfectionist](https://github.com/azat-io/eslint-plugin-perfectionist) | `5.6.0` | `5.9.0` | | [eslint-plugin-react-hooks](https://github.com/facebook/react/tree/HEAD/packages/eslint-plugin-react-hooks) | `7.0.1` | `7.1.1` | | [happy-dom](https://github.com/capricorn86/happy-dom) | `20.8.9` | `20.9.0` | | [prettier](https://github.com/prettier/prettier) | `3.8.1` | `3.8.3` | | [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) | `8.57.0` | `8.58.2` | | [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `8.0.5` | `8.0.9` | | [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) | `4.1.0` | `4.1.4` | Updates `@hey-api/openapi-ts` from 0.94.0 to 0.96.1 - [Release notes](https://github.com/hey-api/openapi-ts/releases) - [Changelog](https://github.com/hey-api/openapi-ts/blob/main/CHANGELOG.md) - [Commits](https://github.com/hey-api/openapi-ts/commits) Updates `@tanstack/react-query` from 5.90.21 to 5.99.2 - [Release notes](https://github.com/TanStack/query/releases) - [Changelog](https://github.com/TanStack/query/blob/main/packages/react-query/CHANGELOG.md) - [Commits](https://github.com/TanStack/query/commits/@tanstack/react-query@5.99.2/packages/react-query) Updates `axios` from 1.15.0 to 1.15.1 - [Release notes](https://github.com/axios/axios/releases) - [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md) - [Commits](axios/axios@v1.15.0...v1.15.1) Updates `react` from 19.2.4 to 19.2.5 - [Release notes](https://github.com/facebook/react/releases) - [Changelog](https://github.com/facebook/react/blob/main/CHANGELOG.md) - [Commits](https://github.com/facebook/react/commits/v19.2.5/packages/react) Updates `react-cookie` from 8.0.1 to 8.1.0 - [Release notes](https://github.com/ItsBenCodes/cookies/releases) - [Commits](ItsBenCodes/cookies@v8.0.1...v8.1.0) Updates `react-dom` from 19.2.4 to 19.2.5 - [Release notes](https://github.com/facebook/react/releases) - [Changelog](https://github.com/facebook/react/blob/main/CHANGELOG.md) - [Commits](https://github.com/facebook/react/commits/v19.2.5/packages/react-dom) Updates `react-hook-form` from 7.71.2 to 7.72.1 - [Release notes](https://github.com/react-hook-form/react-hook-form/releases) - [Changelog](https://github.com/react-hook-form/react-hook-form/blob/master/CHANGELOG.md) - [Commits](react-hook-form/react-hook-form@v7.71.2...v7.72.1) Updates `react-router-dom` from 7.13.1 to 7.14.1 - [Release notes](https://github.com/remix-run/react-router/releases) - [Changelog](https://github.com/remix-run/react-router/blob/main/packages/react-router-dom/CHANGELOG.md) - [Commits](https://github.com/remix-run/react-router/commits/react-router-dom@7.14.1/packages/react-router-dom) Updates `@7nohe/openapi-react-query-codegen` from 2.0.0 to 2.1.0 - [Release notes](https://github.com/7nohe/openapi-react-query-codegen/releases) - [Commits](7nohe/openapi-react-query-codegen@v2.0.0...v2.1.0) Updates `@eslint/compat` from 2.0.3 to 2.0.5 - [Release notes](https://github.com/eslint/rewrite/releases) - [Changelog](https://github.com/eslint/rewrite/blob/main/packages/compat/CHANGELOG.md) - [Commits](https://github.com/eslint/rewrite/commits/compat-v2.0.5/packages/compat) Updates `@typescript-eslint/eslint-plugin` from 8.57.0 to 8.58.2 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.58.2/packages/eslint-plugin) Updates `@typescript-eslint/parser` from 8.57.0 to 8.58.2 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.58.2/packages/parser) Updates `@typescript-eslint/utils` from 8.57.0 to 8.58.2 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/utils/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.58.2/packages/utils) Updates `@vitest/coverage-v8` from 4.1.0 to 4.1.4 - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.4/packages/coverage-v8) Updates `eslint` from 10.0.3 to 10.2.1 - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](eslint/eslint@v10.0.3...v10.2.1) Updates `eslint-plugin-perfectionist` from 5.6.0 to 5.9.0 - [Release notes](https://github.com/azat-io/eslint-plugin-perfectionist/releases) - [Changelog](https://github.com/azat-io/eslint-plugin-perfectionist/blob/main/changelog.md) - [Commits](azat-io/eslint-plugin-perfectionist@v5.6.0...v5.9.0) Updates `eslint-plugin-react-hooks` from 7.0.1 to 7.1.1 - [Release notes](https://github.com/facebook/react/releases) - [Changelog](https://github.com/facebook/react/blob/main/packages/eslint-plugin-react-hooks/CHANGELOG.md) - [Commits](https://github.com/facebook/react/commits/eslint-plugin-react-hooks@7.1.1/packages/eslint-plugin-react-hooks) Updates `happy-dom` from 20.8.9 to 20.9.0 - [Release notes](https://github.com/capricorn86/happy-dom/releases) - [Commits](capricorn86/happy-dom@v20.8.9...v20.9.0) Updates `prettier` from 3.8.1 to 3.8.3 - [Release notes](https://github.com/prettier/prettier/releases) - [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md) - [Commits](prettier/prettier@3.8.1...3.8.3) Updates `typescript-eslint` from 8.57.0 to 8.58.2 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.58.2/packages/typescript-eslint) Updates `vite` from 8.0.5 to 8.0.9 - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/v8.0.9/packages/vite) Updates `vitest` from 4.1.0 to 4.1.4 - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.4/packages/vitest) --- updated-dependencies: - dependency-name: "@hey-api/openapi-ts" dependency-version: 0.96.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: "@tanstack/react-query" dependency-version: 5.99.2 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: axios dependency-version: 1.15.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: 3-2-auth-ui-package-updates - dependency-name: react dependency-version: 19.2.5 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: 3-2-auth-ui-package-updates - dependency-name: react-cookie dependency-version: 8.1.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: react-dom dependency-version: 19.2.5 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: 3-2-auth-ui-package-updates - dependency-name: react-hook-form dependency-version: 7.72.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: react-router-dom dependency-version: 7.14.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: "@7nohe/openapi-react-query-codegen" dependency-version: 2.1.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: "@eslint/compat" dependency-version: 2.0.5 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-2-auth-ui-package-updates - dependency-name: "@typescript-eslint/eslint-plugin" dependency-version: 8.58.2 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: "@typescript-eslint/parser" dependency-version: 8.58.2 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: "@typescript-eslint/utils" dependency-version: 8.58.2 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: "@vitest/coverage-v8" dependency-version: 4.1.4 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-2-auth-ui-package-updates - dependency-name: eslint dependency-version: 10.2.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: eslint-plugin-perfectionist dependency-version: 5.9.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: eslint-plugin-react-hooks dependency-version: 7.1.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: happy-dom dependency-version: 20.9.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: prettier dependency-version: 3.8.3 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-2-auth-ui-package-updates - dependency-name: typescript-eslint dependency-version: 8.58.2 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: 3-2-auth-ui-package-updates - dependency-name: vite dependency-version: 8.0.9 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-2-auth-ui-package-updates - dependency-name: vitest dependency-version: 4.1.4 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-2-auth-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> * Update typescript generated files --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: vincbeck <vincbeck@amazon.com>
…#65874) On v3-2-test, BulkTaskInstanceService had three pre-existing bugs: 1. `_perform_update` did not pass `map_index` to `_patch_ti_validate_request`, so the validation step looked up the wrong TI (default map_index=-1) and persistence silently no-op'd for mapped task instances. 2. `specific_entity_map` and `all_map_entity_map` used raw `(entity.dag_id, entity.dag_run_id, entity.task_id, ...)` tuples as keys. When `dag_id`/`dag_run_id` came from the path and not the request body, those attributes were `None` on the entity, while keys produced by `_categorize_entities` resolved them via `_extract_task_identifiers`. The resulting key mismatch meant matched task keys never resolved back to an entity, so `_perform_update` was never called and `response.update.success` came back empty. 3. `action_name=action.action` rendered as `'BulkAction.UPDATE'` /`'BulkAction.DELETE'` in authorization error messages because Python's str-enum `__format__` returns the enum repr, not the value. Use `.value` to get "update"/"delete". All three fixes already exist on main; this backport is the minimal diff to bring v3-2-test in line with main for this file. Also includes the following pre-existing v3-2-test fixes that the prek hooks force to land alongside any source change: - Regenerated _private_ui.yaml, schemas.gen.ts and types.gen.ts: generate-openapi-spec caught a description drift on `ExtraMenuItem`. - Added `fpr` to docs/spelling_wordlist.txt: codespell flagged the GPG colons-format `^fpr:` regex tag in dev/README_RELEASE_*.md. Same fix already on main from #65501. - Regenerated dev/breeze/doc/images/output_pr_auto-triage.{svg,txt}: the boring-cyborg config update in #65872 added the `backport-to-airflow-ctl-v0-1-test` label, which appears in the `breeze pr auto-triage --help` output and changes its hash.
…#64748) (#65228) * [v3-2-test] UI: Rework Monaco editor theme to match Chakra UI palette (#64748) * UI: Rework Monaco editor theme to match Chakra UI palette Register custom airflow-light and airflow-dark Monaco themes derived from Chakra UI color tokens so that the DAG Code viewer, diff viewer, and JSON editors visually integrate with the rest of the app instead of using Monaco's default vs/vs-dark themes. The new useMonacoTheme hook rasterizes a single pixel through a 2D canvas and reads it back via getImageData to convert Chakra's OKLCH color values into the #rrggbb strings that Monaco's defineTheme accepts — ctx.fillStyle readback cannot be used because modern Chrome preserves the original OKLCH string. Themes are registered once via a module-level flag and then passed to every Monaco editor via the beforeMount callback. closes: #64253 * UI: Remove unnecessary useCallback in useMonacoTheme Address review feedback: React Compiler handles memoization, so the useCallback wrapper around defineAirflowMonacoThemes is redundant. Pass the function reference directly instead. Also fix prettier formatting in tests. * UI: Use culori for Monaco theme color conversion Address review feedback: the canvas-based cssVarToHex was brittle (depended on Canvas2D rendering and browser-specific fillStyle behavior, and required canvas mocking in happy-dom tests). Replace it with culori's parse/formatHex, which handles OKLCH and other modern color spaces directly with no DOM rasterization. Also add afterEach(vi.restoreAllMocks()) to the tests so the getComputedStyle spy does not leak between runs. * UI: Fix codespell typo in useMonacoTheme comment unparseable -> unparsable. (cherry picked from commit bfe46f6) Co-authored-by: Shivam Rastogi <6463385+shivaam@users.noreply.github.com> * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Shivam Rastogi <6463385+shivaam@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com>
…ut field behaviour (#65263) (#65267) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Fix connection schema field not saved for providers without field behaviour (#65263) When a provider (e.g. Oracle) does not define get_ui_field_behaviour(), the API returns standard_fields: null. The convertStandardFields function returned the raw default fields dict which uses `url_schema` as the key. The form then registered a Controller named "url_schema" instead of "schema", so user input never reached the schema field in the request body. On create, schema defaulted to None. On edit, the extra url_schema key caused a 422 from the StrictBaseModel extra="forbid" validation. Remap url_schema to schema in the null branch, matching the non-null path. (cherry picked from commit bb8971d) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> Closes: #65257 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
) (#65322) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] UI: Fix mapped task XCom navigation from Grid #64875 (#65192) * UI: Fix mapped task XCom navigation from Grid * Add regression test for mapped task navigation (cherry picked from commit c902ab6) Co-authored-by: Vishakha Agrawal <119448495+vishakha1411@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Vishakha Agrawal <119448495+vishakha1411@users.noreply.github.com>
* [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Fix task CLI map_index bounds validation (#64133) (cherry picked from commit bbc448f) Co-authored-by: Nandan <samarthachar18@gmail.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Nandan <samarthachar18@gmail.com>
… original start time (#63247) (#65491) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Fix ti.start_date showing deferral-resume time instead of original start time (#63247) * Fix ti.start_date showing deferral-resume time instead of original start time * Add API migration * Add unit test * add unit test to verify the resume start date behavior (cherry picked from commit a90202d) Co-authored-by: Henry Chen <henryhenry0512@gmail.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Henry Chen <henryhenry0512@gmail.com>
…overy, etc. (#65750) (#65854) Expand GitDagBundle test coverage with 12 new tests addressing gaps identified during investigation of repeated recloning after tag force-push. Force-push resilience: - Branch force-pushed to unrelated commit follows the new ref - Tag force-pushed to unrelated commit follows the new ref - Tag moved forward then backward across multiple refreshes - Refresh after force-push never triggers Repo.clone_from - Repeated refreshes after force-push remain stable - Re-initialization reuses existing repos (no reclone) Error handling: - Upstream tag deletion does not break refresh (local copy persists) - Failed bare-repo fetch preserves previous working tree - Refresh on versioned bundle raises AirflowException - Corrupted working repo triggers cleanup and retry Edge cases: - Real submodule fixture with ref change and submodule sync - Ambiguous ref (branch+tag same name) documents branch preference (cherry picked from commit 19ae9ee) Co-authored-by: Dev-iL <6509619+Dev-iL@users.noreply.github.com>
…65308) (#65318) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Handle supervisor remote log upload failures gracefully (#65308) * Handle supervisor remote log upload failures gracefully Prevent remote log upload errors from crashing task supervisor shutdown. Catch upload exceptions in the supervisor, log the failure with task context, and keep task completion behavior intact. * Update task-sdk/src/airflow/sdk/execution_time/supervisor.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> * fixup! Update task-sdk/src/airflow/sdk/execution_time/supervisor.py --------- (cherry picked from commit 6f1c97b) Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
) (#65364) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Validate SMTP server certificate on STARTTLS upgrade (#65346) * Validate SMTP server certificate on STARTTLS upgrade smtplib.SMTP.starttls() does not validate the server certificate unless an SSL context is passed. airflow.utils.email.send_mime_email and the SMTP provider's SmtpHook (both sync get_conn and async aget_conn) were calling starttls() without a context, so the STARTTLS upgrade accepted any certificate and the subsequent login() call could send credentials over a connection terminated by a MITM. Pass the existing SSL-context machinery (the email.ssl_context config in core and the ssl_context connection extra in the provider) to starttls() at all three call sites. The default becomes ssl.create_default_context(), which validates against the system's trusted CAs. Users who intentionally use self-signed certificates can still opt out by setting the value to "none". Generated-by: Claude Opus 4.6 (1M context) following the guidelines at https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions * Add newsfragment and SMTP provider changelog for STARTTLS cert default Document the default behaviour change introduced by passing an SSL context to the STARTTLS upgrade: system-default CA validation now applies to both airflow.utils.email.send_email (via email.ssl_context) and the SMTP provider's SmtpHook (via the ssl_context connection extra). Users who intentionally run against self-signed SMTP servers can preserve the old behaviour by setting the value to "none". (cherry picked from commit 06981d4) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Generated-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com>
…65381) (#65385) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] remove redundant quote in integration-system-tests.yml (#65381) (cherry picked from commit 6f0af81) Co-authored-by: Henry Chen <henryhenry0512@gmail.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Henry Chen <henryhenry0512@gmail.com>
… SQLite (#64888) (#65411) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Enable SQLAlchemy connection pool settings for file-based SQLite (#64888) * Enable SQLAlchemy connection pool settings for file-based SQLite * Fix review comments * Fix import make_url comment * Address Wei's final nit (cherry picked from commit d6ba3c4) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
* [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Embed GPG keys in repo for Docker build (#65408) * Embed GPG keys in repo for Docker build * Fix build of prod image (cherry picked from commit 13bb6a0) Co-authored-by: Daniel Wolf <95075445+wolfdn@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Daniel Wolf <95075445+wolfdn@users.noreply.github.com>
…nd worker logs (#65458) (#65476) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] Include TI UUID in scheduler, DAG processor, triggerer, and worker logs (#65458) Support engineers could not reconstruct a task's full lifecycle from logs because only the Execution API emitted the TaskInstance UUID consistently. Adding ti_id to log lines across the other components makes 'grep ti_id=X' surface every log touching that task, from scheduling through completion. - Worker: bind ti_id to structlog context at startup(). Fresh process per TI means no cross-task leak risk. - Triggerer: extend existing bind_log_contextvars at trigger start. The asyncio.create_task context copy scopes the binding per coroutine. - Scheduler: add ti_id=%s to eight TI-touching log calls across _enqueue_task_instances_with_queued_state, process_executor_events, and _maybe_requeue_stuck_ti. Explicit positional args avoid the contextvar leak a bind+unbind pattern would introduce on exception paths. - DAG processor: add ti_id to callback-processing log lines in _execute_callbacks and _execute_task_callbacks. * Move ti_id into TaskInstance.__repr__; revert redundant log-line additions Addresses review feedback from @jedcunningham on #65458: instead of sprinkling ti_id=%s onto individual scheduler log lines, put the UUID in TaskInstance.__repr__ once and let every %s-formatted TI log line inherit it for free. Strictly better: covers log lines this PR didn't touch and lines added by future PRs without further plumbing. Net diff vs main goes from +61/-16 to +51/-14. Changes: - TaskInstance.__repr__ now appends `ti_id={self.id}` before the closing bracket (matches the existing TaskInstanceNote repr precedent). - Reverted 10 log-line ti_id additions in scheduler_job_runner.py where the existing `%s` format arg was a TaskInstance; repr now supplies ti_id. - Kept the explicit `ti_id=%s` in the "TaskInstance Finished" msg: it formats individual fields (dag_id, task_id, etc.), not %s on the TI, so the repr shortcut does not apply. - Kept DAG processor structlog-kwargs ti_id additions: those go through structlog's kwargs path, not __repr__. - Updated one test assertion in test_scheduler_job.py that hardcoded the exact TaskInstance repr string. * Update test_not_enough_pool_slots for new TaskInstance repr After adding ti_id to TaskInstance.__repr__, test_not_enough_pool_slots needs to include ti_id in the expected log substring. Same fix pattern as test_process_executor_events_with_callback at line 695. * Fix test_not_enough_pool_slots ordering assumption on MySQL dr.task_instances[0] can return can_run first on MySQL (alphabetical default ordering) instead of cannot_run, so the expected ti_id used in the "Not executing" assertion grabbed the wrong task's UUID and the substring check failed on MySQL CI even though it passed on SQLite. Look up the TI by task_id instead to make the assertion order-independent. (cherry picked from commit 1a0efe7) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
…execution on scheduler crash (#65594) (#65711) When a scheduler crashes between dispatching a task to Celery and processing the QUEUED event that persists `external_executor_id`, the replacement scheduler cannot adopt the in-flight task. Without the Celery task ID in the database, `try_adopt_task_instances` has no `AsyncResult` to look up, so the task is reset and re-queued — causing duplicate execution of an already-running task. Fix this by generating `external_executor_id` via a DB-side UUID function (`gen_random_uuid` on PostgreSQL, `UUID()` on MySQL, a Python `uuid4` registered on SQLite) in the same bulk UPDATE that sets state=QUEUED. The ID is committed atomically with the state transition — no second write, no race window. RETURNING is used on PostgreSQL and SQLite to read back the generated UUIDs without a second round-trip; MySQL falls back to a SELECT. The CeleryExecutor passes the pre-assigned ID to `apply_async()` as the Celery `task_id`, making it deterministic from DB state. Other executors ignore it and overwrite with their own ID (e.g. ECS task ARN) during event processing. This also fixes the separate race in #55004 where `external_executor_id` is lost when the task instance row is locked during event processing. `process_executor_events` uses `skip_locked=True`, and `get_event_buffer()` flushes the executor's in-memory buffer into a local variable. If a TI is locked and skipped, its QUEUED event is consumed from the buffer but never processed — the event and its task ID are silently dropped. With the ID now written to the database before the task is even sent to Celery, adoption no longer depends on the event being processed. Closes: #55004 Closes: #58570 Closes: #64971 (cherry picked from commit 3b188b9)
…es (#65807) Bumps the github-actions-updates group with 5 updates in the / directory: | Package | From | To | | --- | --- | --- | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `7.0.0` | `7.0.1` | | [pnpm/action-setup](https://github.com/pnpm/action-setup) | `5.0.0` | `6.0.3` | | [actions/setup-node](https://github.com/actions/setup-node) | `6.3.0` | `6.4.0` | | [github/codeql-action](https://github.com/github/codeql-action) | `4.35.1` | `4.35.2` | | [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) | `8.0.0` | `8.1.0` | Updates `actions/upload-artifact` from 7.0.0 to 7.0.1 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@bbbca2d...043fb46) Updates `pnpm/action-setup` from 5.0.0 to 6.0.3 - [Release notes](https://github.com/pnpm/action-setup/releases) - [Commits](pnpm/action-setup@fc06bc1...903f9c1) Updates `actions/setup-node` from 6.3.0 to 6.4.0 - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@53b8394...48b55a0) Updates `github/codeql-action` from 4.35.1 to 4.35.2 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@c10b806...95e58e9) Updates `astral-sh/setup-uv` from 8.0.0 to 8.1.0 - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@cec2083...0880764) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: 6.4.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates - dependency-name: actions/upload-artifact dependency-version: 7.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions-updates - dependency-name: astral-sh/setup-uv dependency-version: 8.1.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates - dependency-name: github/codeql-action dependency-version: 4.35.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions-updates - dependency-name: pnpm/action-setup dependency-version: 6.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…t worktree (#65771) (#65828) * Breeze: fail fast when building provider sdists from a git worktree flit's --use-vcs silently produces incomplete sdists when run from a `git worktree add ...` directory, because flit.vcs.identify_vcs() checks `(p / ".git").is_dir()` and in a worktree `.git` is a file (gitdir: pointer). flit then falls back to a minimal sdist that omits docs/, tests/, provider.yaml and other tracked files, and the resulting packages fail reproducibility checks against released sdists on dist.apache.org — with no warning. `breeze release-management prepare-provider-distributions` now exits 1 with a clear explanation and a workaround (use a plain checkout, or pass `--distribution-format wheel` — wheels are unaffected) when it detects it is running from a worktree and would build sdists. Wheels and providers using hatchling with explicit sdist includes are not affected, so only `sdist` and `both` formats trigger the check. * Breeze: detect Docker-mounted worktrees in provider-sdist check When Breeze runs prepare-provider-distributions from a git worktree, the worktree's .git file carries an absolute gitdir: pointer to the main repo's .git/worktrees/<name> directory. Only the worktree folder is bind-mounted into Breeze's Docker container, so that pointer target is unreachable from inside the build. flit's VCS detection then either fails or silently produces an incomplete sdist. Expand check_flit_worktree_compatibility to parse .git and branch on the failure mode: read error, unexpected format, missing gitdir target (Docker mount case), or healthy host-side worktree. Tests cover all paths. * Link upstream flit issue/PR + tracking issue at the workaround site Add references to pypa/flit#798 and pypa/flit#799 in the docstring of check_flit_worktree_compatibility, and the Airflow tracking issue #65772 at both the helper and its call site. CLAUDE.md requires the tracking-issue URL to appear at the workaround site in the code so the follow-up work is discoverable from any grep or code review. (cherry picked from commit ddf3c7a) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…age (#65648) (#65848) * Add pr-triage skill for maintainer-driven PR queue sweep A skill-based replacement for the triage mode of breeze pr auto-triage. Scope is triage only — no LLM code review, no approve/request-changes, no merging, no TUI. Shape follows the airflow-translations convention: .github/skills/pr-triage/ holds the canonical content; .claude/skills/pr-triage is a committed symlink so Claude Code picks it up without per-developer setup. .gitignore is updated to let the symlink through, and the agentic-markdown license hook excludes SKILL.md entry point so YAML frontmatter can stay at the top. Also refreshes dev/breeze/doc/images/output_pr_auto-triage.{svg,txt} — upstream's committed image was stale (boring-cyborg.yml added labels the committed image hadn't regenerated yet), and the pre-commit hook regenerates it on every touch of the breeze config. Generated-by: Claude Opus 4.7 (1M context) following the guidelines in contributing-docs/05_pull_requests.rst * pr-triage: add Copilot-review staleness + prefer author-pings Two rule refinements surfaced while running the skill on apache/airflow: 1. New classification stale_copilot_review. When a PR has an unresolved review thread authored by a Copilot-review bot (copilot-pull-request-reviewer[bot] and siblings) and the comment is >= 14 days old with no author reply, the skill classifies it as stale_copilot_review and suggests draft with a new "Unaddressed Copilot review" violation. The 14-day window is deliberately longer than the 24h/96h CI grace — review feedback takes longer to address and a two-day nudge would be noisy. 2. Strongly prefer pinging the PR author over the reviewer on the ping action. Both comment families (review-nudge for stale CHANGES_REQUESTED, reviewer-ping for unresolved threads) now document an author-primary default body and a reviewer-re-review variant. The reviewer-re-review body is only used after an explicit inspection of the post-review diff + in-thread replies confirms the feedback has been addressed in code or explained away. The previous logic (which switched on whether the author had @-mentioned the reviewer) put the re-review request on the reviewer's desk even when the actual work was still on the author's side. Updates classify.md, suggested-actions.md, actions.md, comment-templates.md, SKILL.md, and interaction-loop.md so the group ordering, data requirements, and body-template decision are consistent across the skill. * pr-triage: never suggest rebase on CONFLICTING PRs — route to draft Empirically every rebase attempted on a PR whose mergeable state is CONFLICTING came back with "Cannot update PR branch due to conflicts". GitHub's update-branch endpoint does a side-merge of the base branch into the PR head; when the merge doesn't apply cleanly, it refuses, and retrying burns round-trips without progress. The fix is to stop suggesting rebase in that state at all — those PRs go straight to draft with the merge-conflicts violation so the author is pointed at the local-rebase instructions in the comment body. - suggested-actions.md: the "mergeable == CONFLICTING" rule now maps to draft regardless of other signals. An explicit hard rule spells out why. - actions.md#rebase: a mergeable check runs before the update-branch call; if the state is CONFLICTING the action refuses with a route-to-draft hint. If a non-CONFLICTING call still 422s, the skill does not retry — it falls through to draft. - SKILL.md Step 3 table updated to reflect the new mapping. * pr-triage: tighten untriaged-draft stale threshold from 3 weeks to 2 weeks Drafts that have had no activity for 14 days (was 21) now fall into Sweep 1b (untriaged-draft close). The triaged-draft 7-day threshold for Sweep 1a is unchanged. Maintainer preference: two weeks is enough signal that a draft has stalled; waiting the extra week was leaving queue pressure without adding useful context on author intent. * pr-triage: stale-Copilot threshold 14 days -> 7 days, soften body Two related changes for the stale_copilot_review classification: - Tighten the trigger threshold. Copilot reviews that sit unaddressed for a week already signal a stalled PR — waiting two weeks before drafting adds queue pressure without useful signal. - Revise the draft-comment language to explicitly call out that some Copilot suggestions may be incorrect or irrelevant, and that it is still the author's responsibility to respond — apply the fix, reply in-thread explaining why it does not apply, or resolve the thread if the feedback is no longer relevant. Previously the wording could be read as 'please do everything Copilot said', which is the wrong bar. * pr-triage: make action_required REST check primary for pending_workflow_approval A PR that has real CI runs held in action_required for approval can still report statusCheckRollup.state == SUCCESS, because the rollup aggregates only completed check-runs and fast bot checks (Mergeable, WIP, DCO, boring-cyborg) succeed unconditionally before the real CI is even allowed to start. Trusting the rollup classifies such a PR as passing and leads to premature mark-ready. Changes: - classify.md: rewrite C1 (pending_workflow_approval) so the repo-level REST call GET /repos/.../actions/runs?status=action_required is the primary signal, not a fallback. Index the response by head_sha once per page. Call out that the rollup-based check alone is not sufficient and explain why. - classify.md: rewrite 'Verifying real CI ran' from an optional note into a mandatory pre-passing guard. Expand the real-CI pattern list to include Tests (exact), CodeQL, and 'Check newsfragment PR number'. - fetch-and-batch.md: add an 'Mandatory: action_required run index per page' section that documents the REST call as part of the per-page fetch plan (one extra round-trip per page, well inside the budget). * pr-triage: spell out author responsibility to mark threads resolved The Unresolved review comments / Unaddressed Copilot review violation bodies now explicitly state that it is the author's responsibility to mark a thread as resolved once they believe it has been addressed — whether that was by pushing a fix or by replying with an explanation of why the suggestion doesn't apply. Reviewers do not auto-close their own threads, so a thread that is addressed but left unresolved reads as 'still waiting on the author' and blocks the PR from moving forward. Making this explicit in the draft comment is the difference between first-time contributors wondering why their PR still looks blocked after a fix vs. knowing to click 'Resolve conversation' at the bottom of each thread. * pr-triage: mandatory pre-check against action_required on mark-ready Sole purpose of this commit is to add an executable guard in actions.md that the classifier's real-CI-ran check cannot substitute for. Empirically, the classifier can still green-light a PR that has pending workflow approvals if the GraphQL batch fetch was taken before GitHub indexed a freshly-required workflow run, or if only the bot subset of checks has completed and the classifier's 30-context sample missed a real-CI entry. Changes: - actions.md §mark-ready: add a MUST-DO pre-mutation check — GET /repos/<owner>/<repo>/actions/runs?head_sha=<SHA>&status= action_required returning any result is a hard refuse. On refuse, reclassify the PR as pending_workflow_approval and route accordingly instead of silently dropping the mutation. - SKILL.md Golden rules: add rule 1b elevating the check to the top-of-file contract so a skill implementor can't miss it. The rationale (fast bot checks finish first while Tests / CodeQL / newsfragment sit in action_required) is spelled out so nobody thinks this is belt-and-braces redundancy. * pr-triage: add AI-attribution footer to contributor comments Every contributor-facing triage comment now ends with a short footer that (a) notes the comment was drafted by an AI-assisted tool and may contain mistakes, (b) reassures the contributor that a human maintainer will take the next look once they address the feedback, and (c) links to a new section in the maintainer-triage contributing doc that explains why the first pass is automated — to free scarce maintainer time for the conversation with contributors rather than mechanical queue sweeping. The footer is defined once in comment-templates.md as <ai_attribution_footer> and referenced from every contributor-facing template (draft, comment-only, close, review-nudge, reviewer-ping, stale-draft-close, inactive-to-draft, stale-workflow-approval). The suspicious-changes template is intentionally excluded — it is terse and already directs the contributor to maintainers on Slack, and adding the footer would dilute the signal. SKILL.md carries a new Golden rule 8 capturing the requirement and the link to the rationale anchor. (cherry picked from commit 0ed3e73) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…erges (#65765) (#65851) * CI: Notify open PRs that conflict on uv.lock after main merges When `uv.lock` changes on `main`, open PRs that touch `uv.lock` frequently end up with rebase conflicts. This adds a workflow that, on each push to `main` that modifies `uv.lock`, scans open non-draft PRs (updated in the last 14 days), identifies those currently reporting conflicts, and posts (or updates, deduped via an HTML-comment marker) a single comment with rebase-and-`uv lock` instructions. The source PR that caused the change is resolved from the squash-merge commit headline's trailing `(#NNN)` and linked in the notice. Implemented as a standalone PEP 723 Python script (`httpx` + GitHub's GraphQL API) so logic stays testable; workflow YAML is a thin driver that installs `uv` from `uv.lock` and runs the script. Unit tests cover the regex, classifier, pagination/early-exit, UNKNOWN-mergeable retry loop, and notice posting. * Fix scripts-tests by stubbing httpx in unit-test module The scripts-project CI venv doesn't have httpx installed, so importing ``notify_uv_lock_conflicts`` fails with ``ModuleNotFoundError: No module named 'httpx'`` before any test runs. httpx is a PEP 723 runtime dep of the script; the tests mock the client and never need the real library. Stub ``httpx`` in ``sys.modules`` with ``setdefault`` before loading the module under test: keeps the scripts distribution's declared deps unchanged, and leaves real ``httpx`` alone when present (e.g. in the workspace-wide dev environment). (cherry picked from commit 4fae29c) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…ake (#65867) (#65868) The jaeger container in the otel integration compose file mapped four host ports (34317, 34318, 36686, 39422) inside the Linux default ephemeral port range (32768-60999). The kernel can transiently grab a port in that range as the source port of an outbound connection from any other process on the CI runner, which then races with the docker bind and surfaces as: failed to bind host port for 0.0.0.0:34318:172.18.0.3:4318/tcp: address already in use The retry mechanism (which restarts docker between attempts) does not help, because the holder is a non-docker process. Move the four jaeger host ports below 32768, matching the convention the other integration services in the same file already follow, and update the docstring in test_otel.py that pointed at the old jaeger UI port. (cherry picked from commit c29cc61) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…ng (#64182) (#65896) The PoolBar component links to the task instances page using SearchParamsKeys.STATE ('state') and SearchParamsKeys.POOL ('pool'), but the TaskInstances page reads filters from SearchParamsKeys.TASK_STATE ('task_state') and SearchParamsKeys.POOL_NAME_PATTERN ('pool_name_pattern'). This mismatch causes clicking a pool slot segment (running, queued, etc.) to navigate to the task instances page without any filters being applied, showing all task instances instead of those filtered by the clicked state and pool. Fix by using the correct search parameter keys (TASK_STATE and POOL_NAME_PATTERN) and slot.slotType instead of slot.color for the state filter value. Co-authored-by: Burra Karthikeya <karukarthikeya1111@gmail.com>
…caused by concurrent API schema requests (#63986) (#65892) * Fix TypeError crashes on /users/list and /roles/list in FAB UI caused by concurrent API schema requests * Fix TypeError in FAB UI by isolating ProvidersManager discovery and making MockOptional callable * Fix unrelated Elasticsearch test failure in FAB UI PR branch * Revert unrelated Elasticsearch test changes Co-authored-by: Subham <subhamsangwan26@gmail.com>
* avoid passing parsed input back to component * on change, update component and debounce utc parsing * typo, linting fixes * longer type delay, removed redundant isValid check Co-authored-by: Tomi <74303735+Tomi-1997@users.noreply.github.com>
…5604) (#65746) * Add cursor based pagination for get_dag_runs endpoint Add cursor-based (keyset) pagination as an alternative to offset-based pagination on the get_dag_runs endpoint. Offset pagination remains the default and is not deprecated. Mirrors the pattern introduced for get_task_instances in #64845. - Response schema adds next_cursor / previous_cursor and makes total_entries optional (populated only for offset pagination). - Fetch limit+1 rows to detect last page and return next_cursor=null when no more results exist. - Rewire the DagRuns listing page to use cursor pagination via the DataTable's cursorPagination prop, which eliminates the COUNT(*) query for large datasets. * Small Adjustments * Small Adjustments (cherry picked from commit c57abb9) Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
…lue equals schema default (#65310) (#67097) * Fix max_active_runs lost during DAG serialisation when value equals schema default The serialisation optimisation from #55849 strips DAG fields that match their schema.json default. For max_active_runs, max_active_tasks, and max_consecutive_failed_dag_runs this is wrong because their runtime defaults come from airflow.cfg, not the schema. When a user explicitly sets max_active_runs=16 and the config has max_active_runs_per_dag=1, the value gets stripped and the dag table ends up with 1. Skip the schema-default exclusion for these three config-driven fields so they always survive serialisation. * fix: introduce DAG_DEFAULTS * chore: add test for catchup * fix: revert comment change, that wording was better * fix: preserve DAG fields whose defaults match the schema default * fix: update schema default check script * fix: address review comments on serialisation tests * fix: add disable_bundle_versioning case to parametrised serialisation test * fix: improve docstring for config-driven fields serialisation test * fix: use tuple for pytest.mark.parametrize first argument (cherry picked from commit 76eb2a0) Co-authored-by: Selman <seruman@users.noreply.github.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
… preserve nested-key masking on truncation (#65906) (#67117) * Redact rendered template fields while still structured to preserve nested-key masking on truncation Generated-by: Claude Opus 4.7 (1M context) following the guidelines at https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions * Isolate masker patterns in nested-key truncation test The new test_rendered_templates_mask_nested_keys_with_truncation shares the singleton SecretsMasker with earlier tests in the file. One of those (test_get_connection_from_context) fetches a connection whose password fixture value happens to be the literal string "password", which the SDK runtime registers as a regex mask via mask_secret(). When the new test runs after it, that regex substitutes the literal token "password" inside str(redacted) -- including the dict KEY name -- so the assertion "'password': '***'" fails because the key itself is also masked. Reset patterns/replacer for the test via monkeypatch (auto-restored on teardown) so the assertion isolates value-masking (the behavior under test) from key-token replacement (a side effect of leaked patterns). (cherry picked from commit 4ceb0db) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
* UI: Search task instances by rendered map index Adds 'rendered_map_index_pattern' (substring match) and 'rendered_map_index_prefix_pattern' (index-friendly prefix match) query parameters to the public listMapped and listTaskInstances endpoints, mirroring the existing 'operator_name', 'pool_name', etc. pattern/prefix-pattern pairs. The UI surfaces this as a search input in the TaskInstances filter bar — prefix search by default, with an 'advanced search' toggle to switch to substring match. The TaskInstance.rendered_map_index hybrid_property now has a SQL-level .expression so queries see the same value as the API response: the explicit _rendered_map_index when set, otherwise str(map_index). Without this, filtering on rendered map index silently dropped rows whose _rendered_map_index column is NULL but whose displayed value is the numeric map index. Closes #51820. * Fix rebase linting * Fix mysql tests * Fix conflicts --------- (cherry picked from commit 73a6641) Co-authored-by: Brent Bovenzi <brent.bovenzi@gmail.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
…dpoint (#67112) (#67159) Align GET /eventLogs/{event_log_id} with the collection endpoint GET /eventLogs, which already scopes results to the user's permitted Dags via ReadableEventLogsFilterDep. The detail endpoint only enforced the generic DagAccessEntity.AUDIT_LOG check via requires_access_dag with no dag_id. Introduce requires_access_event_log, mirroring requires_access_backfill: resolve the dag_id from the event log row, then delegate to requires_access_dag scoped to that dag_id. (cherry picked from commit 4498582) Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
…#67171) * Default-deny auth at the API and UI router level Add `dependencies=[Depends(get_user)]` to `authenticated_router` (parent of every route under `/api/v2` except the explicit no-auth carve-outs `monitor_router`, `version_router`, and the public `auth_router`) and to `ui_router` (every route under `/ui`). Today every authenticated route already declares `GetUserDep` or a `requires_access_*` dependency that itself depends on `get_user`, so this is purely additive — FastAPI deduplicates the dependency via its per-request cache, so each request still resolves `get_user` once. The value is preventing a future route from being added under either router without an auth check: the router-level dependency catches the regression at registration time rather than at audit time. Add a structural test that asserts both routers carry the router-level `Depends(get_user)`, so a future refactor that drops the dependency without considering its purpose fails the test rather than silently widening the unauthenticated surface. * Move test imports to top of file Address review feedback from @Lee-W on PR #66505. (cherry picked from commit aa8542f) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
#66571) (#67172) * Surface remote-log upload failures via structured warnings `upload_to_remote()` silently returned in three failure paths (handler load failure, log-path resolution exception, handler.upload exception) with no signal to the operator that logs were not reaching the remote system. The supervisor's `_upload_logs` already logs the outermost exception, but the inner paths inside `upload_to_remote()` were silent — so a misconfigured remote handler or a transient remote-system outage would degrade silently while local-only logs continued. Add a dedicated `airflow.logging.remote` structlog logger and emit a `log.warning` at each of the three failure paths with the TI id and the underlying error string. No behaviour change otherwise — failures still fall through softly so a bad remote handler doesn't abort the task lifecycle. Reported by the L3 ASVS sweep at apache/tooling-agents#24 (FINDING-004). * Address review feedback on remote-log upload warnings - Use ti.id directly (available since 3.0) instead of getattr fallback. - Pass exc_info=exc to structlog so the traceback is preserved on path-resolution and upload failures. - Trim past-tense from the docstring comment. - Add tests covering the three failure paths and the silent success / no-path cases. (cherry picked from commit ef87426) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…67177) * Don't crash supervisor IPC loop on transient network errors handle_requests in the supervisor only caught ServerResponseError. Any non-HTTP exception (httpx.ConnectError, httpx.TimeoutException, socket timeouts, etc.) would propagate, terminate the generator, and permanently break the supervisor-to-task IPC channel. The task subprocess would then get EOFError on every subsequent send, and the worker would be stuck waiting for replies that never come. Add a catch-all except Exception after the ServerResponseError handler that logs the unhandled exception with type info, sends a best-effort ErrorResponse(API_SERVER_ERROR, ...) back to the task so the failure surfaces in task logs (wrapped in suppress(Exception) because if we can't reach the task subprocess via stdin we shouldn't double-fault), and lets the request loop continue to the next request. Test added: a fake httpx.ConnectError on the first call produces an ErrorResponse, the generator stays alive, and a second request is processed normally (the loop is not dead). Reported by the L3 ASVS sweep at apache/tooling-agents#24 (FINDING-005). * Address review comments: shorten comment and use exc_info - Shorten the catch-all comment per amoghrajesh's suggestion. - Use exc_info=e in log.exception instead of exception_type field per jason810496's suggestion (exception type is redundant since the exception itself is logged with full type info and traceback). (cherry picked from commit 1e5d799) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
… against malformed bodies (#66504) (#67182) * Fail closed in _collect_teams_to_check on body parse failure For POST/PUT in multi-team mode, the helper used `with suppress(JSONDecodeError)` around `await request.json()`. If the body was unparseable, the suppress swallowed the exception, `teams.add(raw)` never ran, and the calling `requires_access_*` dependency iterated over an empty set — silently skipping the authorization callback entirely. Today this is unreachable because every POST/PUT route in core_api uses a Pydantic body model, so FastAPI returns 422 before the auth dependency runs. But the pattern would silently bypass team-scoped authz if a future route used a raw `Request` instead. Replace the bare suppress with an explicit try/except that adds `None` to `teams` on parse failure, so the auth callback always runs at least once. * Reject malformed bodies in core_api authz helpers with 400 Builds on the previous fail-closed change in _collect_teams_to_check. Two follow-ups from review: * On JSONDecodeError, raise HTTP 400 directly instead of falling through to a team=None auth call — clearer failure mode and removes any ambiguity about whether authz ran. * Reject non-string `team_name` (in _collect_teams_to_check) and non-string `dag_id` (in requires_access_backfill) from the raw body with HTTP 400 before any authz decision or DB lookup. Without this, a list / dict / int / bool would flow into Team.get_name_if_exists, requires_access_dag, or the existence lookup with undefined behaviour or type-confused authz decisions. Both helpers still read the raw body before Pydantic body validation runs on the endpoint handler, so this is defense-in-depth: every current POST/PUT route uses a Pydantic body model and FastAPI returns 422 before the auth dependency runs on a malformed body. Tests: existing parse-failure test renamed and updated to assert 400; new parametrised tests cover integer / list / dict / bool inputs for both team_name and dag_id. (cherry picked from commit 448f846) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
…ny (#66575) (#67173) * Refuse secrets-backend fallback on Execution-API authz deny ExecutionAPISecretsBackend.get_connection / get_variable returned None on every ErrorResponse, conflating "not found" with "explicitly denied". The secrets-backend dispatcher then fell through to the next backend (typically EnvironmentVariablesBackend, which performs no authz checks) on a 401/403 from the Execution API -- letting tasks read secrets the Execution API had just denied them. The audit calls this a Type C gap: the authz control fires, but its rejection result is treated as a miss and routed around. Three-part fix: 1. New `ErrorType.PERMISSION_DENIED` distinct from `API_SERVER_ERROR`. 2. `ConnectionOperations.get` and `VariableOperations.get` map the API server's 401/403 to `ErrorResponse(PERMISSION_DENIED, ...)` instead of re-raising as a generic `ServerResponseError`. 404 still maps to `*_NOT_FOUND`; other statuses still raise so the existing API_SERVER_ERROR translation in `handle_requests` keeps working. 3. `ExecutionAPISecretsBackend` (sync + async, connection + variable variants) now raises `PermissionError` on `PERMISSION_DENIED`. The surrounding `except Exception:` blocks explicitly re-raise `PermissionError` so the secrets-backend dispatcher sees it. NOT_FOUND types continue to return `None` (allow fallthrough); other ErrorResponses also continue to return `None` (preserve existing recovery behaviour for transient errors). Tests added: - `client.connections.get` and `client.variables.get` return `ErrorResponse(PERMISSION_DENIED)` on 401 and 403 (parametrised). - `ExecutionAPISecretsBackend.get_connection` / `get_variable` / `aget_connection` / `aget_variable` raise `PermissionError` when the response is `PERMISSION_DENIED`, with the resource and key in the message. Reported by the L3 ASVS sweep at apache/tooling-agents#24 (FINDING-017). * Refuse secrets-backend fallback at the dispatcher, not only the backend The earlier change made ExecutionAPISecretsBackend raise on 401/403, but the dispatcher loops in airflow.sdk.execution_time.context and the airflow-core get_*_from_secrets paths catch Exception and silently fall through to the next backend — so the deny was still being swallowed. Introduce AirflowSecretsBackendAccessDenied (subclass of PermissionError) so the dispatchers can special-case the authoritative deny without mis-treating an incidental OSError-family PermissionError from inside an unrelated backend. Patch the three task-SDK dispatcher loops and the two airflow-core dispatcher loops to re-raise it before the generic except. Add TestDispatcherRefusesFallbackOnDeny with three end-to-end tests that insert a spy backend after ExecutionAPISecretsBackend and assert the spy is never called once the first backend raises the deny — pinning the dispatcher behaviour, not just the backend's. Also hoist the repeated imports in test_secrets.py to module top per review feedback. * Hoist AirflowSecretsBackendAccessDenied imports to module top (cherry picked from commit 2b8c805) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…rminal state (#66573) (#67183) * Fail closed when supervisor IPC fails on a non-success terminal state When a task FAILED / SKIPPED / etc. and the IPC send of the terminal-state message to the supervisor itself raised, the existing finally block logged the exception and let run() return normally. The task subprocess then exited with code 0, which the supervisor final_state property maps to SUCCESS for an exit_code-0 process without a _terminal_state (the supervisor never received the message). A genuine task FAILURE was silently being upgraded to SUCCESS on transient IPC failures, breaking downstream pipeline correctness without any signal. Exit non-zero from the finally block when the terminal state is anything other than SUCCESS, so the supervisor's final_state correctly classifies as FAILED (or UP_FOR_RETRY when retries are configured). The SUCCESS exemption preserves the existing softening for the legitimate scenario where the supervisor rejects the terminal-state send with a 409 because the server already terminalised the TI -- covered by test_run_swallows_supervisor_terminal_send_failure, which continues to pass. New regression test: when the task fails and the supervisor IPC send raises (BrokenPipeError simulating a dead Unix socket), run() now raises SystemExit(1). Reported by the L3 ASVS sweep at apache/tooling-agents#24 (FINDING-006). * Address review comments: defer fail-closed exit, narrow guard - Narrow the fail-closed guard from `state != SUCCESS` to FAILED / UP_FOR_RETRY only (kaxil). SKIPPED / UP_FOR_RESCHEDULE / DEFERRED would otherwise be mismapped to FAILED by supervisor's final_state, which is strictly worse than the default mapping. - Stop calling sys.exit(1) inside run()'s finally — SystemExit is a BaseException so main()'s `except Exception:` would not catch it and finalize() would be skipped, silently dropping on_failure_callback / on_retry_callback / listener hooks / email_on_failure / email_on_retry on the same IPC failure (kaxil). Signal via `_terminal_state_send_failed` on the ti and let main() sys.exit(1) after finalize() has run. - Remove the redundant inline `from ... import run` in the test (Lee-W, kaxil) — `run` is already imported at module level. - Rework the existing regression test to assert the new contract (run() returns FAILED + sets the flag) and add a listener-based test that locks in callbacks/listeners still firing on the IPC-broken path (kaxil). * Declare _terminal_state_send_failed on RuntimeTaskInstance for mypy The fail-closed path in _handle_current_task_failure set ti._terminal_state_send_failed = True dynamically without a class-level declaration, so mypy raised [attr-defined]. Add the field as a Pydantic PrivateAttr (bool, default False) matching the existing _cached_template_context pattern. No runtime behavior change — getattr(ti, '_terminal_state_send_failed', False) still returns False for the unset case, now via the PrivateAttr default instead of the getattr fallback. --------- (cherry picked from commit 7e91517) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: vatsrahul1001 <rah.sharma11@gmail.com>
* improve grid/ti_summaries and grid/runs (#64034) * improve grid/ti_summaries and grid/runs * remove serdag * Fix tests * Undo migration file change --------- Co-authored-by: Henry Chen <henryhenry0512@gmail.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
* UI: Bundle Monaco editor instead of loading it from CDN * UI: Configure Monaco worker loading for bundled editor * UI: Lazy load Monaco editor configuration * UI: Bundle Monaco editor without changing runtime version * UI: Fix Dag code tab line number selector * UI: Load bundled Monaco workers with Vite worker imports * UI: Handle Monaco editor load failures * UI: Add Monaco Editor license notice * UI: Add Monaco Editor license notice (cherry picked from commit 3a86142) Co-authored-by: hojeong park <parkhj062@gmail.com>
…ls (#66574) (#67204) * Recover stuck TIs when direct terminal-state API call fails (#66574) * Recover stuck TIs when direct terminal-state API call fails The supervisor's _handle_request for SucceedTask, RetryTask, DeferTask, and RescheduleTask set _terminal_state BEFORE calling the matching client.task_instances.{succeed,retry,defer,reschedule}() API. If that API call raised (transient network blip, server 5xx, etc.), _terminal_state was set on the supervisor but the server never saw the transition. The supervisor's update_task_state_if_needed then saw final_state in STATES_SENT_DIRECTLY and short-circuited the recovery finish() call -- leaving the TaskInstance stuck RUNNING on the server forever, blocking downstream dependencies and triggering false alerts. Two-part fix: 1. Make the direct API call FIRST. Only set _terminal_state and the new _terminal_state_synced_to_server flag after the call returns successfully. If the API raises, both stay unset and the exception propagates to handle_requests, where the existing catch-all sends an ErrorResponse to the task subprocess. 2. Have update_task_state_if_needed always call finish() when _terminal_state_synced_to_server is False, regardless of what final_state happens to return. The finish() API takes the state value, so a SUCCESS / DEFERRED / etc. transition that originally failed is re-attempted via finish() on subprocess exit. Pre-existing semantics for the no-direct-API states (FAILED, UP_FOR_RETRY without RetryTask, etc.) preserved -- those land in the same finish() branch. Tests added: - _terminal_state not set when succeed() raises. - update_task_state_if_needed calls finish() when synced flag is False, even with final_state == SUCCESS. - update_task_state_if_needed skips finish() when synced flag is True (preserves the existing happy-path optimisation). Reported by the L3 ASVS sweep at apache/tooling-agents#24 (FINDING-007). * Refactor terminal-state dispatch and parametrize tests across all 4 states Address review feedback on #66574: - Extract `_send_terminal_state_msg` helper so the per-msg-type dispatch for succeed / retry / defer / reschedule lives in one place. Both `_handle_request` and `_replay_pending_terminal_state_msg` now go through it instead of duplicating the four-branch isinstance chain. - Parametrize the two recovery tests over all four terminal-state message types (was only Succeed + Defer); add UP_FOR_RETRY and UP_FOR_RESCHEDULE coverage. * Narrow _pending_terminal_state_msg type to satisfy mypy The field was annotated as BaseModel | None, but _send_terminal_state_msg expects SucceedTask | RetryTask | DeferTask | RescheduleTask. mypy couldn't prove the narrowing at the _replay_pending_terminal_state_msg call site. Tighten the field type to the exact union the setter assigns and the consumer accepts. --------- Co-authored-by: vatsrahul1001 <rah.sharma11@gmail.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com> (cherry picked from commit 173c2a1) * Don't pass retry_delay_seconds/retry_reason to retry() — not in v3-2-test signature The cherry-picked _send_terminal_state_msg dispatcher passed retry_delay_seconds and retry_reason kwargs (via getattr defensive fallback) to client.task_instances.retry(). The v3-2-test version of retry() in task-sdk/src/airflow/sdk/api/client.py only accepts (id, end_date, rendered_map_index) — those kwargs don't exist on this branch yet. In mock-based unit tests the extra kwargs were silently accepted by Mock but tripped assert_called_once_with. In real DB tests (Postgres test_task_instance_history_is_created_when_ti_goes_for_retry, MySQL/SQLite equivalents) the retry() call raised TypeError, the API server never received the retry transition, and TaskInstanceHistory never got created — the test's UUID-rotation assertion failed. --------- Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…dpoint (#67185) (#67211) * Apply requires_access_event_log to GET /eventLogs list endpoint * Fail closed on non-integer event_log_id; fix list-endpoint test mock requires_access_event_log silently swallowed ValueError on non-integer event_log_id and fell through to the generic AUDIT_LOG check. Raise HTTPException(400) instead — matches the fail-closed pattern used by requires_access_backfill. Also fix test_requires_access_event_log_no_path_param_uses_generic_check: the test mocked request.path_params = {} but left request.query_params as an auto-created Mock attribute, whose .get("dag_id") returned a Mock (truthy non-None). requires_access_dag then resolved dag_id to that Mock and called is_authorized_dag with the wrong DagDetails. Mock both path_params and query_params as empty dicts. --------- (cherry picked from commit b28681f) Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com> Co-authored-by: vatsrahul1001 <rah.sharma11@gmail.com>
…active runs (#67249) (#67256) useRefreshOnNewDagRuns was polling every 5 s indefinitely whenever hasPendingRuns was false, including on Dags that are paused and have no active runs. Pass isPaused from the Dag details into the hook and skip the refetchInterval when the Dag is paused and idle. Note: backport applies only the paused-Dag polling fix from #67249. The deadline-related changes are skipped because the deadline-alerts feature is not present in v3-2-test. (cherry picked from commit 3047ad7) Co-authored-by: Brent Bovenzi <brent@astronomer.io>
…ing dag_run (#67246) (#67264) session.get(TI, id, with_for_update=True) emits a SELECT that joins dag_run (via the lazy="joined" relationship) and applies FOR UPDATE to both tables. Under concurrent task completions this serialises all workers on the same dag_run row, producing deadlock cycles with the scheduler's trigger-rule dependency checks. Three other callsites in this file already use with_for_update={"of": TI} for exactly this reason. Apply the same fix to the two remaining callsites in _create_ti_state_update_query_and_update_state and its error-recovery path. (cherry picked from commit 315d159) Co-authored-by: Arthur <arthur.volant@datadoghq.com>
amoghrajesh
approved these changes
May 21, 2026
Contributor
amoghrajesh
left a comment
amoghrajesh
left a comment
There was a problem hiding this comment.
Few nits mostly, looks good! Let's 🚢 it!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
15 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Time for 3.2.2rc1
Release Notes
Version update change
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.