[v3-2-test] fix: update dependencies to fix dependabot alarms in providers.common.ai (#66244)#66912
Closed
vatsrahul1001 wants to merge 225 commits into
Closed
[v3-2-test] fix: update dependencies to fix dependabot alarms in providers.common.ai (#66244)#66912vatsrahul1001 wants to merge 225 commits into
vatsrahul1001 wants to merge 225 commits into
Conversation
…#63994) (#65226) * Add API check to ensure multi team is enabled when team_name is provided * remove unnecessary arguments in added tests * add variable tests and add slight change to other tests to align with variables test file * Change error message, Modify tests, Add bulk tests, Fix CI issues (cherry picked from commit 6271189) Co-authored-by: ahilashsasidharan <79016853+ahilashsasidharan@users.noreply.github.com>
* Add dag runs filters (Consuming Asset) * Fix: correct consuming asset filter setup using association_table * Trigger CI rebuild * Rename consuming_asset filter to consuming_asset_pattern with database icon * Rename consuming_asset filter to consuming_asset_pattern with database icon * Trigger CI rebuild * Fix consuming_asset_pattern naming * Fix: rename consuming_asset to consuming_asset_pattern * Fix: rename consuming_asset to consuming_asset_pattern * Fix: Resolve PostgreSQL JSON comparison error in _ConsumingAssetFilter * Rebase and fix _ConsumingAssetFilter * Trigger CI * add consumingAsset and filters.searchAsset to en/common.json --------- (cherry picked from commit 5245419) Co-authored-by: fat-catTW <124506982+fat-catTW@users.noreply.github.com> Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…tance (#63923) (#65304) After clearing a task instance, the TaskInstances list page was not refreshing to show the updated state. This was because `useClearTaskInstances` was missing `[useTaskInstanceServiceGetTaskInstancesKey]` in the list of query keys to invalidate on success. Both `useClearRun` and `usePatchTaskInstance` correctly invalidate this query — this change brings `useClearTaskInstances` in line with them. Fixes: #60703 (cherry picked from commit f47038e) Co-authored-by: nagasrisai <59650078+nagasrisai@users.noreply.github.com>
The sphinx_airflow_theme default navbar_links includes Registry, but the docs override navbar_links in get_html_theme_options(), so the theme default never applies. (cherry picked from commit d988f75) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
…e_fileloc + bundle (#65329) (#65343) The public Import Errors API used to match ParseImportError.filename against DagModel.fileloc. In real deployments ``fileloc`` is an absolute path while ``filename`` is relative, so the file-to-DAG resolution often came back empty and the single endpoint fell through to returning the raw error. The list endpoint had a related gap: its CTE was pre-filtered by the caller-visible subset of DAGs, so the per-file authorization check only ever saw the DAGs the caller could already read -- a file containing a mix of readable and unreadable DAGs passed the check on the readable subset alone. * The single endpoint now matches ParseImportError.filename against DagModel.relative_fileloc + DagModel.bundle_name, which is the same key the list endpoint already uses for its join. When the resolved DAG set is empty (parse failed before any DAG was defined, or the name keys did not resolve), the stacktrace is now redacted rather than returned verbatim. * The list endpoint splits the previous ``visible_files_cte`` into two CTEs: ``readable_files_cte`` enumerates the ``(relative_fileloc, bundle_name)`` pairs where the caller can read at least one DAG, and ``file_dags_cte`` enumerates the full ``(relative_fileloc, dag_id, bundle_name)`` set for those files. The per-file authorization check in the groupby loop now receives the complete DAG set for each file and can correctly detect co-located DAGs outside the caller's scope. * The same fall-through in the list endpoint (file has no matching DAGs in DagModel) now redacts the stacktrace before appending. Add a test class that exercises the fix with distinct ``fileloc`` (absolute) and ``relative_fileloc`` (relative) string values, closing the test-fixture gap where both columns previously held the same relative string and the absolute-vs-relative mismatch could not manifest. One existing single-endpoint test documenting the previous fall-through behaviour is updated to assert the new redaction. (cherry picked from commit eba9b65) Generated-by: Claude Opus 4.6 (1M context) following the guidelines at https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…se log folder (#65325) (#65345) * Refuse to follow log symlinks that resolve outside the base log folder FileTaskHandler._read_from_local used to open every file that matched the task's log glob pattern, including symlinks whose real path was outside the configured base_log_folder. On deployments where worker logs are accessible from the api-server, that meant the log viewer could end up streaming content from files outside the configured log tree whenever a symlink in the task log directory happened to match the glob pattern. Canonicalise self.local_base once via os.path.realpath and, for every glob hit, resolve the path with os.path.realpath and skip it if the resolved form is not contained in the canonicalised base log folder (using os.path.commonpath, with a ValueError fallback for the different-drive case on Windows). Open the resolved path rather than the original glob hit so the file we open is the one we just validated. Append to sources only after a successful open so sources and log_streams stay aligned. Drop the @staticmethod decorator so the method can read self.local_base; existing call sites already invoke it via self. Add a test class covering: regular-file-inside-base is still streamed; a symlink whose real path is outside base_log_folder is skipped; a symlink that stays inside base_log_folder is followed (legitimate rotation case); and base_log_folder itself being a symlink works. Generated-by: Claude Opus 4.6 (1M context) following the guidelines at https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions * Fix test__read_from_local to use valid base_log_folder The existing test passed an empty string as base_log_folder, which after the containment check resolves to CWD via os.path.realpath(""), causing all files under tmp_path to be rejected. Use tmp_path instead. (cherry picked from commit 3eda845) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…65348) (#65363) JWTRefreshMiddleware derived the cookie Secure flag from the local api.ssl_cert config only. Deployments with TLS terminated at a reverse proxy (no local SSL cert on the Airflow process) therefore received the JWT refresh cookie without the Secure flag. Match the pattern already used by every other cookie-setting location in the codebase (auth.py, simple/routes/login.py, FAB and Keycloak login routes): treat secure as True when either the request came in over HTTPS or a local ssl_cert is configured. (cherry picked from commit 60db83f) Generated-by: Claude Opus 4.6 (1M context) following the guidelines at https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…_id_pattern (#65309) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * Fix: PATCH /dags pagination bug and document wildcard dag_id_pattern (#63665) * fixed pagination bug and updated docstring to clarify dag_id_pattern wildcard usage * removed batch loop to update all dags in one shot and added additional test case * Fixed MySQL subquery issue (cherry picked from commit 9504886) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com>
…nection. (#65231) (#65368) This was discovered by running with a custom External DB manager that had some gnarly queries that ended up being locked behind this transaction. `_single_connection_pool` replaces `settings.engine` with a SingletonThreadPool engine. But `work_session` was created before that — it still holds an internal reference to the old engine object. _single_connection_pool has no way to rebind work_session. So when _get_current_revision(session=work_session) runs on line 1203 — inside the _single_connection_pool() block — it calls session.connection() which goes through the old engine, not the SingletonThreadPool. The old engine's pool was disposed and recreated empty by engine.dispose(), so it creates a brand new connection. That connection is completely outside _single_connection_pool's control. _single_connection_pool guarantees one connection on the new engine. It can't prevent work_session from creating connections on the old one. The name is a bit of a lie — it's really "single connection pool for new code that uses settings.engine", not "single connection total." (cherry picked from commit e3fea3a) (cherry picked from commit f8e0876) Co-authored-by: Ash Berlin-Taylor <ash@apache.org>
…5167) (#65321) * [v3-2-test] Bump actions/github-script in the github-actions-updates group (#65150) (#65160) Bumps the github-actions-updates group with 1 update: [actions/github-script](https://github.com/actions/github-script). Updates `actions/github-script` from 8.0.0 to 9.0.0 - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@ed59741...3a2844b) (cherry picked from commit e5a047c) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-2-test] Added breeze generate issue content for airflow-ctl (#65042) (#65241) * Add breeze generate issue content for airflow-ctl * add new command to doc (cherry picked from commit b24538b) Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> * [v3-2-test] Run release calendar verification on its own schedule (#65118) (#65242) * Move release calendar verification to its own scheduled workflow Run dev/verify_release_calendar.py from a dedicated daily scheduled workflow instead of as a canary job in the main CI pipeline, and notify the #release-management Slack channel when the check fails so the issue is surfaced to release managers directly. * Include wiki and calendar links in release calendar Slack alert (cherry picked from commit 048e9a1) * [v3-2-test] fix(ui): register trigger and sensor graph node types (#65167) * fix(ui): register trigger and sensor graph node types Adds missing Graph node type mappings for trigger/sensor and includes a focused unit test to prevent regressions where dependency graph rendering breaks for those node kinds. * docs(ui): add graph screenshot showing sensor and trigger nodes * chore(ui): keep PR scoped to graphTypes.ts only --------- (cherry picked from commit e0ed795) Co-authored-by: Windro.xd <88357206+windro-xdd@users.noreply.github.com> Co-authored-by: Kripa Dev <dev@kripa-car-care.local> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Justin Pakzad <114518232+justinpakzad@users.noreply.github.com> Co-authored-by: Windro.xd <88357206+windro-xdd@users.noreply.github.com> Co-authored-by: Kripa Dev <dev@kripa-car-care.local>
…#65326) (#65334) Mypy checks for non-provider projects now synchronize the local virtualenv with uv.lock (uv sync --frozen) before running, so contributors see the same dependency set CI uses and avoid results that drift from CI. The update-uv-lock prek hook now runs with --frozen, so pyproject.toml changes that would touch uv.lock fail the hook and require an explicit uv lock + commit instead of silently rewriting the lock during a commit. (cherry picked from commit 9b08d05) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…64863) (#65473) * CI: Avoid false recovery alerts when failed job lookup fails * Potential fix for pull request finding --------- (cherry picked from commit b41b11d) Co-authored-by: Henry Chen <henryhenry0512@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…)" (#65515) (#65516) * Revert "Stop masking quarantined unit test failures (#65500)" This reverts commit 67e7cc1. * Document why quarantined tests use `|| true` Adds a short comment next to both `breeze testing ... --test-type "All-Quarantined" || true` calls explaining that the shell `|| true` is intentional — quarantined tests are known-flaky and must not fail the overall CI run; they are reported separately. (cherry picked from commit b7fcda5) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…65405) * Add cursor-based pagination to get_task_instances endpoint - Add cursor-based (keyset) pagination as an alternative to offset-based pagination on the get_task_instances endpoint. Offset pagination remains the default and is not deprecated globally. - Response uses a discriminated union: offset responses include total_entries, cursor responses include next_cursor and previous_cursor. - Refactor SortParam to lazily cache column resolution instead of mutating state in to_orm. - Move cursor helpers (encode/decode/apply) to dedicated common/db/cursors.py module. - Cleanly separate cursor vs offset code paths in the endpoint handler. * Simplify cursor token and support first page without sentinel - Remove order_by from cursor token (now just a list of values) - Support empty string cursor for first page (no fake sentinel needed) - Drop order_by consistency check between cursor and query param * Small adjustments * Adjustments * Narrow endpoint return types and encode cursor value types Encode type information directly into cursor tokens as {"type": ..., "value": ...} objects, removing the fragile column-based type guessing during deserialization. Narrow return types for endpoints that only return offset pagination (patch, clear, batch, mapped) so the OpenAPI spec and generated UI client reflect the correct types. Only get_task_instances retains the discriminated union response. Update UI components to use the narrowed types from the spec. * Use msgpack for cursor tokens and nested keyset predicate Switch cursor encoding from typed JSON to msgpack for compactness. Replace flat OR-of-prefix-equalities with nested and/or keyset predicate for better composite index range scans. Always use ascending PK as the final tie-breaker for stable pagination. * Flatten TaskInstanceCollectionRes ponse to avoid oneOf codegen issues Replace the discriminated union (offset | cursor response types) with a single flat model using optional fields. OpenAPI oneOf + discriminator is not handled correctly by hey-api/openapi-ts (#1613, #3270): return types degrade to unknown in generated TypeScript code. * Fix UI * Fix CI * Fix cursor pagination boundary detection and error handling - Fetch limit+1 rows to accurately detect last page, returning next_cursor=null when no more results exist - Return previous_cursor=null on the first page (when no cursor was provided) - Use LimitFilter in apply_filters_to_select for the +1 limit instead of a manual .limit() call - Raise HTTP 400 on invalid UUID in cursor token instead of silently passing the invalid value - Update endpoint docs and add boundary-condition test * Fix backward cursor based pagination (cherry picked from commit e11c603)
…bc provider (#65532) (#65548) jpype1 1.7.0 stopped shipping prebuilt macOS arm64 wheels and tries to build from source against a JDK on Apple Silicon, which breaks `uv sync` out of the box on those machines. Exclude jpype1 1.7.0 as a direct dependency of the jdbc provider, scoped to darwin-arm64 only so every other platform (including macOS x86_64) still resolves to the latest wheels. Upstream plans to restore the arm64 wheels in 1.7.1 (jpype-project/jpype#1357), after which the exclusion can be dropped. (cherry picked from commit 8ed50ae)
…ts (#65344) (#65538) The partitioned_dag_runs endpoints enforced only asset-level access control. Add requires_access_dag and ReadableDagsFilterDep to match the pattern used by the sibling next_run_assets endpoint in assets.py. (cherry picked from commit e36d10a) Generated-by: Claude Opus 4.6 (1M context) following the guidelines at https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…5552) The provider_dependency_bump and common_compat_changed_without_next_version checks in selective_checks are meant to guard the main branch from accidental release-manager-only changes. On release branches (v3-X-test), those same changes are expected during cherry-picks, and forcing contributors to set override labels on every backport adds friction without value. Skip both checks when the target branch is not main so release-branch PRs no longer need the override labels. (cherry picked from commit 3dd2ba0) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…on stdout (#65528) (#65541) This is very useful when running things in a container, where it's much easier to get container logs than files on disk. This config option was added in 2.4.0 but got broken around 3.0 or 3.1. Most of capabilities existed already, we just needed to pass the right value down. Example log line: ``` 2026-04-20T10:44:13.977892Z [info ] Ptint from dag [dag_processor.stdout] bundle_name=main dag_file=dags/example_dag_advanced.py ``` Since we've got structured logging now it includes the bundle name and rel path of the dag file. (cherry picked from commit 1193073) Co-authored-by: Ash Berlin-Taylor <ash@apache.org>
* UI: Switch TaskInstances table to cursor-based pagination Replace offset-based pagination with cursor-based pagination for the TaskInstances listing page, leveraging the new cursor API endpoint. Pagination now shows only previous/next buttons without page numbers or total count, which eliminates the expensive COUNT(*) query for large datasets. Add generic cursor pagination support to DataTable via an optional cursorPagination prop so other tables can adopt it. * Small adjustments and cleaning * Address review: cursor in TableState, no useCallback/useEffect * Reuse Pagination.Root for cursor pagination instead of custom buttons * Small updates (cherry picked from commit 44ae2bc)
…icated .build/ venvs (#65492) (#65549) * Isolate mypy prek hooks, cover all non-provider dirs, and clean up type errors Each non-provider mypy prek hook now builds and caches its own virtualenv at .build/mypy-venvs/<hook>/ and its own mypy cache at .build/mypy-caches/<hook>/. UV_PROJECT_ENVIRONMENT redirects uv away from the project's .venv, so running the hook never mutates a contributor's regular development environment while still matching CI's frozen dependency set. Mypy runs with --follow-imports=silent so each hook only reports errors for files it owns; transitive code is covered by its own hook and different venvs no longer produce divergent results on shared code. Adds mypy hooks for the non-provider directories that were previously uncovered: airflow-ctl-tests, helm-tests, airflow-e2e-tests, task-sdk-integration-tests, docker-tests, kubernetes-tests, and shared. The mypy-shared hook iterates every shared/<dist> workspace distribution and builds a separate venv + cache per distribution so each shared library is type-checked against its own dependency set. breeze down --cleanup-mypy-cache additionally removes .build/mypy-venvs/ and .build/mypy-caches/ so all per-hook state is wiped alongside the existing .mypy_cache and mypy-cache-volume. Also fixes pre-existing type errors surfaced by the newly added and cleaned-up checks: platform-specific ignores for Linux-only os.posix_fadvise in the shared logging helper, narrower types and type: ignore where appropriate in shared configuration/observability/ timezones/secrets_backend/secrets_masker, Liskov override markers on the AirflowConfigParser subclass methods, and small correctness fixes in dev/breeze and the docker-tests / kubernetes-tests helpers so the full non-provider mypy suite runs clean on macOS and in CI. * Move mypy prek hooks to their respective distribution configs The new mypy hooks for airflow-ctl-tests, helm-tests, airflow-e2e-tests, task-sdk-integration-tests, docker-tests, and kubernetes-tests now live in each distribution's own .pre-commit-config.yaml, matching the pattern already used by airflow-core, task-sdk, and airflow-ctl. New .pre-commit- config.yaml files are added to distributions that didn't have one. prek auto-discovers nested configs, so the hooks remain part of the default check set. mypy-dev (covers dev + scripts), mypy-devel-common, and mypy-shared stay at the repo root: dev/scripts/devel-common don't have their own configs, and mypy-shared iterates every shared/<dist> distribution so has no single home. * Split mypy-dev and mypy-scripts, each with its own pyproject.toml config Previously the mypy-dev prek hook ran mypy against dev/ and scripts/ in a single invocation under the dev project's virtualenv. The two now get independent hooks — mypy-dev in dev/.pre-commit-config.yaml and mypy-scripts in scripts/.pre-commit-config.yaml — so each can evolve its own dependency set and check its own folder. Copy the full [tool.mypy] section from the root pyproject.toml into both dev/pyproject.toml and scripts/pyproject.toml so each sub-project owns its mypy configuration. Paths inside mypy_path are rewritten from $MYPY_CONFIG_FILE_DIR/ to $MYPY_CONFIG_FILE_DIR/../ so they still resolve to the repo-root siblings from the sub-project location. The decorator/ outputs plugins are scoped to dev only (scripts does not author DAG code). mypy_local_folder.py now passes --config-file <project>/pyproject.toml when the folder maps to one of these sub-project configs, so mypy uses the sub-project's configuration rather than the root one. * Teach selective-checks about the new non-provider mypy hooks Add FileGroupForCi entries and regex patterns for helm-tests, airflow-e2e-tests, docker-tests, kubernetes-tests, scripts, and shared Python files, then wire them into skip_prek_hooks so the corresponding mypy-* prek hook is only kept when its folder changed: - mypy-scripts (split off from the old combined mypy-dev) - mypy-airflow-ctl-tests, mypy-helm-tests, mypy-airflow-e2e-tests, mypy-task-sdk-integration-tests, mypy-docker-tests, mypy-kubernetes-tests - mypy-shared Update test_selective_checks.py skip-list constants and per-case inline skip lists to include the new hooks. Targeted test cases for files under the new-hook directories override skip-prek-hooks to leave the matching hook out of the skip set, confirming it will run when its folder changes. * Trim dev/scripts pyproject mypy_path to just relevant distributions Drop the 200+ provider path entries that were blindly copied from the root pyproject.toml. dev and scripts only import from other non-provider workspace members, so listing every provider src/tests directory under mypy_path just adds noise. The remaining non-provider entries cover everything dev or scripts plausibly import from. * Install mypy into per-hook venvs from uv.lock via a `mypy` dep group Each non-provider distribution with a mypy prek hook now declares a `mypy` dependency group in its pyproject.toml resolving to `apache-airflow-devel-common[mypy]`. mypy_local_folder.py syncs each dedicated virtualenv with `uv sync --frozen --project <X> --group mypy` and runs mypy with `uv run --frozen --project <X> --group mypy` — so mypy and its type stubs come from the workspace uv.lock, not from an ephemeral `--with` overlay whose resolution is independent of the main lockfile. uv.lock is refreshed to include the new group. Covers airflow-core, task-sdk, airflow-ctl, devel-common, dev, scripts, airflow-ctl-tests, helm-tests, airflow-e2e-tests, task-sdk-integration- tests, docker-tests, kubernetes-tests, and every shared/<dist> workspace member. * Drop mypy_path from dev/scripts pyprojects — venv site-packages is enough After the switch to installing mypy (and every transitive workspace dependency) directly into each hook's virtualenv via the `mypy` dep group, workspace packages like airflow, airflow.sdk, airflowctl, airflow_breeze, tests_common are all available via the venv's site-packages. mypy resolves them without needing mypy_path entries, so drop the copied list and leave a short comment explaining why. * Split mypy-shared into per-distribution hooks and enforce the pattern Each shared/<dist> workspace member now owns a mypy-shared-<dist> prek hook backed by its own shared/<dist>/.pre-commit-config.yaml. The single mypy-shared iterator is gone — mypy_local_folder.py accepts shared/<dist> as a first-class folder and the per-hook virtualenv now lives at .build/mypy-venvs/shared-<dist>/ (slash in the folder name is replaced with a dash in the venv/cache path). Adds a new check-shared-mypy-hooks prek hook that fails when a shared/<dist> workspace member is missing its dedicated .pre-commit- config.yaml, printing the exact YAML to add. Selective-checks emits one skip entry per dist, enumerated from shared/ at run time. Contributing docs cover the two-step process for adding a new shared library. * Pin minimum_prek_version to 0.3.4 consistently across all configs All .pre-commit-config.yaml files now require prek >= 0.3.4 (the version already declared by the root config). Previously the nested configs pinned a mix of 0.2.0, 0.3.2, and 0.3.4, so a contributor could pass the root's version check and still trip on stale subproject pins as they moved between directories. * Refresh uv.lock after rebase to reflect the `mypy` dep groups The rebase onto main resolved the uv.lock conflict by taking main's version, so `uv sync --group mypy` would fail against uv.lock until the groups added to the per-distribution pyprojects were re-resolved. Regenerates the lockfile to include them. * Add explicit selective-checks test for per-shared-dist mypy hook skipping Verifies that when a file under shared/logging/ changes, only mypy-shared-logging is kept among the thirteen mypy-shared-* hooks; all other shared distributions' hooks land in the skip list. Pins the contract that the runtime enumeration over shared/*/pyproject.toml works as intended. * Refresh mypy docs to match the per-hook venv + --group mypy workflow Fills in the docs that still referenced the pre-split workflow: - AGENTS.md: mentions `mypy-shared-<dist>` per shared workspace member and the `uv sync --group mypy` install path for mypy itself. - scripts/ci/prek/AGENTS.md: clarifies that non-provider mypy hooks run locally through mypy_local_folder.py (Breeze image only needed for the providers hook). - dev/breeze/doc/03_developer_tasks.rst: renames stale `mypy-airflow` to `mypy-airflow-core`, and expands the cache note to cover the per-hook virtualenvs and caches under .build/. - dev/breeze/doc/ci/04_selective_checks.md: expands the file-group and skip-reason lists so every new mypy hook (scripts, task-sdk, airflow-ctl, the six test-dir hooks, and mypy-shared-<dist> enumerated at runtime) is documented. * Rename mypy_local_folder.py to run_mypy_full_dist_local_venv_or_breeze_in_ci.py Updates every .pre-commit-config.yaml entry and prose references so they point at the new script name. Two shared configs use YAML folded-scalar entries to stay under the 110-char yamllint limit; updates the validation script's expected template to match. (cherry picked from commit 4f3b228)
…cture_data endpoint (#65342) (#65534) The structure_data endpoint returned external dependency nodes for linked DAGs without checking whether the caller had read permission on those DAGs. Add the ReadableDagsFilterDep and skip dependency entries that reference DAGs outside the caller's readable set. (cherry picked from commit 01888df) Generated-by: Claude Opus 4.6 (1M context) following the guidelines at https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…edge cases (#65190) (#65565) When a pattern like "term|" or "|term" was passed, the OR-split guard `if len(search_terms) > 1` evaluated False for the single remaining term, falling through to `ilike("%term|%")` / `ilike("%|term%")` which matched nothing. Changing the guard to `if search_terms:` ensures a single valid term is used correctly. (cherry picked from commit 43556cc) closes: #65129 Co-authored-by: Cole Heflin <75401093+coleheflin@users.noreply.github.com>
* Add deadlines support with name and description fields in alerts and UI * Add 'viewAll' label to deadlineStatus in dag.json * Refactor deadlineAlerts referenceType structure * Refine deadline alert translations in dag.json Updated deadline alert messages for clarity and consistency. * Add completion rule text for deadline alerts in UI * Remove duplicate completionRule entry in deadlineAlerts * Remove alert description from DeadlineAlert and related models * Enhance deadline handling: return name updates alongside UUID mapping in SerializedDagModel * Add alert_id field to DeadlineResponse and update tests for alert handling * Remove DEADLINES option from MenuItem enum * Update airflow-core/src/airflow/serialization/encoders.py --------- (cherry picked from commit e9d1066) Co-authored-by: Richard Wu <richard9@ualberta.ca> Co-authored-by: D. Ferruzzi <ferruzzi@amazon.com>
…5599) * Fix backfill params not overriding existing DAG run conf When reprocessing an existing DAG run during backfill, the dag_run_conf was not being applied to the cleared run. This adds the dag_run_conf parameter to _handle_clear_run() and conditionally updates conf in the DagRun UPDATE statement. Also adds conf validation in _create_backfill() using dag.params.deep_merge().validate() to match the validation done in create_dagrun() for new runs, ensuring invalid conf is rejected before any runs are created or cleared. closes: #59043 * UI: Add override params checkbox to backfill form Add an "Override parameters on existing runs" checkbox to the backfill form, unchecked by default. When unchecked, dag_run_conf is sent as null, preserving existing run params during reprocessing. When checked, the form values are sent and override existing conf. Also changes BackfillPostBody.dag_run_conf from dict={} to dict|None=None so the API can distinguish between "no conf provided" (null, preserve existing) and "empty conf" ({}, override). related: #59043 * Address code review: pass null through and regenerate openapi artifacts * Address review: InvalidBackfillConf exception and partitioned path comment * UI: Disable max-lines in RunBackfillForm with justification * Apply suggestion from @uranusjr * Apply suggestion from @uranusjr: chained .values() --------- (cherry picked from commit 8b401f2) Co-authored-by: Shivam Rastogi <6463385+shivaam@users.noreply.github.com> Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
vatsrahul1001
requested review from
Lee-W,
XD-DENG,
amoghrajesh,
ashb,
bbovenzi,
bolkedebruin,
bugraoz93,
choo121600,
dheerajturaga,
dstandish,
ephraimbuddy,
guan404ming,
hussein-awala,
jason810496,
jedcunningham,
jscheffl,
mobuchowski,
o-nikolas,
pierrejeambrun,
potiuk,
rawwar,
romsharon98,
ryanahamilton,
shahar1,
shubhamraj-git,
uranusjr and
vincbeck
as code owners
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick of #66244
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.7) following the guidelines