Return only the latest Dag version from iter_all_latest_version_dags - #70935
Open
Eason09053360 wants to merge 2 commits into
Open
Return only the latest Dag version from iter_all_latest_version_dags#70935Eason09053360 wants to merge 2 commits into
Eason09053360 wants to merge 2 commits into
Conversation
The method promised latest-version dags in its name and docstring but read every serialized_dag row, so FAB permission syncing processed each Dag once per version and could apply an outdated access_control last.
Eason09053360
marked this pull request as ready for review
August 2, 2026 08:36
Eason09053360
force-pushed
the
fix-iter-all-latest-version-dags
branch
from
August 2, 2026 17:29
3b3e4a6 to
b40af4f
Compare
SameerMesiah97
left a comment
Contributor
There was a problem hiding this comment.
Just one comment on the tests.
| db.clear_db_runs() | ||
| db.clear_db_dags() | ||
| db.clear_db_serialized_dags() | ||
| db.clear_db_dag_bundles() |
Contributor
There was a problem hiding this comment.
Should we also add a regression covering multiple DAG IDs with multiple versions, to explicitly verify we return the latest version per DAG rather than just the latest serialized row overall?
Contributor
Author
There was a problem hiding this comment.
Good catch. With a single Dag the test couldn't distinguish a per-Dag max from a
global one, so group_by(dag_id) was never exercised .
I confirmed the old test still passed with that group_by removed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The method promised latest-version dags in its name and docstring but read every serialized_dag row, so FAB permission syncing processed each Dag once per version and could apply an outdated access_control last.
DBDagBag.iter_all_latest_version_dagspromises latest-version Dags in both itsname and its docstring, but it read every
serialized_dagrow without filteringby version.
Its only production caller is FAB's
create_dag_specific_permissions, so DAG-levelpermission syncing iterated each Dag once per version. Besides the wasted work, the
query has no
ORDER BY, so when versions carry differentaccess_controlthe oneapplied last is undefined and an outdated policy can win.
SerializedDagModel._latest_by_version_select()already supported "all dags" viadag_ids=None; this exposes that through the publicget_latest_serialized_dags()wrapper, which keeps the existing
load_op_linkshook in the loop intact. All fiveexisting callers pass
dag_ids=explicitly, so the signature change is backwardscompatible.
The existing test mocked
session.scalars, so it could not catch a wrong query.Added a regression test that writes two real versions to the database.
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.