Skip to content

Fix race condition on sort param when returning dagRuns#68842

Merged
pierrejeambrun merged 2 commits into
apache:mainfrom
HasnainRaz:fix-race-on-get-dagRuns
Jun 24, 2026
Merged

Fix race condition on sort param when returning dagRuns#68842
pierrejeambrun merged 2 commits into
apache:mainfrom
HasnainRaz:fix-race-on-get-dagRuns

Conversation

@HasnainRaz

@HasnainRaz HasnainRaz commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

I have a use case where I continously poll the GET: /api/v2/dags/{dag_id}/dagRuns endpoint with a sort parameter defined. The response of the endpoint is fine, but once in a couple of hours, it will return the wrong ordering.

I traced it to this change in the PR, essentially, the instantiated object state is mutated if there is a race condition between two requests with different sort parameters, leading to incorrect results for one of the requests.

An example to reproduce:

session = requests.Session()
## AIRFLOW_BASE_URL is internal airflow hostname + /api/v2
def fetch_runs(order_by, limit=5):
    url = f"{AIRFLOW_BASE_URL}/dags/{DAG_ID}/dagRuns"
    resp = session.get(url, params={"order_by": order_by, "limit": limit}, headers=auth_headers)
    print(f"order_by='{order_by}', limit={limit} (HTTP {resp.status_code})")
    runs = resp.json().get("dag_runs", [])
    for r in runs:
        print(f" dag_run_id={r['dag_run_id']}  logical_date={r['logical_date']}  state={r['state']}")
    return runs

a = fetch_runs("-id")

for _ in range(10000):
    assert a = fetch_runs("-id")

This raises AssertionError after a couple thousand iterations. And the print statements show the returned results have suddenly changed (older dagRuns being returned, probably because default sort args got used in the race condition). The airflow instance is accessed by other users, so I suspect the race is happening between different user's requests. Mostly it would go unnoticed if people were using default sort params, but using descending sort makes this surface.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added the area:API Airflow's REST/HTTP API label Jun 22, 2026
@boring-cyborg

boring-cyborg Bot commented Jun 22, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@pierrejeambrun pierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@pierrejeambrun pierrejeambrun merged commit 31f7ba7 into apache:main Jun 24, 2026
77 checks passed
@pierrejeambrun pierrejeambrun added this to the Airflow 3.3.1 milestone Jun 24, 2026
@boring-cyborg

boring-cyborg Bot commented Jun 24, 2026

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@pierrejeambrun pierrejeambrun added the backport-to-v3-3-test Backport to v3-3-test label Jun 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-3-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Jun 24, 2026
…pache#68842)

(cherry picked from commit 31f7ba7)

Co-authored-by: Hasnain Raza <4294680+HasnainRaz@users.noreply.github.com>
aws-airflow-bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Jun 24, 2026
…pache#68842)

(cherry picked from commit 31f7ba7)

Co-authored-by: Hasnain Raza <4294680+HasnainRaz@users.noreply.github.com>
pierrejeambrun pushed a commit that referenced this pull request Jun 25, 2026
…68842) (#68948)

(cherry picked from commit 31f7ba7)

Co-authored-by: Hasnain Raza <4294680+HasnainRaz@users.noreply.github.com>
karenbraganz pushed a commit to karenbraganz/airflow that referenced this pull request Jun 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API backport-to-v3-3-test Backport to v3-3-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants