Skip to content

select unassigned asset triggers from db#65792

Merged
jscheffl merged 3 commits into
apache:mainfrom
renat-sagut:bugfix/get-asset-triggers-from-db
Apr 28, 2026
Merged

select unassigned asset triggers from db#65792
jscheffl merged 3 commits into
apache:mainfrom
renat-sagut:bugfix/get-asset-triggers-from-db

Conversation

@renat-sagut
Copy link
Copy Markdown
Contributor

@renat-sagut renat-sagut commented Apr 24, 2026

Fixes a bug where multiple triggerer replicas could pick up the same asset trigger from db. The query now selects only unassigned triggers, similar to task instance triggers.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@renat-sagut renat-sagut marked this pull request as ready for review April 24, 2026 16:55
@renat-sagut renat-sagut changed the title select unassigned triggers from db select unassigned asset triggers from db Apr 24, 2026
Copy link
Copy Markdown
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable for me - with current code ALL triggeres pick up the task in parallel? As I am not an expert would need another pair of eyes for review.

Would request an expert review from @uranusjr or @dabla maybe if not @ashb.

@dabla
Copy link
Copy Markdown
Contributor

dabla commented Apr 24, 2026

Sounds reasonable for me - with current code ALL triggeres pick up the task in parallel? As I am not an expert would need another pair of eyes for review.

Would request an expert review from @uranusjr or @dabla maybe if not @ashb.

I think @ashb or @uranusjr are the biggest experts on that field. Looks good to me but a proof of logging that it works would maybe be an option. Also dunno how SQLAlchemy queries are being unit tested at all. Would be good if we would have a unit test that guards this specific code from regression if future changes would occur. WDYT @jscheffl ?

@jscheffl
Copy link
Copy Markdown
Contributor

Would be good if we would have a unit test that guards this specific code from regression if future changes would occur. WDYT @jscheffl ?

Would be good having a unit test but testing the query for elements is a bit self-fulfuilling. Not sure how a "good test" would look like. I have only bad tests in mind which do not help. But I see there are explicit 4 tests. @renat-sagut can you add a test for this (Or extend one of the existing tests for this)?

@ashb
Copy link
Copy Markdown
Member

ashb commented Apr 26, 2026

Agreed, generally the important thing is to test the behaviour of the query, not the exact query produced

Copy link
Copy Markdown
Member

@ashb ashb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This query uses SELECT ... FOR UPDATE SKIP LOCKED so only one Triggerer will get each row.

What about unit tests?

@renat-sagut renat-sagut reopened this Apr 26, 2026
@renat-sagut
Copy link
Copy Markdown
Contributor Author

Would be good if we would have a unit test that guards this specific code from regression if future changes would occur. WDYT @jscheffl ?

Would be good having a unit test but testing the query for elements is a bit self-fulfuilling. Not sure how a "good test" would look like. I have only bad tests in mind which do not help. But I see there are explicit 4 tests. @renat-sagut can you add a test for this (Or extend one of the existing tests for this)?

added tests

Copy link
Copy Markdown
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good for me! Would like to have a second reviewer looking at it prior merge.

*) Assuming the nit in ruff format is fixed of course...

@renat-sagut renat-sagut force-pushed the bugfix/get-asset-triggers-from-db branch from cd1ffa3 to 52da60e Compare April 26, 2026 13:34
@renat-sagut renat-sagut requested a review from ashb April 26, 2026 20:56
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Apr 27, 2026
Copy link
Copy Markdown
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — small, well-targeted fix that brings the asset-trigger query in line with the task-instance-trigger query at line 427 (alive_triggerer_ids filter). Test coverage is solid: unassigned-included, dead-triggerer-included, alive-triggerer-excluded, and ordering+capacity. Following @jscheffl's approval.

(Out of scope for this PR, but worth a quick look as a follow-up: the callback-trigger branch a few lines above doesn't carry the same alive_triggerer_ids filter — if callbacks can race in the same way, that may want the same fix.)


This review was drafted by an AI-assisted tool and confirmed by an Apache Airflow maintainer. The maintainer approving this PR has read the findings and signed off. If something feels off, please reply on the PR and a maintainer will follow up.

More on how Apache Airflow handles maintainer review: contributing-docs/05_pull_requests.rst.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

@jscheffl jscheffl added this to the Airflow 3.2.2 milestone Apr 28, 2026
@jscheffl jscheffl added the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label Apr 28, 2026
@jscheffl
Copy link
Copy Markdown
Contributor

Marking for backporting to have this in next patch release. As no other feedback... merging now...

@jscheffl jscheffl merged commit 8546c91 into apache:main Apr 28, 2026
78 checks passed
@boring-cyborg
Copy link
Copy Markdown

boring-cyborg Bot commented Apr 28, 2026

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions
Copy link
Copy Markdown
Contributor

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-2-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Apr 28, 2026
* select unassigned triggers from db

* tests

* cleanup
(cherry picked from commit 8546c91)

Co-authored-by: renat-sagut <renat.sagut@gmail.com>
aws-airflow-bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Apr 28, 2026
* select unassigned triggers from db

* tests

* cleanup
(cherry picked from commit 8546c91)

Co-authored-by: renat-sagut <renat.sagut@gmail.com>
jscheffl pushed a commit that referenced this pull request Apr 28, 2026
* select unassigned triggers from db

* tests

* cleanup
(cherry picked from commit 8546c91)

Co-authored-by: renat-sagut <renat.sagut@gmail.com>
vatsrahul1001 pushed a commit that referenced this pull request May 20, 2026
* select unassigned triggers from db

* tests

* cleanup
(cherry picked from commit 8546c91)

Co-authored-by: renat-sagut <renat.sagut@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Triggerer backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants