Extract seams for Dag processor callback fetching to allow DB-to-API swap#65543
Open
ephraimbuddy wants to merge 4 commits intoapache:mainfrom
Open
Extract seams for Dag processor callback fetching to allow DB-to-API swap#65543ephraimbuddy wants to merge 4 commits intoapache:mainfrom
ephraimbuddy wants to merge 4 commits intoapache:mainfrom
Conversation
ephraimbuddy
force-pushed
the
refactor-callback-fetching
branch
from
b34eb94 to
d949bea
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Refactors the Dag processor manager’s callback-fetching/queuing flow to introduce overrideable “seams” so the default metadata-DB callback source can later be swapped for an API-backed implementation, without changing the DB-backed behavior.
Changes:
- Adds
fetch_callbacks()as the outer callback-fetching seam (default delegates to the existing DB implementation). - Extracts bundle resolution and bundle initialization into
resolve_callback_bundle()andinitialize_callback_bundle(). - Updates/extends unit tests to cover the new seams and initialization failure behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| airflow-core/src/airflow/dag_processing/manager.py | Introduces fetch_callbacks/resolve_callback_bundle/initialize_callback_bundle and routes the parsing loop through the new seam methods. |
| airflow-core/tests/unit/dag_processing/test_manager.py | Updates existing callback-fetch test docstring and adds unit tests for the new seam methods. |
Lee-W
reviewed
Apr 21, 2026
ephraimbuddy
force-pushed
the
refactor-callback-fetching
branch
from
d949bea to
4abdbb6
Compare
…swap Split the DagFileProcessorManager callback path into discrete methods so metadata-DB access can be swapped for API calls: - fetch_callbacks() — the outer seam; default hits the DB - resolve_callback_bundle() — bundle lookup - initialize_callback_bundle() — bundle preparation, returning whether the callback can proceed No behavior change for the default DB-backed flow.
ephraimbuddy
force-pushed
the
refactor-callback-fetching
branch
from
4abdbb6 to
3a71571
Compare
ashb
reviewed
Apr 21, 2026
ashb
reviewed
Apr 21, 2026
kaxil
reviewed
Apr 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce two seams in
DagFileProcessorManagercallback handling so metadata-DB access can later be swapped for API calls:fetch_callbacks()— the outer seam; default delegates to the DB-backed_fetch_callbacks().prepare_callback_bundle(request) -> BaseDagBundle | None— resolves the bundle for a callback request and, for versioned requests on bundles that support versioning, callsbundle.initialize(). ReturnsNoneto signal the callback should be skipped (bundle no longer configured, or initialization failed)._add_callback_to_queuechecks the returned bundle and skips onNone. No behavior change for the default DB-backed flow — log messages and skip semantics are preserved.Was generative AI tooling used to co-author this PR?
Generated-by: claude opus 4.7