Skip to content

Thread version_data to callbacks#69185

Merged
o-nikolas merged 4 commits into
apache:mainfrom
aws-mwaa:onikolas/pr3/s3-bundle-version-callbacks
Jul 7, 2026
Merged

Thread version_data to callbacks#69185
o-nikolas merged 4 commits into
apache:mainfrom
aws-mwaa:onikolas/pr3/s3-bundle-version-callbacks

Conversation

@o-nikolas

@o-nikolas o-nikolas commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

PR #67217 wired version_data into the task execution path but not callbacks, so callbacks for a pinned run initialized their bundle without the manifest tasks used. Populate version_data on the callback producer paths (ExecuteCallback.make and the CallbackRequest creation sites) under the same pin guard as tasks, carry it on BaseCallbackRequest, and forward it through prepare_callback_bundle to get_bundle.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added area:DAG-processing area:Executors-core LocalExecutor & SequentialExecutor area:Scheduler including HA (high availability) scheduler area:Triggerer labels Jun 30, 2026
@o-nikolas o-nikolas force-pushed the onikolas/pr3/s3-bundle-version-callbacks branch from e3bace1 to a9cf234 Compare June 30, 2026 16:39

@SameerMesiah97 SameerMesiah97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits that you can choose to address. Otherwise, this looks good to me.

Comment thread airflow-core/src/airflow/callbacks/callback_requests.py Outdated
Comment thread airflow-core/src/airflow/models/dag_version.py Outdated
Comment thread airflow-core/src/airflow/models/dag_version.py Outdated
Comment thread airflow-core/tests/unit/dag_processing/test_manager.py
Comment thread airflow-core/src/airflow/models/trigger.py
Comment thread airflow-core/src/airflow/models/trigger.py

@ramitkataria ramitkataria left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the already open threads, looks good to me!

o-nikolas added 4 commits July 6, 2026 16:11
PR apache#67217 wired version_data into the task execution path but not
callbacks, so callbacks for a pinned run initialized their bundle
without the manifest tasks used. Populate version_data on the
callback producer paths (ExecuteCallback.make and the
CallbackRequest creation sites) under the same pin guard as tasks,
carry it on BaseCallbackRequest, and forward it through
prepare_callback_bundle to get_bundle.
@o-nikolas o-nikolas force-pushed the onikolas/pr3/s3-bundle-version-callbacks branch from 91bdcf5 to bb04f5e Compare July 7, 2026 00:48

@ferruzzi ferruzzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like all concerns have been addressed.

@o-nikolas o-nikolas merged commit 101bc15 into apache:main Jul 7, 2026
79 checks passed
@o-nikolas o-nikolas deleted the onikolas/pr3/s3-bundle-version-callbacks branch July 7, 2026 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:DAG-processing area:Executors-core LocalExecutor & SequentialExecutor area:Scheduler including HA (high availability) scheduler area:Triggerer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants