fix serialize_template_field handling callable value in dict#63871
Open
wjddn279 wants to merge 4 commits intoapache:mainfrom
Open
fix serialize_template_field handling callable value in dict#63871wjddn279 wants to merge 4 commits intoapache:mainfrom
wjddn279 wants to merge 4 commits intoapache:mainfrom
Conversation
4a10da9 to
6f6156f
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes non-deterministic DAG serialization when templated dict fields contain callable values (e.g., lambdas), preventing serialized output from changing on every run due to function object addresses.
Changes:
- Sanitizes callable values during recursive dict sorting in
serialize_template_field. - Refactors callable stringification into a helper for consistent formatting.
- Adds unit tests intended to validate stable serialization when dicts contain callables.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| airflow-core/src/airflow/serialization/helpers.py | Makes callable values in dicts serialize to a stable string during recursive sorting. |
| airflow-core/tests/unit/serialization/test_helpers.py | Adds regression tests around dicts containing callables to prevent unstable serialization. |
2e7b627 to
7c4b179
Compare
7c4b179 to
8eb05f8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closed: #63334
Cause
The serialization result of the DAG is as follows.
When serializing a dict containing lambda function definitions, the object's id value is reflected as-is, causing version changes on every serialization. The root cause is that when a dict contains callable objects, is_jsonable returns False, and since the dict itself is not callable, it falls through to that code path.
Solution
I added logic to filter out this case and convert callable objects consistently within the existing dict handling (sorting) logic. I have confirmed locally that this resolves the issue.
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.