feat: Type-aware Durable Functions payload serialization#343
Open
andystaples wants to merge 1 commit into
Open
feat: Type-aware Durable Functions payload serialization#343andystaples wants to merge 1 commit into
andystaples wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an opt-in, type-aware JSON codec for Durable Functions payloads in
azure.functions._durable_functions, routesActivityTriggerConverterthrough it, and replaces the on-demandimportlib.import_modulecall in_deserialize_custom_objectwith asys.moduleslookup.What changed
azure/functions/_durable_functions.pyNew public API:
df_dumps(value) -> str— JSON-encode a value using the Durable Functions convention.df_loads(s, expected_type=None) -> Any— JSON-decode a string, optionally validating against an expected Python type and using it to reconstruct custom objects.New helper exposed for callers that build their own
json.dumpsinvocations (e.g. orchestrator state encoders):_get_serialize_default()— returns thedefault=callback to pass tojson.dumpsunder the active typing mode.Two operating modes, selected at runtime via the environment variable
AZURE_FUNCTIONS_DURABLE_STRICT_TYPING(truthy values:1,true,yes, case-insensitive):df_dumpsisjson.dumps(value, default=_serialize_custom_object).df_loadsruns the existingobject_hookpath so nested custom objects continue to be reconstructed automatically. Passingexpected_typeadds a class/module check that logs a warning on mismatch but otherwise preserves the legacy decode path.df_dumpsonly wraps the top-level custom object —__data__is serialized as plain JSON with nodefault=hook, soto_json()is responsible for serializing nested custom objects explicitly.df_loadsparses without anobject_hook; deserializing a custom-object envelope requires the caller to passexpected_type, which is then used to callexpected_type.from_json(__data__)directly. Type mismatches raiseTypeError._serialize_custom_objectis unchanged._deserialize_custom_objectnow resolves the declared class viasys.modules.get(...)instead ofimportlib.import_module(...).azure/functions/durable_functions.pyActivityTriggerConverter:decodenow calls_durable_functions.df_loads(data.value)in place of the inlinejson.loads(..., object_hook=...).encodenow calls_durable_functions.df_dumps(obj)in place of the inlinejson.dumps(..., default=...).expected_typebecauseInConverter.decodedoesn't receive the function's parameter type annotation from the worker — flagged as a follow-up.Other converters (
OrchestrationTriggerConverter,EnitityTriggerConverter,DurableClientConverter) are not touched.Behavior changes for existing callers
The wire format is unchanged. Default
df_dumpsoutput equals whatjson.dumps(value, default=_serialize_custom_object)produced before. The behavior deltas are:_deserialize_custom_object(and thereforedf_loads/json.loads(..., object_hook=...)) now resolves the declared class viasys.modules.get(module_name)instead ofimportlib.import_module(module_name). A custom-object envelope whose module has not already been imported by the host process can no longer be deserialized.import_modulechanges a couple of error types raised from the loose-mode decode path:sys.modules→ValueError(previouslyImportError/ModuleNotFoundError).AttributeError(preserved from the oldgetattrbehavior; no change).from_json→TypeError(no change).AZURE_FUNCTIONS_DURABLE_STRICT_TYPINGnow influences this module. Hosts that happen to set it for unrelated reasons will get strict-mode behavior.Strict mode is opt-in. With
AZURE_FUNCTIONS_DURABLE_STRICT_TYPINGunset, behavior matches today's contract apart from items (1) and (2) above.Follow-up
ActivityTriggerConverter.decodecannot currently forward anexpected_typetodf_loadsbecause the worker'sInConverter.decodedispatch doesn't pass the function's parameter type annotation. Consequence: Custom objects are not supported at all in strict-type mode. Adding that plumbing inazure-functions-python-workerwould let strict-mode activity decode fully reconstruct custom objects viafrom_json. Flagged inline in the converter for the library/worker owners' input.