Skip to content

Performance [P3]: Serialize history events in one pass #191

Description

@berndverst

What is the issue?

History event export deep-copies the dataclass graph and then recursively walks it again to convert values for JSON.

What is the impact?

Exporting large orchestration histories performs avoidable allocations and CPU work. This is limited to the history export feature but can be material for large archive jobs.

Details about the issue including code reference

Relevant code:

def to_dict(self) -> dict[str, Any]:
return _to_serializable(asdict(self))

def _to_serializable(value: Any) -> Any:
if isinstance(value, datetime):
return value.isoformat()
if isinstance(value, list):
return [_to_serializable(item) for item in cast(list[Any], value)]
if isinstance(value, dict):
return {
key: _to_serializable(item)
for key, item in cast(dict[Any, Any], value).items()
}
return value

def event_to_dict(event: history.HistoryEvent) -> dict[str, Any]:
"""Convert a :class:`history.HistoryEvent` into a JSON-safe dict.
A discriminator field ``event_type`` is added so downstream
consumers can distinguish event subclasses without inspecting
their fields.
"""
payload: dict[str, Any] = event.to_dict()
# Insert the discriminator first so the resulting dict orders it
# near the front of the JSON object even before any sorting.
return {"event_type": type(event).__name__, **payload}

HistoryEvent.to_dict() calls _to_serializable(asdict(self)). dataclasses.asdict() recursively deep-copies nested dataclasses, lists, and dictionaries; _to_serializable() then walks lists and dictionaries again. History export invokes this path through event_to_dict().

A potential or proposed solution

Implement a one-pass recursive serializer over dataclass fields, lists, dictionaries, and datetimes instead of calling asdict(). Preserve the existing output shape, datetime format, and handling of nested history dataclasses.

Metadata

Metadata

Assignees

No one assigned

    Labels

    performance / optimizationUsed for issues or PRs purely focused on performance and optimizations, not bugs.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions