-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Description
Is your feature request related to a problem? Please describe.
Yes. In several LLM-agent use cases — especially when using Agent with a list of ChatMessage objects as input — the deserialization process in _deserialize_content (from chat_message.py) produces error messages that are too vague to guide either developers or LLMs toward a correct fix.
For example, when the content field of a ChatMessage is malformed (e.g. a raw string instead of a dict with a required field like text), the system currently raises:
Unsupported part in serialized ChatMessage: `...`
This does not explain what structure is expected, leading to failed tool invocations and wasted cycles debugging and unnecessary token cost, even Agent outright failure .
Describe the solution you'd like
Improve the error raised in _deserialize_content to include explicit guidance on the expected schema. For instance:
Unsupported part in serialized ChatMessage: "...". A valid ChatMessage must contain a 'text', 'tool_call', or 'tool_call_result' in the 'content' field.
This will improve both human debugging and LLM-driven self-correction, which is increasingly important as LLMs generate their own structured inputs.
In addition make a list of all "LLM-contact points" as this one above, and revisit their error messages. If they need updates to make errors more clear and include guidance on expected input schema - so be it.
Describe alternatives you've considered
- Setting
tools_strict=True, which could catch more issues earlier but currently doesn’t work cleanly withChatMessage, as described in [issue #9411](feat: Better support fortools_strict=Truewhen using theOpenAIChatGenerator#9411) - Expanding documentation and schema definitions — helpful, but not a substitute for precise runtime errors in high-feedback scenarios like tool calling.
Additional context
This issue surfaced during tool invocation via ComponentTool, where models often produced malformed ChatMessage content. After applying a more descriptive error message manually, the LLM was able to self-correct — underscoring the need for clearer errors at every touchpoint between structured formats and model output.
More generally, this is a call to strengthen error reporting throughout the stack where LLMs interact with structured schemas — especially in agents, function calls, and tool wrapping — so that both humans and LLMs can recover more gracefully from mistakes.