.NET: [Feature]: Automatically fix malformed JSON responses from LLMs

[thnak]

Description

While working with LLMs for tool calling and structured outputs, the models occasionally return malformed or slightly corrupted JSON responses. Common issues include:

• Markdown Fences: Wrapping the JSON block inside markdown formatting (e.g., ```json ... ```).
• Trailing Commas: Leaving trailing commas at the end of objects or arrays (which standard System.Text.Json strictly rejects with a JsonException).
• Truncated Outputs: Cut-off responses due to token limits, resulting in missing closing brackets (}, ]).
• Stringified Fields: Embedding raw objects inside stringified attributes (e.g., "arguments": "{\"key\": \"value\"}" instead of "arguments": {"key": "value"}).

Implementing this at the gateway of our parser layer will drastically improve system resilience, cut down on redundant error handling, and make our integration with LLMs much more production-ready.

Code Sample

Language/SDK

.NET