Skip to content

.Net: Bug?: Structured Output with DateTimes/DateTimeOffset #10507

@rwjdk

Description

@rwjdk

Describe the bug
Not sure if this should be considered a "bug" or a "feature", but if you use a class with a DateTime Property for the "typeof()" ResponseFormat (Structured Output), the JSON back for that property will not automatically be possible to deserialize into the input object-type since the date is not returned in the invariant DateTime format (is often returned by LLM in full: eg: 12 February 2025, where C# Deserialization needs it structured by default "yyyy-MM-dd HH:mm:ss").

To Reproduce
Run the following code:

var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.AddAzureOpenAIChatCompletion(chatModel, azureOpenAiEndpoint, azureOpenAiKey);
var kernel = kernelBuilder.Build();

var agent = new ChatCompletionAgent
{
    Kernel = kernel,
    Arguments = new KernelArguments(new AzureOpenAIPromptExecutionSettings
    {
        ResponseFormat = typeof(SuperBowlFacts),
    })
};

var chatHistory = new ChatHistory();
chatHistory.AddUserMessage("List the Superbowls for 2020, 2021 and 2022");
await foreach (var response in agent.InvokeAsync(chatHistory))
{
    string json = response.Content!; //<<<<<<<<<<<<<<<<<<<<<<<<<<<Content come back fine but with the date format the LLM choose
    var superBowlFacts = JsonSerializer.Deserialize<SuperBowlFacts>(json); //<<<<<<<< That means the deserialization fails here
}

public class SuperBowlFacts
{
    public SuperBowlFact[] Facts { get; set; }
}

public class SuperBowlFact
{
    public int Year { get; set; }
    //[Description("Format of date should be Invariant C# Date Format")] //<<<<<< This line of instructions can "fix" it
    public DateTime DateTheGameWasPlayed { get; set; }
    public string Name { get; set; }
    public string WhoWon { get; set; }
}

Expected behavior
I'm unsure if the ResponseFormatBuilder should include custom additional instructions on behalf of the developer to ensure the correct format back, or if this is too "dangerous to do" and the "headache" need to be on the developer needing custom instructions or custom JSON deserialization options

Platform

  • Language: C#
  • Source: 1.36.1
  • AI model: Any (GPT-4o-mini)
  • IDE: Visual Studio
  • OS: Windows

Additional context
As mentioned it is an open question if this should be considered a bug or not; just wanted to raise the observation... If seen as something you wish to support DateTimeOffset is properly also required

Thank for an awesome Package SK Team 🙌

Metadata

Metadata

Assignees

Labels

.NETIssue or Pull requests regarding .NET codebugSomething isn't workingstaleIssue is stale because it has been open for a while and has no activity

Type

Projects

Status

Sprint: Planned

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions