Here's repo code:

   public static async Task Main(string[] args)
   {
       var app = new TestApp(); // Create an instance of TestApp  

       var modelName = "Phi-4-mini-instruct-cuda-gpu";
       var manager = new FoundryManager();
       await manager.StartServiceAsync();
       var modelInfo = await manager.GetModelInfoAsync(modelName);
       Console.WriteLine($"Model: {modelInfo.Alias} ({modelInfo.ModelId}) {modelInfo.Version}");
       await manager.LoadModelAsync(modelName);

       var schema =
           $$"""
           {
             "type": "object",
             "additionalProperties": false,
             "required": [
               "Explanation",
               "Result"
             ],
             "properties": {
               "Explanation": {
                 "type": [
                   "string",
                   "null"
                 ],
                 "description": "Explain your reasoning"
               },
               "Result": {
                 "type": [
                   "array",
                   "null"
                 ],
                 "items": {
                   "type": "string"
                 },
                 "description": "The result"
               }
             }
           }
           """;
       var responseFormat = ChatResponseFormat.CreateJsonSchemaFormat("result", 
           jsonSchema: BinaryData.FromString(schema), 
           jsonSchemaFormatDescription: "An JSON object with two properties: Explanation and Result",
           jsonSchemaIsStrict: true);

       ApiKeyCredential key = new ApiKeyCredential(manager.ApiKey);
       OpenAIClient client = new OpenAIClient(key, new OpenAIClientOptions
       {
           Endpoint = manager.Endpoint
       });

       var chatClient = client.GetChatClient(modelInfo.ModelId);

       var result = chatClient.CompleteChat(["List five common male first names?"], new ChatCompletionOptions()
       {
           ResponseFormat = responseFormat 
       });
       Console.WriteLine(result.Value.Content[0].Text);
       Console.WriteLine("Press any key to exit...");
       Console.ReadKey(true);
   }

If you comment out the ResponseFormat = responseFormat line the local model returns text response correctly
If you change it to call OpenAI with following changes:

set modelName to "gpt-4o"
configure a real "OpenAI" key
comment out Endpoint = manager.Endpoint
The OpenAI response correctly returns a JSON object matching the schema.

This could be because either:
a. the phi-4 model doesn't support structured output OR
b. the schema is blowing up foundry.

Foundry returns 500 status code when using ResponseFormat (aka structured response) in ChatClient options #81

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions