Skip to content

Add ImageGenerationToolCallContent and ImageGenerationToolResultContent to JSON serialization infrastructure#7275

Merged
stephentoub merged 2 commits intomainfrom
copilot/add-image-generation-tool-content
Feb 9, 2026
Merged

Add ImageGenerationToolCallContent and ImageGenerationToolResultContent to JSON serialization infrastructure#7275
stephentoub merged 2 commits intomainfrom
copilot/add-image-generation-tool-content

Conversation

Copy link
Contributor

Copilot AI commented Feb 8, 2026

ImageGenerationToolCallContent and ImageGenerationToolResultContent were missing from the JSON serialization infrastructure, preventing polymorphic serialization through AIContent references and breaking NativeAOT scenarios.

Changes

  • Added type registrations in AIJsonUtilities.Defaults.cs:

    • Called AddAIContentType() for both types with discriminator IDs "imageGenerationToolCall" and "imageGenerationToolResult"
    • Added [JsonSerializable] attributes to JsonContext for source generation support
  • Added comprehensive tests:

    • ImageGenerationToolCallContentTests.cs - validates properties and polymorphic serialization
    • ImageGenerationToolResultContentTests.cs - validates properties including nested Outputs list and polymorphic serialization
    • Updated AIContentTests.Serialization_DerivedTypes_Roundtrips to include both types

Example

// Now works correctly - type discriminator ensures proper deserialization
AIContent content = new ImageGenerationToolResultContent 
{ 
    ImageId = "img123",
    Outputs = [new DataContent(imageBytes, "image/png")]
};

string json = JsonSerializer.Serialize(content, AIJsonUtilities.DefaultOptions);
// Emits: {"$type":"imageGenerationToolResult","ImageId":"img123",...}

AIContent deserialized = JsonSerializer.Deserialize<AIContent>(json, AIJsonUtilities.DefaultOptions);
// Correctly reconstructs as ImageGenerationToolResultContent

Follows the same pattern as other experimental content types (CodeInterpreterToolCallContent, McpServerToolCallContent, etc.).

Original prompt

Problem

ImageGenerationToolCallContent and ImageGenerationToolResultContent are missing from AIJsonUtilities.Defaults.cs. Both types are [Experimental] AIContent-derived types (just like CodeInterpreterToolCallContent, McpServerToolCallContent, etc.) but were not added to the JSON serialization infrastructure. This means:

  1. No polymorphic type discriminator is emitted when serializing these types as AIContent, so deserialization cannot reconstruct the correct derived type.
  2. No source-generated type info exists for NativeAOT scenarios (where JsonSerializer.IsReflectionEnabledByDefault is false), so serialization will fail entirely.

Required Changes

1. src/Libraries/Microsoft.Extensions.AI.Abstractions/Utilities/AIJsonUtilities.Defaults.cs

In CreateDefaultOptions() method (around line 61, after the CodeInterpreterToolResultContent line), add:

AddAIContentType(options, typeof(ImageGenerationToolCallContent), typeDiscriminatorId: "imageGenerationToolCall", checkBuiltIn: false);
AddAIContentType(options, typeof(ImageGenerationToolResultContent), typeDiscriminatorId: "imageGenerationToolResult", checkBuiltIn: false);

In the JsonContext source-generation attributes (around line 136, after CodeInterpreterToolResultContent), add:

[JsonSerializable(typeof(ImageGenerationToolCallContent))]
[JsonSerializable(typeof(ImageGenerationToolResultContent))]

These should be placed near the existing // IImageGenerator section or alongside the other experimental content types — follow the existing pattern.

2. Add tests to validate polymorphic serialization round-tripping

There is an existing test file for AIJsonUtilities serialization tests. Add tests that validate ImageGenerationToolCallContent and ImageGenerationToolResultContent can be serialized and deserialized polymorphically through AIContent references, similar to how other experimental content types are tested.

Look at how other AIContent-derived types (especially experimental ones like CodeInterpreterToolCallContent, CodeInterpreterToolResultContent, McpServerToolCallContent, etc.) are tested for polymorphic JSON round-tripping in the test suite, and follow the same pattern for the two image generation content types.

For ImageGenerationToolCallContent, ensure the ImageId property round-trips.
For ImageGenerationToolResultContent, ensure both the ImageId and Outputs properties round-trip (including nested AIContent items in the Outputs list).

Context

  • ImageGenerationToolCallContent is defined in src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/ImageGenerationToolCallContent.cs
  • ImageGenerationToolResultContent is defined in src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/ImageGenerationToolResultContent.cs
  • Both are marked [Experimental(DiagnosticIds.Experiments.AIImageGeneration)]
  • The fix follows the exact same pattern used for CodeInterpreterToolCallContent/CodeInterpreterToolResultContent and the MCP content types

This pull request was created from Copilot chat.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Microsoft Reviewers: Open in CodeFlow

…nt to JSON serialization infrastructure

Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
Copilot AI changed the title [WIP] Add ImageGenerationToolCallContent and ImageGenerationToolResultContent Add ImageGenerationToolCallContent and ImageGenerationToolResultContent to JSON serialization infrastructure Feb 8, 2026
Copilot AI requested a review from stephentoub February 8, 2026 04:31
@stephentoub stephentoub marked this pull request as ready for review February 8, 2026 04:33
@stephentoub stephentoub requested a review from a team as a code owner February 8, 2026 04:33
@stephentoub stephentoub enabled auto-merge (squash) February 8, 2026 04:33
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds missing JSON polymorphic/source-gen registrations for image generation tool content types so they can round-trip through AIContent and work in NativeAOT scenarios.

Changes:

  • Registered ImageGenerationToolCallContent / ImageGenerationToolResultContent in AIJsonUtilities.DefaultOptions polymorphic configuration with type discriminators.
  • Added [JsonSerializable] entries for both types to the source-generated JsonContext.
  • Added targeted unit tests and expanded the existing derived-type roundtrip test set.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
src/Libraries/Microsoft.Extensions.AI.Abstractions/Utilities/AIJsonUtilities.Defaults.cs Registers the two image generation tool content types for polymorphic serialization and source generation.
test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Contents/ImageGenerationToolCallContentTests.cs Verifies default/property behavior and polymorphic JSON roundtrip via AIContent.
test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Contents/ImageGenerationToolResultContentTests.cs Verifies Outputs behavior and both concrete + polymorphic JSON roundtrips.
test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Contents/AIContentTests.cs Extends derived-type serialization roundtrip coverage to include the new types.

@stephentoub stephentoub added the area-ai Microsoft.Extensions.AI libraries label Feb 8, 2026
Copy link
Member

@ericstj ericstj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for adding more tests to cover these.

@stephentoub stephentoub merged commit 66bfbac into main Feb 9, 2026
11 of 12 checks passed
@stephentoub stephentoub deleted the copilot/add-image-generation-tool-content branch February 9, 2026 23:01
joperezr pushed a commit that referenced this pull request Feb 10, 2026
…nt to JSON serialization infrastructure (#7275)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
joperezr added a commit that referenced this pull request Feb 11, 2026
* Merged PR 57712: Getting ready for 10.3 release

Getting ready for 10.3 release

----
#### AI description  (iteration 1)
#### PR Classification
This pull request implements release preparation changes for the 10.3 release.

#### PR Summary
The PR bumps dependency versions and adjusts configuration settings to ready the repository for the upcoming release while streamlining build pipeline tasks.
- **`eng/Version.Details.xml` & `eng/Versions.props`**: Update dependency versions from 9.0.12 to 9.0.13 (including net10 and LTS versions) and set release stabilization flags (`StabilizePackageVersion=true`, `DotNetFinalVersionKind=release`).
- **`azure-pipelines.yml`**: Remove the entire code coverage stage and its related dependency from the post-build steps.
- **`.github/skills/prepare-release/SKILL.md`**: Add documentation with instructions for preparing an internal release branch.
- **`Directory.Build.props` & `NuGet.config`**: Add NU1507 warning suppression and remove the package source mapping section to align with internal feed usage.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

* Fix OpenAI responses streaming to preserve encrypted reasoning content (#7266)

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update OpenAIResponsesChatClient to handle streaming code interpreter content (#7267)

* Update OpenAIResponsesChatClient to handle streaming code interpreter content

Right now it outputs it but in a bulk fashion only at the end of the response item. This makes it yield the deltas instead.

* Dedup code block

* Update OpenTelemetry semantic convention version references from 1.38 to 1.39 (#7274)

* Initial plan

* Update OpenTelemetry semantic convention version comments from 1.38 to 1.39

Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

* MEAI: Update Experimental / Preview Features (#7235)

* Remove [Experimental] attribute from IChatReducer

* Annotate APIs that use experimental OpenAI APIs. Remove prerelease label.

* Fix typo

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove project-wide OpenAI experimental suppressions. Finish annotating.

* Use granular constants for openai experimental diagnostics

* Update API baselines

* Remove unused const

* Remove redundant [Experimental] attributes for OpenAI Responses members

* Update ApiChief baselines for MEAI

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add ImageGenerationToolCallContent and ImageGenerationToolResultContent to JSON serialization infrastructure (#7275)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

* Mark all of MicrosoftExtensionsAIChatExtensions as experimental

---------

Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com>
Co-authored-by: Jeff Handley <Jeff.Handley@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-ai Microsoft.Extensions.AI libraries

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants