Skip to content

Fix HostedFileContent with image MIME type sent as input_file instead of input_image#7438

Merged
jozkee merged 2 commits intomainfrom
copilot/fix-hosted-file-mime-type-issue
Mar 31, 2026
Merged

Fix HostedFileContent with image MIME type sent as input_file instead of input_image#7438
jozkee merged 2 commits intomainfrom
copilot/fix-hosted-file-mime-type-issue

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 30, 2026

HostedFileContent with image media types (e.g. image/png) was unconditionally mapped to CreateInputFilePart, causing OpenAI to reject it with: "Expected a file with an application/pdf MIME type, but got unsupported MIME type 'image/png'".

The tool output path already handled this correctly via HasTopLevelMediaType("image") — the user message path did not.

Changes

  • OpenAIResponsesChatClient.cs: Added a HostedFileContent case guarded by HasTopLevelMediaType("image") before the generic fallback, routing image files through CreateInputImagePart (with detail level support)
  • OpenAIResponseClientTests.cs: Extended UserMessageWithVariousContentTypes_ConvertsCorrectly to cover hosted image files with and without detail level
// Before: always input_file
case HostedFileContent fileContent:
    (parts ??= []).Add(ResponseContentPart.CreateInputFilePart(fileContent.FileId));
    break;

// After: image-aware routing
case HostedFileContent fileContent when fileContent.HasTopLevelMediaType("image"):
    (parts ??= []).Add(ResponseContentPart.CreateInputImagePart(fileContent.FileId, GetImageDetail(item)));
    break;

case HostedFileContent fileContent:
    (parts ??= []).Add(ResponseContentPart.CreateInputFilePart(fileContent.FileId));
    break;

…t instead of CreateInputFilePart

Agent-Logs-Url: https://github.com/dotnet/extensions/sessions/42e9240d-dc02-474b-9af4-373db24ca35b

Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix wrong assumption of hosted file MIME type Fix HostedFileContent with image MIME type sent as input_file instead of input_image Mar 30, 2026
Copilot AI requested a review from stephentoub March 30, 2026 11:15
@stephentoub stephentoub marked this pull request as ready for review March 30, 2026 11:35
@stephentoub stephentoub requested a review from a team as a code owner March 30, 2026 11:35
Copilot AI review requested due to automatic review settings March 30, 2026 11:35
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes OpenAI Responses “user message” content conversion so HostedFileContent with an image MIME type (e.g., image/png, image/jpeg) is sent as input_image (optionally with detail) instead of incorrectly being sent as input_file (which can be rejected by OpenAI).

Changes:

  • Route HostedFileContent with top-level media type image/* through ResponseContentPart.CreateInputImagePart(...) in OpenAIResponsesChatClient.
  • Extend UserMessageWithVariousContentTypes_ConvertsCorrectly to validate hosted image files are serialized as input_image (with and without detail).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs Adds an image-aware HostedFileContent switch case so hosted image files map to input_image rather than input_file.
test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/OpenAIResponseClientTests.cs Updates the verbatim request JSON + content list to assert hosted image files are serialized as input_image (including detail).

@stephentoub stephentoub added area-ai Microsoft.Extensions.AI libraries and removed area-telemetry labels Mar 30, 2026
@stephentoub stephentoub requested a review from jozkee March 31, 2026 14:46
@jozkee jozkee merged commit 4e8e9a2 into main Mar 31, 2026
10 checks passed
@jozkee jozkee deleted the copilot/fix-hosted-file-mime-type-issue branch March 31, 2026 17:15
jozkee pushed a commit to jozkee/extensions that referenced this pull request Apr 3, 2026
… of input_image (dotnet#7438)

* Initial plan

* Fix HostedFileContent with image MIME type to use CreateInputImagePart instead of CreateInputFilePart

Agent-Logs-Url: https://github.com/dotnet/extensions/sessions/42e9240d-dc02-474b-9af4-373db24ca35b

Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-ai Microsoft.Extensions.AI libraries

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wrong assumption when working with hosted files (HostedFileContent - MIME type)

5 participants