ChatGPT Can't See Image Responses From MCP Tools

[hsaeed3]

Small issue, not sure if it classifies as a full bug, but when running tools through connected apps on ChatGPT, the client seems to be
omitting any image content within responses. For example in a response similar to:

"content": [
  {
    "type": "text",
    "text": "Here is the image you requested:"
  },
  {
    "type": "image",
    "data": base64Data,
    "mimeType": "image/png"
  }
],

When using tools such as MCP inspector, the response is retrieved normally and contains both the text content and image content, but within ChatGPT the image is omitted and only this response is visible in the client:

{"text":"Here is the image you requested:"}

Are there any workarounds for this, or is it something that hasn't been implemented yet?

[ainergiz]

I think the best workaround would be uploading image to S3 bucket and serving image url to client, similar to how official examples are passing image urls.

Cloudflare has an easy to use S3 compatible R2 storage with generous free tier. I'd be happy to show some code examples to achieve this

Another thing to consider is what kind of use case you're trying to serve. ChatGPT can and probably will call it's own image generation model when user asked for an image. Why do you think this app would improve the experience of the user inside ChatGPT?

[hsaeed3]

@ainergiz was more of a thought experiment rather an application, it was just in context of providing image input to the LLM, I realized a little after posting this issue that on the docs themselves OpenAI mentions the client takes in only text and markdown content from MCP tool responses, I was able to workaround by just injecting the image URL within the markdown content itself. I appreciate the suggestion though, thanks!