Skip to content

fix(#3934): use MCP ImageContent for screenshots instead of embedding base64 in JSON#4255

Merged
laithrw merged 3 commits into
mainfrom
fix/mcp-screenshot-imagecontent
Mar 1, 2026
Merged

fix(#3934): use MCP ImageContent for screenshots instead of embedding base64 in JSON#4255
laithrw merged 3 commits into
mainfrom
fix/mcp-screenshot-imagecontent

Conversation

@laithrw
Copy link
Copy Markdown
Member

@laithrw laithrw commented Mar 1, 2026

Summary by cubic

Screenshots are now returned as MCP ImageContent instead of base64 in JSON. This reduces payload size and aligns outputs with MCP content types for browser_get_state and browser_screenshot.

  • Refactors

    • Updated handle_call_tool to return mixed content lists (TextContent + ImageContent) when tools produce images.
    • Changed _get_browser_state and _screenshot to return (metadata_json, screenshot_b64 | None); dimensions remain in metadata.
    • Removed embedded "screenshot" from JSON; kept "size_bytes" and "viewport".
    • Updated browser_screenshot tool description to reflect the new format.
    • Lint cleanup and synced with main; no functional changes.
  • Migration

    • Update clients to handle ImageContent in tool responses; parse TextContent for metadata and use ImageContent for the image.
    • Do not read "screenshot" from JSON; use "screenshot_dimensions" (browser_get_state) or "viewport" (browser_screenshot) for coordinates.
    • Expect handle_call_tool to return a list of content items for image-producing tools.

Written for commit a1e54ef. Summary will update on new commits.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 1, 2026

Agent Task Evaluation Results: 2/2 (100%)

View detailed results
Task Result Reason
amazon_laptop ✅ Pass Skipped - API key not available (fork PR or missing secret)
browser_use_pip ✅ Pass Skipped - API key not available (fork PR or missing secret)

Check the evaluate-tasks job for detailed task execution logs.

Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

@laithrw laithrw merged commit 71ad655 into main Mar 1, 2026
80 checks passed
@laithrw laithrw deleted the fix/mcp-screenshot-imagecontent branch March 1, 2026 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant