Skip to content

feat(tool): Add media attachments to read tool#22258

Open
soaringk wants to merge 1 commit intoanomalyco:devfrom
soaringk:video-read-media-attachments
Open

feat(tool): Add media attachments to read tool#22258
soaringk wants to merge 1 commit intoanomalyco:devfrom
soaringk:video-read-media-attachments

Conversation

@soaringk
Copy link
Copy Markdown

@soaringk soaringk commented Apr 13, 2026

Issue for this PR

Closes #22260

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

The read tool already returned images and PDFs as file attachments, but it rejected audio and video files as generic binary files. This adds audio/video MIME detection before binary rejection, returns them as file attachments, and adds an attachment size guard before loading media into memory.

The attachment guard defaults to 256 MB and can be changed with OPENCODE_READ_MAX_ATTACHMENT_BYTES. Media routing now checks the model's explicit input capabilities for image, PDF, audio, and video. Audio/video are not kept inside tool results for providers that only support image/PDF tool-result media; they are routed through synthetic user file parts when supported, or converted to explicit error text when unsupported.

This also adds OpenAI-compatible chat-completions conversion for video file parts. Supported video file parts now serialize as video_url content parts, while unknown file media types still throw UnsupportedFunctionalityError.

How did you verify your code works?

  • TMPDIR=/tmp/opencode-bun-tmp npm_config_cache=/tmp/opencode-npm-cache npx --yes bun@1.3.11 test test/provider/copilot/convert-to-copilot-messages.test.ts
  • TMPDIR=/tmp/opencode-bun-tmp npm_config_cache=/tmp/opencode-npm-cache npx --yes bun@1.3.11 test test/tool/read.test.ts
  • TMPDIR=/tmp/opencode-bun-tmp npm_config_cache=/tmp/opencode-npm-cache npx --yes bun@1.3.11 test test/session/message-v2.test.ts
  • TMPDIR=/tmp/opencode-bun-tmp npm_config_cache=/tmp/opencode-npm-cache npx --yes bun@1.3.11 test test/session/compaction.test.ts --test-name-pattern 'replays the prior user turn on overflow'
  • git diff --check

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actions github-actions bot added needs:title needs:compliance This means the issue will auto-close after 2 hours. labels Apr 13, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Hey! Your PR title Add media attachments to read tool doesn't follow conventional commit format.

Please update it to start with one of:

  • feat: or feat(scope): new feature
  • fix: or fix(scope): bug fix
  • docs: or docs(scope): documentation changes
  • chore: or chore(scope): maintenance tasks
  • refactor: or refactor(scope): code refactoring
  • test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

@github-actions
Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

Based on the search results, I found no duplicate PRs that are currently addressing the same feature.

The current PR (#22258) "Add media attachments to read tool" stands alone. While there are related PRs in the history (like #17204 about guarding text-only models from image attachments, and #21917 about omitting unsupported PDF tool-result attachments), these are separate concerns and not duplicates of the current PR.

@soaringk soaringk changed the title Add media attachments to read tool feat(tool): Add media attachments to read tool Apr 13, 2026
@github-actions github-actions bot removed needs:title needs:compliance This means the issue will auto-close after 2 hours. labels Apr 13, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Thanks for updating your PR! It now meets our contributing guidelines. 👍

@Demonese
Copy link
Copy Markdown

Demonese commented Apr 13, 2026

Hello, please consider adding a capability check to prevent the LLM from mistakenly assuming it has successfully read the content.

See:

Sorry, I just found out the check is already there in the message-v2.ts.

@soaringk soaringk force-pushed the video-read-media-attachments branch from f1a57b1 to 60b7db1 Compare April 13, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

read tool should support audio and video attachments

2 participants