fix(media): preview host files and stream videos across clients - #251
Merged
Conversation
Pylon adopted #8803's useWorkspaceMutationRefresh hook and improved it with burst coalescing, but never plumbed workspaceMutationId into FileBrowserPanel. The preview pane and diff panel refresh on agent edits; the tree beside them did not, so a file the agent created stayed missing until a manual refresh. Completes that adoption. #9023 also depends on the prop existing.
Thread transfer impact✅ Thread transfer remains within every enforced ceiling.
Baseline: Scenario and decoded snapshot size10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.
Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adopts
#9023, plus the small piece of#8803Pylon had left unfinished.Images and videos anywhere the environment can read now preview inline across
web, desktop, and mobile, with byte-range streaming so a video plays without
downloading in full first. Agent "Read file" entries that name an image render a
thumbnail in the expanded work-log row.
On the access model
I previously flagged this one for a maintainer decision, on the grounds that any
orchestration:readclient can mint a signed URL for any host image or video.That was my mistake — the same commit adds 34 lines to
docs/internals/environment-auth.mdreasoning about exactly that and choosingthe environment-wide authorization model over per-project filesystem permissions,
which is the model Pylon already runs on. The URL is scoped to one canonical path
plus device/inode, expires in an hour, and re-validates identity on serve.
Dependency Pylon was missing
#9023expectsFileBrowserPanelto take aworkspaceMutationId, which comesfrom
#8803. Pylon had adopted the rest of#8803— the hook, its tests,ChatView,DiffPanel,FilePreviewPanel— and had even improved the hook withburst coalescing that upstream lacks, but never plumbed the prop into the tree.
So the preview pane refreshed on agent edits while the tree beside it did not.
First commit completes that; cherry-picking
#8803wholesale would have revertedPylon's coalescing.
Carve-outs
packages/shared/src/video.tskeeps Pylon's extension-firstvideoMimeTypeagain; upstream's rewrite of that file would have restored MIME-first ordering
and the
.ts→video/mp2tbug. Upstream's newVIDEO_FILE_EXTENSIONSexportis taken.
pylon-code:rather thant3code:, while takingupstream's new
http:/https:media sources that host video streaming needs.ChatMarkdown's menu keeps Pylon's lazygetOpenInEditorMenuLabel()getterrather than upstream's eager
openInEditorMenuLabelvalue, with the new"Preview media" entry added alongside.
ChatViewkeeps Pylon'ssessionInteractionhelpers andmergeFailedComposerSend;loadVideoPreviewUrlis dropped because#9023removes the function.
MessagesTimeline's work-log row is Pylon's, with upstream'sviewedImagederivation wired into it — upstream's version of that component reads a
context Pylon's row does not use.
docs/user/composer.mdarrived with three "T3 Code" strings; they say Pylon.ChatMarkdownWorkspaceImageis replaced by upstream'sChatMarkdownAssetImage.That is the feature rather than a regression: the successor handles media-file
resources, video, retry, and a failure fallback.
Pylon had lost
work-log/presentation.test.tsin an old rebase reconciliation andhad no coverage for that module; upstream's version restores it. One of its
assertions then failed, which surfaced a hunk that had silently not applied:
#9023loosens thedynamic_tool_callmatch from exact"Read File"to atrimmed case-insensitive compare, so Pylon would have kept missing read entries
whose title differs only in case.
Verification
@t3tools/web,@t3tools/mobile,t3,@t3tools/client-runtime,@t3tools/sharedtypecheck clean. Lint clean. 3398 web, 1177 mobile, 869client-runtime, 614 desktop, 56 server asset tests passing.
Pre-existing and unrelated:
src/terminal/ghostty/runtimeAbi.test.tsfails toparse a
.wasm?inlineimport locally onpylontoo.Model: Claude Opus 5 in Claude Code.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.