feat(media): add audio and file display - #1405
Merged
Merged
Conversation
Share display_image prompt across MCP-bridge flavors (Cursor, Gemini, Kimi, Codex, Claude, OpenCode), auto-approve the tool in buildHapiMcpBridge, handle ACP image content blocks, and harden generated-image registration with content sniffing. Closes tiann#956 Co-authored-by: Cursor <cursoragent@cursor.com>
Keep object URLs stable across refetch, upscale tiny inline images, fetch generated-image bytes with cache no-store (avoid empty 304 bodies), and load hapiMcpUrl from per-session API in hapi-display-image tooling. Co-authored-by: Cursor <cursoragent@cursor.com>
Add display_video alongside display_image, video MIME sniffing with avif guard, web GeneratedImageCard video player, and hapi-display-image auto-routing. Co-authored-by: Cursor <cursoragent@cursor.com>
Share display_video prompts across MCP-bridge flavors, auto-approve the tool, register mp4/webm via path sniffing, render inline video in web on the existing generated-image RPC path, and restore robust media card fetch. Co-authored-by: Cursor <cursoragent@cursor.com>
Flush buffered assistant text before async generated_image emit from ACP image blocks (PR tiann#958 review Major). Add optional source metadata on generated-image wire messages (ingress, flavor, toolCallId, toolName) for MCP, ACP, and Codex tool-result paths. Seeds artifact-event follow-up tiann#966. Co-authored-by: Cursor <cursoragent@cursor.com>
…tale blobs Queue ACP session updates and await async image registration before later events; clear GeneratedImageCard blob state when imageId changes. Co-authored-by: Cursor <cursoragent@cursor.com>
Straggler session/update during drainLateBuffers can queue async image registration; re-await sessionUpdateQueue so generated_image is not emitted after turn_complete. Co-authored-by: Cursor <cursoragent@cursor.com>
Match server-side detectImageMimeType so .avif files are not sent to display_video and rejected as unsupported video. Co-authored-by: Cursor <cursoragent@cursor.com>
- hapi doctor inline-media: probe bridges, print per-session inline commands - Expose hapiMcpUrl on session list summaries (stops false "no MCP" scans) - Helper script: match cursorSessionId prefixes; HAPI_SESSION_ID path-only mode - ACP bridge prompt: shell fallback + HAPI session id vs agent id rule Co-authored-by: Cursor <cursoragent@cursor.com>
Drop cache: no-store on generated-image fetch so browser can reuse hub immutable responses; on 304 re-read via force-cache (tiann#927, PR review). Co-authored-by: Cursor <cursoragent@cursor.com>
Cursor ACP ignores session/new mcpServers. Write .cursor/mcp.json and run agent mcp enable hapi instead. Remove HAPI_MCP_BRIDGE_PROMPT from user turns on ACP remotes; enrich MCP tool descriptions for discovery. Co-authored-by: Cursor <cursoragent@cursor.com>
Cleanup only removes or restores the hapi MCP entry instead of rewriting the full pre-session snapshot, so concurrent edits to other servers survive. Co-authored-by: Cursor <cursoragent@cursor.com>
Upstream Grok launcher exhaustiveness broke after AgentMessage gained generated_image for cross-flavor inline media. Co-authored-by: Cursor <cursoragent@cursor.com>
Gate display_video in the STDIO bridge, restore OpenCode first-prompt TITLE_INSTRUCTION (skill_lookup), and update tool-list test expectations. Co-authored-by: Cursor <cursoragent@cursor.com>
Only undo mcpServers.hapi when it still matches the exact entry this session installed; concurrent Cursor/user edits of that key survive. Co-authored-by: Cursor <cursoragent@cursor.com>
Move media tools off substring name/id hints onto the exact-name set so forged lookalike tools are not approved in default permission mode. Co-authored-by: Cursor <cursoragent@cursor.com>
…tions Cursor must not get a user-turn media prepend (prompt-taint). Remove unused HAPI_MCP_BRIDGE_PROMPT_CURSOR and embed DISPLAY_*_PROMPT_CURSOR in the display_image/display_video MCP tool descriptions instead. Co-authored-by: Cursor <cursoragent@cursor.com>
Those tools read arbitrary local paths into chat; keep them on MCP approval_mode prompt and out of default-mode auto-approve exact names. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Passive ACP agentMessageChunk handling must not load file:// or bare paths; local media goes through prompt-gated display_image/display_video. Co-authored-by: Cursor <cursoragent@cursor.com>
Reject HEIC/HEIF and other non-video ISO-BMFF containers instead of treating every non-AVIF ftyp as video/mp4. Co-authored-by: Cursor <cursoragent@cursor.com>
Wrap installCursorMcpOverlay so a malformed project .cursor/mcp.json cannot abort the session; continue without inline media tools. Co-authored-by: Cursor <cursoragent@cursor.com>
Exit non-zero whenever required checks fail, even if an active hapiMcpUrl bridge is present. Co-authored-by: Cursor <cursoragent@cursor.com>
Native ACP title mode still exposes display_image and display_video; update startHappyServer test after rebase onto 0.23.4. Co-authored-by: Cursor <cursoragent@cursor.com>
session_info_update title forwarding (tiann#1028) must not wait on the async message-handler queue used for inline media ordering. Co-authored-by: Cursor <cursoragent@cursor.com>
Read 16 bytes for detectMediaTool instead of the whole file, and include hapi_display_video in OPENCODE_NATIVE_TOOL_INSTRUCTION for remote ACP. Co-authored-by: Cursor <cursoragent@cursor.com>
cursor/generate_image only emitted a tool card; register filePath or base64 imageData into generatedImages and emit generated_image so the web chat card renders (issue tiann#956 / swear01 report). Co-authored-by: Cursor <cursoragent@cursor.com>
Path-only filePath registration bypassed permission-gated display_image / display_video MCP tools. Keep base64 imageData only; local paths must go through MCP approval. Co-authored-by: Cursor <cursoragent@cursor.com>
Reject oversized ACP/Cursor base64 payloads by character count so the CLI never allocates past the 25 MB generated-image cap. Co-authored-by: Cursor <cursoragent@cursor.com>
Bare EBML magic matches Matroska/MKV too; only accept DocType webm. Also restore annotated Playwright cursor in annotatedVideoUseOption. Co-authored-by: Cursor <cursoragent@cursor.com>
Close two Major review findings on tiann#958: suppress queue leak after restore, and Claude --allowedTools auto-approving local-path video. Co-authored-by: Cursor <cursoragent@cursor.com>
Preserve user Cursor MCP symlinks on atomic overlay writes, and require explicit Load video before fetching large generated-video blobs. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Reject ambiguous prefix matches so images/videos cannot land in the wrong HAPI chat when multiple agent session ids share a prefix. Co-authored-by: Cursor <cursoragent@cursor.com>
Run overlay cleanup in finally so cancelAll/disconnect failures cannot leave a dead hapi-<sessionId> entry in .cursor/mcp.json. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep hapiMcpUrl in toSessionSummaryMetadata; retain both ACP suppress queue-leak and compact-escape regression tests. Co-authored-by: Cursor <cursoragent@cursor.com>
Unblock typecheck after Antigravity/Copilot merge, and fail closed when .cursor/mcp.json or .cursor is a project-controlled symlink. Co-authored-by: Cursor <cursoragent@cursor.com>
Unblock web typecheck after steer merge, and recover orphaned hapi-* mcp.json entries via HAPI_MCP_OVERLAY_PID ownership stamps. Co-authored-by: Cursor <cursoragent@cursor.com>
Token-matched unlock so a crash mid-lock no longer permanently disables inline media; keep live-owner waits identity-safe. Co-authored-by: Cursor <cursoragent@cursor.com>
Acquire an exclusive recovery lock before token-matched unlink so two recoverers cannot remove a successor's live mcp.json lock. Co-authored-by: Cursor <cursoragent@cursor.com>
Withdraw racy auto-recovery: pathname check-then-unlink/rename can steal a successor lock. Stale locks throw with an explicit rm hint instead. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep cross-flavor display_image/display_video prompt constants; adopt upstream buildSessionCitationSteerInstruction for peer-citation steering. Co-authored-by: Cursor <cursoragent@cursor.com>
Merge left both steer and queue; keep queue so retries after abort do not re-bind to a later Pi turn. Co-authored-by: Cursor <cursoragent@cursor.com>
Close TOCTOU between pathname size check and readFile for display_image / display_video and registerGeneratedImageFromPath. Also preserve non-PID env edits on Cursor MCP overlay cleanup. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep display_video + bounded media reads; adopt upstream list_peers MCP tool, auto-approve wiring, and peer-list helpers. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Keep display_image/display_video prompt constants; adopt upstream session-summary instruction wrappers (getSystemPrompt / getCodexSystemPrompt). Co-authored-by: Cursor <cursoragent@cursor.com>
Keep ephemeral hapi-<sessionId> bridges out of the checked-out tree so agents cannot git-add a live loopback URL. Tests inject mcpConfigDir. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Findings
- None.
Summary
Review mode: initial
No issues found at ≥80% confidence in the latest full PR diff. Residual risk: real-browser playback/download behavior across supported audio and ISO-BMFF variants was not executed locally; the repository test workflow was still pending at review time.
Testing
- Not run (automation). Static review only; GitHub
testcheck pending.
HAPI Bot
heavygee
added a commit
to heavygee/hapi
that referenced
this pull request
Aug 8, 2026
Resolve hapi-display-image.mjs: take upstream tiann#1405 display_media detection; keep fork hapi-hub-auth note. Co-authored-by: Cursor <cursoragent@cursor.com>
heavygee
added a commit
to heavygee/hapi
that referenced
this pull request
Aug 8, 2026
Upstream feat(media) (tiann#1405) landed display_video/display_media and the Cursor ~/.cursor MCP overlay path. Conflict resolutions take upstream; tree now matches upstream/main — tiann#958 is superseded. Co-authored-by: Cursor <cursoragent@cursor.com>
4 tasks
heavygee
added a commit
to heavygee/hapi
that referenced
this pull request
Aug 8, 2026
tiann#958→tiann#1405: closed empty-vs-main should retarget chip to absorber; soften merged statusAction when layer/worktree already clean; document first-class superseded close in lifecycle + peer retro. Co-authored-by: Cursor <cursoragent@cursor.com>
heavygee
added a commit
to heavygee/hapi
that referenced
this pull request
Aug 8, 2026
…#1405) Tracked config still had an active branch after peers only edited ~/.config; remat failed on missing ref. Canon path is config/. Co-authored-by: Cursor <cursoragent@cursor.com>
heavygee
added a commit
to heavygee/hapi
that referenced
this pull request
Aug 8, 2026
Heal 61 + upstream tiann#1405 left duplicate object keys in the unit test; tsc TS1117 failed remat typecheck and rolled back the wave rebuild. Co-authored-by: Cursor <cursoragent@cursor.com>
Collaborator
|
@NightWatcher314 🤜 💥 🤛 - nicely done! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a stacked follow-up to #958. Please review the final follow-up commit (
f57783e7) for the changes introduced here; the earlier commits are inherited from #958.display_mediaMCP tool while keepingdisplay_imageanddisplay_videocompatibilityContent-Disposition: attachmentand all generated media withX-Content-Type-Options: nosniffTest plan
bun typecheckbun run test— CLI 216 files / 2284 passed / 1 skipped; Hub, Web, and Shared passedbun run buildgit diff --checkFindings: NoneNotes