CLI: send screenshots from studio code to Telegram remote sessions

[gcsecsey]

Related issues

• Related to STU-1652
• Builds on the Telegram remote-session bridge from apps/cli: Telegram remote-session bridge for studio code (PoC) #3196
• Companion PR: 213611-ghe-Automattic/wpcom

How AI was used in this PR

Claude wrote the bulk of the implementation and the tests. I reviewed and tested the changes end-to-end in my sandbox using the companion PR.

Proposed Changes

The Telegram remote-session bridge is currently text-only. When the agent finishes a visible task, the user gets a prose summary but no image. This PR lets the local agent deliver screenshots inline:

• New share_screenshot tool. Captures a 16:9 above-the-fold view of a URL by default and emits a media.share JSON event. fullPage: true is opt-in for the rare case where the user wants the whole scroll length. take_screenshot stays unchanged as the model-internal reasoning tool.
• Remote-session controller (turn-runner + poll-loop) collects media.share events from the spawned studio code --json child and posts each photo before the text reply.
• respondMessage now picks transport based on payload. Photo present means multipart/form-data with a photo file part (matches the wpcom contract); text-only stays on the existing JSON path.
• Spawned child gets STUDIO_REMOTE_SESSION=1 so the system prompt knows to keep replies short, deliver visible work via share_screenshot, follow up with a "Want me to publish this as a preview site?" line, and stop fabricating "gist stored / preview link saved" epilogues that aren't backed by any actual storage.

Testing Instructions

Prerequisites

• Be an Automattician (backend gates on is_automattician())
• Be logged in via studio auth login so the bearer falls through from ~/.studio/shared.json
• Follow the testing steps on 213611-ghe-Automattic/wpcom to apply the backend changes on your sandbox

Testing

• Build the CLI: STUDIO_ENABLE_REMOTE_SESSION=true npm run cli:build
• Start the bridge: STUDIO_ENABLE_REMOTE_SESSION=true node apps/cli/dist/cli/main.mjs code --remote-session
• In a second terminal, tail the log: tail -F ~/.studio/remote-session.log
• From Telegram, send: "send to my local agent: take a screenshot of my site and show me"
• Check in Telegram:
  • A 1280x720 above-the-fold screenshot arrives inline
  • A follow-up text message asks about publishing a preview site
• Test for the text-only regressions by sending a non-visual request like "list my local sites"
• Ask explicitly for a full page screenshot and confirm share_screenshot is called with fullPage: true and the long capture is delivered

Pre-merge Checklist

• Have you checked for TypeScript, React or other console errors?

[Copilot]

Pull request overview

This PR extends the existing Telegram remote-session bridge for studio code to support inline screenshot delivery by introducing a new user-facing screenshot tool and a new JSON event type that the remote-session controller forwards as Telegram photos.

Changes:

• Add a new share_screenshot tool that emits a media.share JSON event (plus returns the image to the agent) for user-visible screenshot delivery.
• Extend the remote-session controller to collect media.share events and post photos before the text reply.
• Update Telegram response transport to use multipart/form-data when a photo is present (JSON for text-only), plus add remote-session-specific system prompt guidance toggled by STUDIO_REMOTE_SESSION=1.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tools/common/ai/tools.ts	Adds display name + URL detail extraction for the new share_screenshot tool.
tools/common/ai/json-events.ts	Introduces MediaShareEvent and extends JsonEvent union with media.share.
apps/cli/ai/tools.ts	Implements share_screenshot, refactors screenshot capture into captureScreenshotPng, and registers the new tool.
apps/cli/ai/system-prompt.ts	Adds Telegram remote-session guidance addendum (including share_screenshot usage expectations).
apps/cli/ai/agent.ts	Enables the remote-session system prompt addendum when STUDIO_REMOTE_SESSION=1.
apps/cli/remote-session/turn-runner.ts	Collects media.share events from the subprocess and returns them in TurnOutcome.
apps/cli/remote-session/poll-loop.ts	Posts collected media shares to Telegram before posting the text reply; avoids “no result” warning when media exists.
apps/cli/remote-session/telegram-client.ts	Updates respondMessage to support multipart photo uploads + caption; logs partial failures without throwing.
apps/cli/remote-session/tests/*	Adds/updates unit tests for media collection, ordering, and multipart photo transport behavior.
apps/cli/remote-session/tests/fixtures/mock-studio-code.mjs	Adds a media-share fixture scenario emitting media.share events.
apps/cli/ai/tests/system-prompt.test.ts	New tests verifying remote-session prompt addendum is included/excluded appropriately.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[wpmobilebot]

📊 Performance Test Results

Comparing 7a331d0 vs trunk

app-size

Metric	trunk	7a331d0	Diff	Change
App Size (Mac)	1511.23 MB	1511.29 MB	+0.06 MB	⚪ 0.0%

site-editor

Metric	trunk	7a331d0	Diff	Change
load	1478 ms	1775 ms	+297 ms	🔴 20.1%

site-startup

Metric	trunk	7a331d0	Diff	Change
siteCreation	8102 ms	8093 ms	9 ms	⚪ 0.0%
siteStartup	4953 ms	4960 ms	+7 ms	⚪ 0.0%

---

Results are median values from multiple test runs.

Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff)

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

apps/cli/remote-session/poll-loop.ts:190

• handleTurn() always awaits mediaStreamer.drain() before checking signal.aborted. This means a user-initiated detach can be delayed by in-flight photo uploads (and MediaStreamer doesn’t pass the abort signal into respondMessage, so those POSTs can’t be cancelled). Consider short-circuiting the drain when signal.aborted/timeout, or threading the abort signal into MediaStreamer so pending uploads are cancelled and detach can complete promptly.

	// Wait for any in-flight photos to finish posting so a text reply that
	// follows them lands in chat order, even if the photo POST is still
	// running when the turn ends.
	const mediaSummary = await mediaStreamer.drain();

	if ( outcome.sessionId && outcome.sessionId !== sessionId ) {
		await deps.writeSession( target.chatId, outcome.sessionId );
	}

	const duration = Date.now() - started;
	deps.logger.info( 'Turn finished', {
		chat_id: target.chatId,
		duration_ms: duration,
		status: outcome.status,
		exit_code: outcome.exitCode,
		chars_out: outcome.replyText?.length ?? 0,
		session_id: outcome.sessionId,
		aborted: signal.aborted,
		media_posted: mediaSummary.posted,
		media_failed: mediaSummary.failed,
	} );

	// Detach was requested mid-turn. Skip posting any reply — the detach flow
	// will announce "🔴 Local agent detached." on its own.
	if ( signal.aborted ) {
		return;

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[epeicher]

Thanks for this @gcsecsey! I have tested it, and it works great 🙌 I have also added a commit to gate the new tool mcp__studio__share_screenshot behind the existing feature flag so it is not exposed until we release the changes.

Feature flag enabled	Feature flag disabled

And I want to share here my testing, it works like a charm!