v3.4.0 - CDP screencast: paint-driven Browser panel

Browser Panel: paint-driven streaming

V3.0.0 shipped a 2 fps polling-based viewer. V3.4.0 replaces that with native Chrome DevTools Protocol screencast — frames now flow at the actual paint rate of the page (10-30 fps during interaction, 0 when idle). The browser view goes from "watching a slideshow" to "watching the agent in real time."

Architecture pivot

Before (V3.0)	After (V3.4)
Bridge polls Playwright MCP every 500ms via browser_take_screenshot	Bridge subscribes to Page.startScreencast over CDP WebSocket — push-based
Each frame = full MCP round-trip (~250-500ms latency)	Each frame = CDP push (~100ms)
Flat 2 fps regardless of page activity	Native paint rate when active, 0 when static
Co-pilot clicks via browser_evaluate(elementFromPoint().click())	Real Input.dispatchMouseEvent with proper coords
Scrolling: not supported	Input.dispatchMouseEvent type=mouseWheel works natively
Playwright MCP spawns its own Chromium	We launch Chromium ourselves (scripts/start_chromium_cdp.bat); Playwright MCP attaches via --cdp-endpoint=http://127.0.0.1:9222. Agent and bridge share one browser.

Implementation notes

• sentinel-miniapp-v2/cdp_client.py — new CDP WebSocket client, ~250 lines
• Bridge starts both threads at boot:
  • browser-cdp — primary path, reconnects automatically when Chromium becomes available
  • browser-poll — fallback path, only fires when CDP frames are stale (last frame age > 3 sec)
• Chromium runs in --headless=new with persistent user-data-dir at %LOCALAPPDATA%\sentinel-chromium-profile so cookies/login state survive restarts
• --remote-allow-origins=* is required for the WebSocket handshake to succeed (loopback-only binding mitigates exposure risk)
• New POST /api/browser/scroll endpoint — accepts {x, y, deltaY}, dispatches via CDP

What stays the same

• SSE pipeline for the mini app side: canvas decode + ring buffer scrubbing + Hold/Resume
• Co-pilot click/type endpoints exist; primary path via Playwright MCP tools is preserved, CDP is the upgrade target for V3.5
• Polling fallback ensures the panel still works if you don't want to run Chromium externally

Setup change

Existing installs need to:

1. Run scripts/start_chromium_cdp.bat to launch Chromium with CDP exposed
2. Restart Playwright MCP (the launch scripts now use --cdp-endpoint)
3. Restart bridge

Or just run scripts\START_AI_STACK.bat which now sequences these properly.

Diagnostic scripts kept

• scripts/find_chromium_cdp.py — discovers active Chromium debug port
• scripts/probe_cdp_screencast.py — standalone CDP screencast smoke test
• scripts/probe_cdp_with_interaction.py — paint-rate measurement under input

Next: V3.5 will replace co-pilot click/type to use CDP Input.* directly (currently still going through Playwright MCP). Same architecture, same transport, drops one layer of latency.