Browser Access MCP v2.0.0

v2.0.0 — MCP-powered browser automation stack for Notion Custom Agents

A hybrid open-source stack: Chromium MV3 extension + always-on local bridge exposing browser capabilities over MCP, with ngrok HTTPS tunneling for cloud agent access.

---

✨ Features

• 9 Master Tools — Observe, act, tabs, macros, teach, safety, skills, diff, done
• Persistent Automation Indicator — Badge, glowing border, and "Browser Access" tab group stay visible for the entire automation session (surviving navigations), not just per action; input blocking stays transient so you can still read the page while the agent thinks
• Session-Scoped Cleanup — The agent dismisses the indicator with browser-done when finished; fallbacks clear it on MCP session close, bridge disconnect, or 3 minutes of inactivity
• First-class Media Controls — fullscreen, play, pause, mute, unmute, PiP, media_state — no DOM button hunting
• Real Click Events — PointerEvent + MouseEvent dispatch with coordinates for proper user-activation
• Stale Snapshot Detection — Pass snapshotId to browser-act; warns when element IDs are from an outdated snapshot
• Tab Groups — Create, manage, and tear down Chrome tab groups for organized automation sessions
• Predicate Wait — wait_until polls a JS expression until truthy (e.g. !video.paused)
• execute_js Returns Values — Scripts auto-wrapped in return(); objects JSON-serialized; DOM nodes return outerHTML
• Smart Screenshot Pipeline — Auto downscale/recompress (JPEG q70, max 1280px), 3MB hard size guard, element_screenshot, screenshot_summary, resource_link mode, auto-tile
• Chrome Extension Bridge — MV3 extension communicates via WebSocket to local Node bridge
• Ngrok HTTPS Tunneling — Auto-tunnels for Notion Custom Agent access (with auto-reconnect)
• Teach Mode — Record intelligent skills with NL annotations and parameter substitution
• Macro Record & Replay — Blind action recording for quick automation
• Safety Layer — Domain allowlist, confirmation policy/gate, audit logging, kill switch
• Network Interception — URL pattern matching, blocking, header injection
• Cookie Management — Set, delete, and read cookies programmatically
• Incognito Support — Open incognito windows/tabs for private browsing sessions
• Multiple Snapshot Modes — Accessibility tree, semantic JSON, sanitized HTML, screenshot+text

---

🏗️ Architecture

┌─────────────────────┐     MCP (HTTPS/SSE)     ┌──────────────────┐
│  Notion Custom Agent │ ◄──────────────────────► │   Local Bridge   │
│  (or any MCP client) │     via ngrok tunnel     │   (Node.js)      │
└─────────────────────┘                          └────────┬─────────┘
                                                          │ WebSocket
                                                          ▼
                                                 ┌──────────────────┐
                                                 │  Chrome Extension │
                                                 │  (MV3)           │
                                                 └──────────────────┘

---

🚀 Getting Started

Prerequisites

• Node.js >= 18
• Google Chrome or Chromium
• ngrok account (free tier works) — dashboard.ngrok.com

1. Install dependencies

npm install

2. Configure

cp bridge/.env.example bridge/.env
# Edit bridge/.env with your NGROK_AUTHTOKEN, MCP_TOKEN, and NGROK_DOMAIN

3. Build

npm run build

4. Start the bridge

npm start

The bridge prints a secure connection URL + auth header — paste into your Notion Custom Agent's MCP settings.

5. Install the Chrome extension

1. Open chrome://extensions
2. Enable Developer mode
3. Click Load unpacked → select the extension/ directory
4. The extension auto-connects to the bridge via WebSocket

---

🛠️ Tool Surface (9 Tools)

🔭 browser-observe (read-only — 16 actions)

Action	Description
list_windows / list_tabs / get_active_tab	Tab and window discovery
snapshot_active_tab / snapshot_tab	Structured DOM/AX tree snapshots (preferred for verification)
screenshot	Viewport or full-page image capture (auto-compressed JPEG, 3MB guard)
element_screenshot	Scoped capture of a single element by ID or bounding box
screenshot_summary	Dimensions, file size, tiny thumbnail — decide before fetching full image
get_page_content	Clean text/markdown/HTML extraction
get_cookies / set_cookies / delete_cookies	Read, write, and delete browser cookies
get_local_storage	Read localStorage from any tab
get_console_logs	Captured browser console output
get_network_log	Recent XHR/fetch requests
intercept_requests / stop_intercept	Network interception with URL pattern matching, blocking, header injection

Screenshot params: quality (1-100), maxWidth (px), imageFormat (jpeg/png/webp), returnAs (inline/resource_link), autoTile (bool for full-page tiling).

✋ browser-act (write — 16 actions)

Action	Description
navigate / go_back / go_forward	Navigation (domain allowlist enforced)
click / double_click / hover / focus	Element interaction
type / select / fill_form	Input and form filling
scroll / key_press	Page control
drag_drop	Drag from source to target element
execute_js	Run arbitrary JavaScript
wait_for	Wait for element or text
file_upload	Upload local file to file input element

📑 browser-tabs (tab management — 9 actions)

Action	Description
new_tab / close_tab / switch_tab	Tab lifecycle
set_clipboard / get_clipboard	System clipboard
new_incognito_window / new_incognito_tab	Private browsing sessions
list_incognito_tabs / close_incognito_window	Manage incognito sessions

🎬 browser-macros (record & replay — 5 actions)

Action	Description
record_start / record_stop	Record user interactions
run / list / delete	Replay and manage macros

🧠 browser-teach (intelligent skills — 7 actions)

Action	Description
start / annotate_step / stop	Teach a reusable skill with NL annotations
run	Replay skill with parameter substitution + AI recovery
list / delete / export	Manage saved skills

🛡️ browser-safety (policy — 3 actions)

Action	Description
set_domain_allowlist	Restrict allowed domains (allowlist + denylist)
set_confirm_policy	Configure confirmation requirements
get_audit_log	View recent audit entries

⚡ browser-skills (built-in workflows — 10 skills)

One-call workflows that run entirely inside the extension, replacing 5-10 manual round-trips:

Skill	Description
youtube_play / google_search	Search + play / search + return top results in one call
fill_and_submit	Fill every named field in a form and submit
scroll_to_bottom / extract_table	Load dynamic content / parse a table to JSON
smart_observe	Fresh a11y snapshot + media state in one call (default observation primitive)
wait_for_idle / watch_dom	Wait for network idle / for a matching DOM mutation
click_reliable / type_reliable	CDP-level input for SPAs with synthetic event systems

🔀 browser-diff (change detection — 1 action)

Action	Description
diff	Returns only { added, changed, removed } since the last snapshot — far cheaper than a full re-snapshot

✅ browser-done (end automation session)

Action	Description
(call once, when finished)	Dismisses the in-browser "Automation in progress" indicator and releases the Browser Access tab group. Optional reason recorded in the audit log.

---

📸 Screenshot Pipeline (v0.5.0)

Screenshots are automatically processed through a smart pipeline to prevent payload size issues:

1. Downscale + Recompress — Default JPEG quality 70, capped at 1280px on the long edge. Configurable via quality, maxWidth, imageFormat params.
2. Hard Size Guard — If encoded result exceeds 3MB, progressively downscales until under limit. Adds truncated: true flag.
3. Element Screenshot — Crop to a single element by elementId from snapshot or manual bbox.
4. Screenshot Summary — Returns dimensions, file size, and a 256px thumbnail. Agent decides if full image is worth fetching.
5. Resource Link Mode — returnAs: 'resource_link' saves to temp dir and returns a fetch URI. Keeps step payload tiny.
6. Auto-Tile — autoTile: true for full-page captures splits into viewport-sized tiles returned as resource_links with a manifest.

Prefer snapshot_tab (a11y) over screenshot for verification — it's faster, smaller, and gives element IDs.

---

📸 Snapshot Modes

Mode	Description
a11y_markdown_ids	(preferred) Accessibility tree as Markdown with stable element IDs
semantic_json	Structured JSON with semantic content
sanitized_html	Cleaned HTML (fallback)
screenshot_plus_text	Screenshot + extracted text (vision fallback)

---

🔵 Automation Indicator (v2.0.0)

While an agent drives the browser, the user always sees what's happening:

1. Session visuals — On the first action, the tab gets a floating "Browser Access — Automation in progress" badge, a glowing border, and is placed in a blue Browser Access tab group. These persist for the entire task — across actions, agent thinking time, and page navigations.
2. Transient input lock — Mouse/keyboard on the tab are blocked only around each action (released 2s after), so the user can still scroll and read between steps.
3. Session end — The agent calls browser-done as its final step to dismiss everything. Fallbacks guarantee cleanup: last MCP session closed, bridge disconnected, tab closed, or 3 minutes without actions.

---

🛡️ Safety & Security

• Bearer token auth — Every MCP request requires MCP_TOKEN
• Kill switch — Create ~/.mcp_kill file to instantly block all tool calls (503)
• Confirmation gate — Set ENABLE_CONFIRMATION_GATE=true to require confirmation for moderate/dangerous actions
• Domain allowlist — Navigation restricted to approved domains
• Confirmation policy — Dangerous actions can require user approval
• Audit log — All actions logged with timestamps and outcomes
• Session TTL — MCP sessions auto-expire after 30 minutes

---

🔧 Configuration

Variable	Default	Description
PORT	9876	MCP HTTP server port
WS_PORT	9877	Chrome extension WebSocket port
NGROK_DOMAIN	—	Ngrok static domain (optional, for stable URLs)
NGROK_AUTHTOKEN	—	From dashboard.ngrok.com
MCP_TOKEN	—	Bearer token for authentication
ENABLE_ACT	true	Allow browser write actions
ENABLE_MACROS	true	Allow macro recording/playback
ENABLE_TEACH	true	Allow skill teaching
ENABLE_CONFIRMATION_GATE	false	Require confirmation for moderate/dangerous actions

---

📦 Project Structure

browser-access/
├── bridge/                     # Node.js MCP server
│   ├── .env                    # Active config (secrets, gitignored)
│   ├── .env.example            # Configuration template
│   ├── src/
│   │   ├── start.ts            # Ngrok boot orchestrator (with auto-reconnect)
│   │   ├── server.ts           # Composition root (re-exports public surface)
│   │   ├── http.ts             # Express transport (SSE + Streamable HTTP)
│   │   ├── mcp-server.ts       # browser-* tool registrations
│   │   ├── config.ts           # Env/config, auth, kill switch, policy, audit
│   │   ├── security.ts         # Pure, unit-tested security helpers
│   │   ├── extension-rpc/      # WebSocket server to the Chrome extension
│   │   └── screenshot-pipeline.ts  # Image processing (sharp: downscale, tile, summary)
│   └── package.json
├── extension/                  # Chromium MV3 extension
│   ├── manifest.json
│   ├── src/background/service-worker.ts  # WebSocket client, tab/window mgmt
│   └── src/content/            # DOM capture, action execution
├── shared/                     # Shared TypeScript types
│   └── src/protocol.ts         # WebSocket message protocol
├── agent-instructions.md       # Notion Custom Agent instructions
├── tests/e2e-test-prompt.md    # E2E test prompts for agent verification
├── package.json                # Workspace root (npm workspaces)
└── README.md

---

📋 Changelog

v2.0.0 — 2026-07-09

• Persistent automation indicator — Badge, glowing border, and "Browser Access" tab group now stay visible for the entire automation session instead of flickering per action (previously torn down 2s after each action). Visuals are re-injected after every page navigation.
• Transient input lock — The click/keyboard blocker is decoupled from the session visuals: it engages only around actions and releases 2s after, so users can scroll and read the tab while the agent thinks.
• New browser-done tool — Agents call it once as their final step to dismiss the indicator and release the tab group (optional reason recorded in the audit log).
• Automatic cleanup fallbacks — The indicator also clears when the client's last MCP session closes, when the bridge disconnects, when the tab closes, or after 3 minutes of inactivity.
• Protocol — New END_AUTOMATION / AUTOMATION_ENDED messages between bridge and extension.
• Docs — README tool surface now covers browser-skills, browser-diff, and browser-done; agent instructions updated to require a final browser-done call.

v1.0.0 — 2026-05-03

• Real click events — click now dispatches PointerEvent + MouseEvent with coordinates (fixes requestFullscreen, clipboard, autoplay-with-sound failing due to missing user-activation)
• execute_js returns values — Expressions auto-wrapped in return(); objects JSON-serialized; DOM nodes return outerHTML snippet; undefined explicitly handled
• First-class media actions — fullscreen, exit_fullscreen, media_play, media_pause, media_mute, media_unmute, enter_pip, exit_pip, media_state — operate directly on <video> element, no DOM button hunting
• mediaState in snapshots — Snapshots now include mediaState when a <video> is detected (paused, muted, volume, currentTime, duration, fullscreen, pip, ytMiniplayer)
• Predicate-based wait_until — Pass a JS expression (e.g. !document.querySelector("video").paused) and it polls until truthy or timeout
• Stale snapshot detection — Pass snapshotId to browser-act; server warns when it doesn't match the last snapshot for that tab
• Tab groups — create_tab_group, add_to_group, ungroup_tabs, list_tab_groups, update_tab_group
• SPA stale-element rules — Agent instructions updated with mandatory re-snapshot-after-click rules, YouTube-specific patterns, miniplayer detection

v0.5.0 — 2026-04-27

• Screenshot pipeline overhaul — Auto downscale/recompress (JPEG q70, 1280px max), 3MB hard size guard, progressive fallback
• New actions — element_screenshot (scoped capture by element ID/bbox), screenshot_summary (dimensions + thumbnail)
• Resource link mode — returnAs: 'resource_link' saves to temp dir, returns fetch URI. Express /screenshots/:id endpoint.
• Auto-tile — autoTile: true splits full-page into viewport-sized tiles as resource_links with manifest
• Removed browser-youtube tool — YouTube automation now uses general browser-act + browser-observe tools
• Bug fixes — Missing os import, args not in scope for cookie/intercept handlers, file_upload case in browser-act switch
• Updated tool descriptions — A11y/snapshot biased as default verification primitive over screenshot

v0.4.1 — 2026-04-25

• Kill switch (~/.mcp_kill), confirmation gate, cookie management, network interception
• File upload action, incognito tab support, E2E test prompts
• Pinned MCP SDK to ^1.12.1, ngrok auto-reconnect with backoff

v0.3.0 — 2026-04-19

• Initial scaffolding: bridge + extension + shared workspaces
• 7 tool groups: observe, act, macros, safety, teach, tabs, YouTube
• Safety layer: domain allowlist, confirm policy, audit log

---

📄 License

MIT