CDP Switches Cursor Agents in 10ms: Multi-Agent Patrol via Chrome DevTools Protocol

[joinwell52-AI]

Why CDP for Cursor Agent Patrol?

We built CodeFlow to orchestrate multi-role AI teams (PM/DEV/QA/OPS) in Cursor IDE. The biggest challenge? Monitoring and switching between agents reliably.

Our first approach was OCR — screenshot the screen, run image recognition, guess which agent is active. It worked, but poorly:

Dimension	OCR	CDP
Accuracy	~90% (affected by font, DPI, theme)	100% (reads DOM attributes directly)
Latency	300-800ms per scan	10-15ms per scan
Active detection	Color/brightness heuristics	aria-selected=true exact match
Busy detection	Guessing spinner characters from pixels	Stop button visibility check
Click method	pyautogui.click(x, y) screen coords	Input.dispatchMouseEvent window coords

So we rebuilt everything on Chrome DevTools Protocol.

How It Works

Cursor runs on Electron, which means it supports --remote-debugging-port=9222. Once enabled, we get full DOM access.

Two-Layer Agent Scanning

Layer 1 — Tab bar: document.querySelectorAll('div[role=tab]')

• Regex extracts role names: /^\s*(\d{1,2})[\s\-\.]+([A-Za-z][A-Za-z0-9\-]+)\s*$/
• Normalization: 01-PM, 02-DEV, 03-QA, 04-OPS
• Active state: aria-selected === true

Layer 2 — Agent sidebar: document.querySelectorAll('span.agent-sidebar-cell-text')

• Catches overflow roles not visible in the tab bar
• Deduplication via Set

Agent Switching with Native Mouse Events

Every switch uses real-time coordinates (never cached — tabs move with window resize, sidebar toggle, DPI):

1. Query tab bar first, then sidebar — getBoundingClientRect() center point
2. Input.dispatchMouseEvent: mousePressed + mouseReleased (not el.click() — Electron/React may swallow synthetic events)
3. Post-click verification: scan() after 500ms confirms agentRole matches target

3-Layer Busy Detection

Before sending any message, we check if the target agent is working:

1. Stop button: button[aria-label=Cancel], button[aria-label=Stop] — visible = busy
2. Spinner animation: [class*=animate-spin] within Composer area only (excludes global loading)
3. Status text: Elements matching generating, thinking, planning, running terminal, applying patch, searching

If busy: defer nudge, put back in queue, retry next patrol cycle.

Message Sending (Bypassing React Controlled Components)

1. Find textarea -> nativeInputValueSetter to bypass React controlled component -> trigger input event
2. Fallback: Input.insertText (CDP native text insertion)
3. Send: Input.dispatchKeyEvent (keyDown + keyUp, key=Enter)

Graceful Degradation

Every CDP step auto-falls back to OCR:

Any CDP step fails |-- Port unreachable -> entire cycle uses OCR |-- Connection dropped -> this cycle uses OCR, auto-reconnect next |-- Role not found -> this role uses OCR |-- Click verify failed -> this role retries with OCR |-- Send failed -> send step uses OCR OCR also fails? -> Retry queue, next patrol cycle -> Dashboard shows failure status

Full Technical Documentation

Read the complete 400+ line deep-dive: CDP Multi-Agent Documentation

Source code: cursor_cdp.py

---

Questions, feedback, or ideas for improvement? Drop a comment below!

[joinwell52-AI]

📄 Bilingual Documentation / 中英文双语文档

The full CDP technical documentation is now available in both languages:

🇺🇸 English: CDP Multi-Agent Differentiation Mechanism

🇨🇳 中文: CDP 多 Agent 区分机制

Both documents cover the complete CDP architecture: two-layer DOM scanning, real-time coordinate switching, 3-layer busy detection, graceful OCR degradation, and Cursor upgrade resilience.