TextWeb

A text-grid web renderer for AI agents — see the web without screenshots.

Instead of taking expensive screenshots and piping them through vision models, TextWeb renders web pages as structured text grids that LLMs can reason about natively. Full JavaScript execution, spatial layout preserved, interactive elements annotated.

🐙 Fork: sammcj/textweb · 🐙 Upstream: chrisrobison/textweb · 📄 Upstream docs

Why?

Approach	Size	Requires	Speed	Spatial Layout
Screenshot + Vision	~1MB	Vision model ($$$)	Slow	Pixel-level
Accessibility Tree	~5KB	Nothing	Fast	❌ Lost
Raw HTML	~100KB+	Nothing	Fast	❌ Lost
TextWeb	~2-5KB	Nothing	Fast	✅ Preserved

Note: This is a personal fork of chrisrobison/textweb with pnpm migration, the official @modelcontextprotocol/sdk, bounded MCP session lifecycle, TypeScript type definitions, and assorted cleanup.

Quick Start

pnpm install
pnpm exec playwright install chromium

# Render any page
textweb https://news.ycombinator.com

# Explicitly request grid mode (same as default)
textweb --output grid https://news.ycombinator.com

# Semantic JSON output for agent workflows
textweb --output semantic https://example.com

# Hybrid output (grid + semantic metadata)
textweb --output hybrid https://example.com

# Interactive mode
textweb --interactive https://github.com

# Legacy JSON output (backward compatible)
textweb --json https://example.com

Example Output

[0]Hacker News [1]new | [2]past | [3]comments | [4]ask | [5]show | [6]jobs | [7]submit      [8]login

 1. [9]Show HN: TextWeb – text-grid browser for AI agents (github.com)
    142 points by chrisrobison 3 hours ago | [10]89 comments
 2. [11]Why LLMs don't need screenshots to browse the web
    87 points by somebody 5 hours ago | [12]34 comments

[13:______________________] [14 Search]

~500 bytes. An LLM can read this, understand the layout, and say "click ref 9" to open the first link. No vision model needed.

Integration Options

TextWeb works with any AI agent framework. Pick your integration:

🔌 MCP Server (Claude Desktop, Cursor, Windsurf, Cline, Claude Code)

The fastest way to add web browsing to any MCP-compatible client. Built on the official @modelcontextprotocol/sdk.

# From a clone
pnpm install
pnpm run mcp

# Or via the bin entry once globally linked
pnpm link --global
textweb-mcp

Claude Desktop - add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "textweb": {
      "command": "node",
      "args": ["/absolute/path/to/textweb/mcp/index.js"]
    }
  }
}

Cursor - add to .cursor/mcp.json:

{
  "mcpServers": {
    "textweb": {
      "command": "node",
      "args": ["/absolute/path/to/textweb/mcp/index.js"]
    }
  }
}

Session bounds (override via env if needed):

• TEXTWEB_MAX_SESSIONS - cap on concurrent browser sessions (default: 8)
• TEXTWEB_SESSION_IDLE_MS - idle timeout before a session is closed (default: 30 minutes)

MCP capabilities:

• session_id on every tool call for isolated parallel workflows
• textweb_storage_save / textweb_storage_load for persistent auth/session state
• textweb_wait_for for multi-step async UI transitions
• textweb_assert_field for flow guards before submit
• textweb_session_list / textweb_session_close for session introspection

🛠️ OpenAI / Anthropic Function Calling

Drop-in tool definitions for any function-calling model. See tools/tool_definitions.json.

Pair with the system prompt to teach the model how to read the grid:

import json

# Load tool definitions
with open("tools/tool_definitions.json") as f:
    textweb_tools = json.load(f)["tools"]

# Load system prompt
with open("tools/system_prompt.md") as f:
    system_prompt = f.read()

# Use with OpenAI
response = openai.chat.completions.create(
    model="gpt-4",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": "Go to example.com and click the first link"},
    ],
    tools=textweb_tools,
)

🦜 LangChain

from tools.langchain import get_textweb_tools

# Start the server first: textweb --serve 3000
tools = get_textweb_tools(base_url="http://localhost:3000")

# Use with any LangChain agent
from langchain.agents import initialize_agent
agent = initialize_agent(tools, llm, agent="zero-shot-react-description")
agent.run("Find the top story on Hacker News")

🚢 CrewAI

from tools.crewai import TextWebBrowseTool, TextWebClickTool, TextWebTypeTool

# Start the server first: textweb --serve 3000
researcher = Agent(
    role="Web Researcher",
    tools=[TextWebBrowseTool(), TextWebClickTool(), TextWebTypeTool()],
    llm=llm,
)

🌐 HTTP API

# Start the server
textweb --serve 3000

# Navigate
curl -X POST http://localhost:3000/navigate \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://example.com"}'

# Click, type, scroll
curl -X POST http://localhost:3000/click -d '{"ref": 3}'
curl -X POST http://localhost:3000/type -d '{"ref": 7, "text": "hello"}'
curl -X POST http://localhost:3000/scroll -d '{"direction": "down"}'

📦 Node.js Library

const { AgentBrowser } = require('textweb');

const browser = new AgentBrowser({ cols: 120 });
const { view, elements, semantic, meta } = await browser.navigate('https://example.com');

console.log(view);        // The text grid
console.log(elements);    // { 0: { selector, tag, text, href }, ... }
console.log(semantic);    // { mode, url, title, elements: [...] }
console.log(meta.stats);  // { totalElements, interactiveElements, renderMs }

await browser.click(3);              // Click element [3]
await browser.type(7, 'hello');      // Type into element [7]
await browser.scroll('down');        // Scroll down
await browser.waitFor({ selector: '.step-2.active' }); // Wait for next step
await browser.assertField(7, 'hello', { comparator: 'equals' }); // Validate field state
await browser.saveStorageState('/tmp/textweb-state.json');
await browser.loadStorageState('/tmp/textweb-state.json');
await browser.query('nav a');        // Find elements by CSS selector
await browser.screenshot();          // PNG buffer (for debugging)
console.log(browser.getCurrentUrl());// Current page URL
await browser.close();

Grid Conventions

Element	Rendering	Interaction
Links	[ref]link text	click(ref)
Buttons	[ref button text]	click(ref)
Text inputs	[ref:placeholder____]	type(ref, "text")
Checkboxes	[ref:X] / [ref: ]	click(ref) to toggle
Radio buttons	[ref:●] / [ref:○]	click(ref)
Dropdowns	[ref:▼ Selected]	select(ref, "value")
File inputs	[ref:📎 Choose file]	upload(ref, "/path")
Headings	═══ HEADING ═══	—
Separators	────────────────	—
List items	• Item text	—

How It Works

┌─────────────────────────────────────────────┐
│  Your Agent (any LLM)                        │
│  "click 3" / "type 7 hello" / "scroll down"  │
├─────────────────────────────────────────────┤
│  TextWeb                                     │
│  Pixel positions → character grid            │
│  Interactive elements get [ref] annotations  │
├─────────────────────────────────────────────┤
│  Headless Chromium (Playwright)              │
│  Full JS/CSS execution                       │
│  getBoundingClientRect() for all elements    │
└─────────────────────────────────────────────┘

1. Real browser renders the page (full JS, CSS, dynamic content)
2. Extract every visible element's position, size, text, and interactivity
3. Map pixel coordinates to character grid positions (spatial layout preserved)
4. Annotate interactive elements with [ref] numbers for agent interaction

Selector Strategy

TextWeb builds stable CSS selectors for each interactive element, preferring resilient strategies over brittle positional ones:

Priority	Strategy	Example
1	#id	#email
2	[data-testid]	[data-testid="submit-btn"]
3	[aria-label]	input[aria-label="Search"]
4	[role] (if unique)	[role="navigation"]
5	[name]	input[name="email"]
6	a[href] (if unique)	a[href="/about"]
7	nth-child (fallback)	div > a:nth-child(3)

This means selectors survive DOM changes between snapshots — critical for multi-step agent workflows.

ATS Workflow Examples (Greenhouse / Lever)

For multi-step ATS flows, use a stable session_id and combine wait/assert guards:

// Keep one session for the whole application
await textweb_navigate({ url: 'https://job-boards.greenhouse.io/acme/jobs/123', session_id: 'apply-acme' });

// Fill + continue
await textweb_type({ ref: 12, text: 'Christopher', session_id: 'apply-acme' });
await textweb_type({ ref: 15, text: 'Robison', session_id: 'apply-acme' });
await textweb_click({ ref: 42, session_id: 'apply-acme', retries: 3, retry_delay_ms: 400 });

// Guard transition
await textweb_wait_for({ selector: '#step-2.active', timeout_ms: 8000, session_id: 'apply-acme', retries: 2 });

// Validate before submit
await textweb_assert_field({ ref: 77, expected: 'San Francisco', comparator: 'includes', session_id: 'apply-acme' });

// Persist auth/session for follow-up flow
await textweb_storage_save({ path: '/tmp/ats-state.json', session_id: 'apply-acme' });

Useful session tools:

• textweb_session_list → inspect active sessions
• textweb_session_close → close one session or all

App Runtime Prototype (Manifest + LARC) - Experimental

⚠️ Experimental. This is an early scaffold inherited from upstream. APIs and file layout may change. The core CLI, MCP server, HTTP API, and Node library are unaffected.

Manifest-driven user runtime shell (separate from low-level raw admin tooling):

• Manifest validator: src/app-runtime/manifest.js
• PAN topic contract: src/app-runtime/topics.js
• Runtime shell + left nav + tabbed content: canvas/app-runtime/app-shell.html
• Sample manifest: canvas/app-runtime/sample-app.json

The runtime shell uses LARC PAN (@larcjs/core-lite) for in-page event communication, with a local fallback bus if the module cannot be loaded.

To open the prototype:

# Start API server for integration hooks (save manifest/components)
pnpm run serve

# In another terminal, open the runtime shell (path is relative to your clone)
open ./canvas/app-runtime/app-shell.html

Integration actions implemented on the API server:

• POST /integrations/sync_saved_form
• POST /integrations/save_manifest
• POST /integrations/upsert_nav_item
• POST /integrations/runtime_state

Testing

# Run all tests (app-runtime manifest + integrations + CLI output modes)
pnpm test

# Individual suites
pnpm run test:app-runtime
pnpm run test:integrations
pnpm run test:output

Test fixtures live in test/fixtures/ (HTML form and an ATS-style multi-step application). Tests are intentionally lightweight - this is a personal project, not a production tool.

Design Principles

1. Text is native to LLMs — no vision model middleman
2. Spatial layout matters — flat element lists lose the "where"
3. Cheap and fast — 2-5KB per render vs 1MB+ screenshots
4. Full web support — real Chromium runs the JS
5. Interactive — reference numbers map to real DOM elements

License

MIT © Christopher Robison