Skip to content

ARM-142: add vera-browser CLI for headless browsing - #81

Merged
stepandel merged 2 commits into
masterfrom
arm-142-headless-browser
Apr 9, 2026
Merged

ARM-142: add vera-browser CLI for headless browsing#81
stepandel merged 2 commits into
masterfrom
arm-142-headless-browser

Conversation

@agent-vera

@agent-vera agent-vera Bot commented Apr 9, 2026

Copy link
Copy Markdown

Summary

Gives Vera a real headless browser via a new vera-browser CLI so she can render websites, capture screenshots at specific viewports, extract post-JS-execution DOM, and evaluate JS in page context — things web_fetch (raw HTML only) can't do.

Closes ARM-142.

What's in the box

scripts/vera-browser.ts — bun CLI with three subcommands:

vera-browser screenshot <url> [--viewport <preset>] [--out <path>] [--wait-for <sel>] [--no-full-page] [--timeout <ms>]
vera-browser dom        <url> [--viewport <preset>] [--wait-for <sel>] [--timeout <ms>]
vera-browser eval       <url> --script <js> [--viewport <preset>] [--wait-for <sel>] [--timeout <ms>]

Viewport presets (width × height, default = desktop-1280):

preset css size mobile UA
iphone-se 375 × 667 iOS Safari
iphone-14-pro 393 × 852 iOS Safari
pixel-7 412 × 915 Android Chrome
desktop-1280 1280 × 800 Linux Chrome
desktop-1920 1920 × 1080 Linux Chrome

Mobile presets set isMobile: true, hasTouch: true, and a real iOS / Android UA string. Desktop presets use a Linux Chrome UA.

Behavior:

  • Launches Chromium headless with --no-sandbox --disable-dev-shm-usage (we trust the URLs the agent visits).
  • Default 30 s per-op timeout.
  • Navigation waits for networkidle by default; if --wait-for <selector> is given, waits for domcontentloaded + the selector instead (many SPAs never go fully idle).
  • Screenshots default to full-page; --no-full-page clips to the viewport.
  • Default output path: /tmp/vera-browser/<ts>-<slug>.png. The path prints on stdout so it can be piped straight into pi's read tool.
  • dom prints document.documentElement.outerHTML after JS runs (differs from web_fetch which gets raw HTML).
  • eval runs the user script via page.evaluate(new Function("return (" + script + ");")) and prints the JSON-stringified result. Subject to page CSP (fine for document.title-type probes).

Dockerfile — installs playwright@1.59.1 into /opt/vera/scripts and runs playwright install --with-deps chromium as root before the USER vera switch, so Chromium + system libs + fonts land in a shared, world-readable /opt/ms-playwright. PLAYWRIGHT_BROWSERS_PATH is set via ENV so it persists to runtime. A symlink at /usr/local/bin/vera-browser puts the script on PATH.

.vera/skills/vera-browser/SKILL.md — without a skill the CLI lands dark, so the agent has no keyword-triggered guidance to reach for it and falls back to web_fetch + CSS reasoning like before. The skill front-loads triggers (screenshot, mobile layout, responsive, viewport, visual review, frontend QA, SPA, headless browser, …) so they survive the 120-char in-prompt truncation in buildSkillsPrompt, and documents the concrete patterns:

  • read-back flow (vera-browser screenshot → read <path>)
  • mobile-audit loop (sweep iphone-se / iphone-14-pro / pixel-7 / desktop-1280)
  • SPA post-hydration DOM via vera-browser dom --wait-for
  • page-state probing via vera-browser eval
  • gotchas (timeout, full-page default, CSP on eval, --wait-for for SPAs that never go idle)

Auto-loaded by seedStateDirloadSkillSummariesbuildSkillsPromptForAgent. Zero code change required.

Image size impact: ≈ 400–500 MB (Chromium ≈ 200 MB, system libs + fonts ≈ 200 MB). Documented in the ticket; not a blocker.

Smoke tests (local simulation)

I reproduced the runtime stage layout locally (bookworm-slim userland, non-root vera user, same npm install playwright@1.59.1 + npx playwright install chromium steps, system libs and fonts backfilled via apt-get download + dpkg-deb -x into LD_LIBRARY_PATH / FONTCONFIG_FILE) and ran every DoD check against the real built script invoked via the /usr/local/bin/vera-browser symlink:

Test Result
vera-browser --help ✅ prints usage
vera-browser screenshot https://example.com --viewport iphone-se → PNG ✅ 750 × 1334 (375 × 667 @ 2× DPR), text visible
vera-browser eval https://example.com --script "document.title" "Example Domain"
vera-browser dom https://example.com ✅ contains Example Domain, exits 0
All 5 viewport presets ✅ 750×1334, 1179×2556, 1082×2402, 1280×800, 1920×1080
--wait-for h1 on example.com ✅ success
--wait-for "#definitely-not-here" on example.com vera-browser: forSelector: Timeout 3000ms exceeded., exit 1
--no-full-page clips to exact viewport ✅ 1280 × 800
Unreachable host (https://this-domain-does-not-exist.invalid) vera-browser: goto: net::ERR_NAME_NOT_RESOLVED …, exit 1, < 5 s
Localhost reachability (python3 -m http.server 8765vera-browser eval) ✅ returns "Directory listing for /"
SPA rendering (react.dev): rendered DOM bytes > raw curl HTML bytes ✅ 272,544 > 272,471 (post-hydration captured)
Screenshot readable via pi's read tool (png attachment) ✅ verified — I actually read the iphone-se PNG and saw real rendered text ("Example Domain" heading + paragraph + "Learn more" link)
quick_validate.py on the new skill ✅ valid frontmatter, conventional naming, triggers survive 120-char truncation

Proof of end-to-end render attached as a comment on ARM-142.

What's not in this PR

Per the ticket, these nice-to-haves are deferred (spin off follow-ups if/when needed):

  • Lighthouse integration
  • Click / type / scroll interactions
  • Console + network request capture
  • prefers-color-scheme / reduced-motion / geolocation emulation
  • Video recording / tracing

Follow-up ops notes

  • --ipc=host is Playwright's recommendation for Chromium memory (docker-run flag, not Dockerfile). If the host harness doesn't already set it, a runaway page could OOM the agent container. Out of scope here; worth a follow-up.
  • Playwright version is pinned to 1.59.1 so image rebuilds are reproducible. Browser version is implicitly pinned via Playwright.

Test plan for reviewer

docker build -t vera-local .
docker run --rm vera-local vera-browser --help
docker run --rm -v /tmp:/out vera-local \
  vera-browser screenshot https://example.com --viewport iphone-se --out /out/shot.png
docker run --rm vera-local vera-browser eval https://example.com --script "document.title"

Expected: help text, a 750 × 1334 PNG on the host, "Example Domain".

Gives Vera a real headless browser so she can render websites, take
screenshots at specific viewports, extract post-JS-execution DOM, and
evaluate JS in page context — things web_fetch can't do.

- scripts/vera-browser.ts: bun CLI with screenshot/dom/eval subcommands
  and five viewport presets (iphone-se, iphone-14-pro, pixel-7,
  desktop-1280, desktop-1920). Mobile presets set the iOS/Android UA
  and isMobile/hasTouch. 30s default timeout, --no-sandbox launch
  args, full-page screenshots by default. Navigation waits for
  networkidle unless --wait-for is supplied (then domcontentloaded +
  waitForSelector). Output path prints on stdout so it can be piped
  into pi's read tool.

- Dockerfile: install playwright@1.59.1 into /opt/vera/scripts and
  run 'playwright install --with-deps chromium' as root before the
  USER vera switch. PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright so
  the browser lives in a shared world-readable path. Symlinks the
  script to /usr/local/bin/vera-browser. Adds ~400–500 MB to the
  final image.
@linear

linear Bot commented Apr 9, 2026

Copy link
Copy Markdown
ARM-142 Give Vera a headless browser so she can actually render websites

Problem

When asked to "look at" a website, Vera can currently:

  • Fetch the HTML/markdown via web_fetch
  • Read the source in the repo and review CSS by hand

She cannot:

  • Render the page at a given viewport
  • Take screenshots
  • Evaluate JS-driven UI state
  • Verify responsive breakpoints visually
  • Run Lighthouse / accessibility audits that need a real browser

This came up today during a mobile-layout review of agent-vera.com — Vera had to caveat every finding with "I can't actually render the page, this is CSS reasoning only." That's a real gap for any frontend, QA, design-review, or visual-regression work.

Proposal

Add a headless browser capability to Vera's environment, exposed as a CLI she can drive via bash.

Requirements

Must have:

  • Load an arbitrary URL (local dev server OR public site)
  • Set viewport size (mobile presets: iPhone SE 375×667, iPhone 14 Pro 393×852, Pixel 7 412×915; desktop: 1280×800, 1920×1080)
  • Set user-agent (mobile UA strings)
  • Take a full-page screenshot → returned as an image Vera can actually view
  • Extract the rendered DOM after JS execution (differs from web_fetch which gets raw HTML)
  • Wait for network idle / specific selector before capturing

Nice to have:

  • Run arbitrary JS in page context (page.evaluate)
  • Click / type / scroll interactions (for flow testing)
  • Intercept console logs and network requests
  • Run Lighthouse programmatically and return the JSON report
  • Emulate prefers-color-scheme, reduced motion, geolocation
  • Record a short video / trace for complex flows

Implementation sketch

Playwright CLI wrapper
Install Playwright + Chromium in the base image, write a small vera-browser shell script that accepts subcommands:

vera-browser screenshot <url> --viewport mobile --out /tmp/shot.png
vera-browser dom <url> --wait-for "main"
vera-browser eval <url> --script "document.title"
vera-browser lighthouse <url> --preset mobile

Vera calls it from bash, reads output files with read (which supports png/jpg/webp as attachments — already confirmed in the environment).

Constraints / watch-outs

  • Playwright + Chromium is ~500MB — non-trivial image size bump
  • Sandbox: needs --no-sandbox or user namespace tricks when running as non-root in a container
  • Screenshots should land somewhere Vera can read them back as file attachments (confirmed read handles png/jpg/gif/webp)
  • For local dev server testing, needs to be able to reach localhost:<port> from wherever the browser runs
  • Probably want a per-run timeout + memory cap so a runaway page doesn't OOM the agent

Success criteria

After this lands, Vera can redo the agent-vera.com mobile audit with real screenshots at 375/393/768/1280 px viewports attached to her review — not just CSS reasoning. She can also close ARM-XXX (mobile fixes) by visually verifying the result rather than caveating.

Related

  • Blocks / enables deeper verification on the mobile-layout fix ticket
  • Enables future visual-regression workflows, design reviews, and frontend QA skills

🛠️ Prep — Playwright CLI wrapper

Sizing

Well-sized for one coding session. The work is a Dockerfile change + one script + a smoke test. No splitting.

Research Notes

  • Install path on Debian bookworm-slim: npx playwright install --with-deps chromium (run as root during build) handles both the Chromium download and the apt system libs in one shot. Avoids the historical install-deps breakage on bookworm.
  • Non-root launch: Vera runs as the non-root vera user. The simple path is to always launch Chromium with --no-sandbox --disable-dev-shm-usage. This is acceptable because we control the agent and the URLs it visits. Seccomp profile gymnastics (Playwright's crawling-untrusted guidance) are overkill here.
  • Browsers path: set ENV PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright before the install step so Chromium lands in a shared, world-readable location instead of /root/.cache, and survives the USER vera switch. Same env var needs to be present at runtime.
  • Image size: Chromium ≈ 200 MB + browser system deps ≈ 200 MB. Expect a ~400–500 MB bump as the ticket predicted. Not a blocker, worth noting in the PR description.
  • Bun + Playwright: Bun resolves the playwright npm package fine via node_modules. A TS script with #!/usr/bin/env bun shebang is the cleanest fit for this repo (matches the existing scripts/slack-test.ts style).
  • read tool image support: confirmed via the pi extension docs (docs/extensions.md references event.images with image/png base64 payloads). Also independently validated by ARM-139 (merged), which wired slack image attachments through the same image pipeline. Vera will actually see returned screenshots.

Entry Points

  • repos/anton/Dockerfile (lines 21–50) — runtime stage. Add Playwright install before the USER vera switch on line 69, so Chromium is installed as root into /opt/ms-playwright. The existing apt-get install block at lines 24–50 is the natural place to append; playwright install --with-deps chromium handles the system libs so no manual apt additions are needed.
  • repos/anton/Dockerfile lines 75–80 — ENV block. Add ENV PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright here (and also export it before the install step earlier in the file).
  • repos/anton/scripts/ — new file vera-browser.ts. Model after scripts/slack-test.ts (same bun-script style). Use #!/usr/bin/env bun shebang so it's directly executable.
  • repos/anton/Dockerfile line 58 (COPY scripts /opt/vera/scripts) — already copies the scripts dir. Add two new lines after it:
    1. RUN chmod +x /opt/vera/scripts/vera-browser.ts
    2. RUN ln -s /opt/vera/scripts/vera-browser.ts /usr/local/bin/vera-browser
  • repos/anton/package.json — add playwright to dependencies (not dev) so the module is available to bun's node_modules resolution inside the runtime image. The runtime stage currently only copies the compiled vera-server/vera-headless binaries and package.json, NOT node_modules — so you'll also need to either (a) install playwright globally via npm during the apt step, or (b) copy node_modules/playwright* from the deps stage into /opt/vera/scripts/node_modules. (a) is simplernpm install -g playwright then PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright npx playwright install --with-deps chromium, and have the script import from the global install via NODE_PATH or by resolving /usr/lib/node_modules/playwright.

Subcommand surface (must-have for this ticket)

Match the sketch in the ticket body:

vera-browser screenshot <url> [--viewport <preset>] [--out <path>] [--wait-for <selector>] [--full-page] [--timeout <ms>]
vera-browser dom        <url> [--viewport <preset>] [--wait-for <selector>] [--timeout <ms>]
vera-browser eval       <url> --script <js> [--viewport <preset>]
  • Viewport presets (required): iphone-se (375×667), iphone-14-pro (393×852), pixel-7 (412×915), desktop-1280 (1280×800), desktop-1920 (1920×1080). Default: desktop-1280. Mobile presets must also set the mobile UA and isMobile: true / hasTouch: true.
  • Default output: /tmp/vera-browser/<timestamp>-<slug>.png if --out omitted. Print the path to stdout so Vera can pipe it into read.
  • Default timeout: 30s per page op; navigation waits for networkidle unless --wait-for selector is given.
  • Launch args (hardcoded): ['--no-sandbox', '--disable-dev-shm-usage'].
  • dom subcommand: prints document.documentElement.outerHTML to stdout after JS execution settles — this is the whole point vs. web_fetch.
  • eval subcommand: runs the provided JS in page context via page.evaluate(), prints the JSON-stringified result to stdout.

Nice-to-haves deferred (do NOT do in this ticket — spin off follow-ups if needed): lighthouse, interactions (click/type/scroll), console/network capture, video recording, prefers-color-scheme emulation. The ticket body correctly marks these as "nice to have" and this ticket should not balloon.

Definition of Done

  • docker build of repos/anton/Dockerfile completes cleanly; image size bump documented in PR description (expect ≈ 400–500 MB).
  • Inside a built container, running vera-browser --help prints usage.
  • vera-browser screenshot https://example.com --viewport iphone-se --out /tmp/shot.png produces a valid PNG at mobile viewport dimensions as the non-root vera user, with no sandbox errors.
  • The produced PNG is readable via pi's read tool (i.e. Vera can actually see it).
  • vera-browser dom https://example.com prints rendered HTML to stdout (non-empty, contains expected elements).
  • vera-browser eval https://example.com --script "document.title" prints "Example Domain" (or equivalent) to stdout.
  • All 5 viewport presets work (spot-check one mobile + one desktop in PR description).
  • Default 30s timeout kicks in cleanly on an unreachable URL (no hung process, clear error message).
  • PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright is set in the final image ENV and the script honors it.
  • Script runs under bun (matches repo convention — no new runtime).
  • PR description includes the screenshot-of-example.com smoke test output attached as proof it works end-to-end.

Test cases

  1. Mobile screenshot smoke testvera-browser screenshot https://example.com --viewport iphone-se --out /tmp/iphone.png produces a ~375px-wide PNG; Vera reads it back and can describe the content.
  2. JS-rendered DOM — point at a SPA (e.g. https://react.dev) and verify vera-browser dom returns post-hydration HTML that differs meaningfully from curl's raw HTML.
  3. Wait-for selectorvera-browser screenshot https://example.com --wait-for "h1" succeeds; same command with --wait-for "#definitely-not-here" fails within timeout with a clean error.
  4. Localhost reachability — start a trivial python3 -m http.server 8000 in the container and screenshot http://localhost:8000 to confirm the browser can reach in-container services. (If this fails, document the workaround — it shouldn't, since everything is in one container.)
  5. Unreachable hostvera-browser screenshot https://this-domain-does-not-exist.invalid exits non-zero within the timeout with a readable error on stderr.

Edge cases / gotchas

  • The Playwright Chromium download must happen as root, before the USER vera switch (line 69), otherwise the non-root user can't write to /opt/ms-playwright. Symmetrically, /opt/ms-playwright must be readable (default 755) by vera after the switch.
  • The runtime stage does not copy node_modules from the deps stage — only the compiled binaries. So playwright the npm package must be installed directly in the runtime stage (via npm install -g playwright during the existing apt block on lines 40–47), not relied on from /build/node_modules.
  • Bun script resolving a globally-installed npm package: pass NODE_PATH=/usr/lib/node_modules in the script's shebang env or set it globally in the Dockerfile ENV block. Verify with bun -e "import('playwright').then(p => console.log(!!p))" during the Dockerfile build.
  • /tmp inside the container is writable by vera and ephemeral per run — fine as the default screenshot directory. Create /tmp/vera-browser/ in the script on first use.
  • --ipc=host is a Playwright recommendation for Chromium memory, but that's a docker-run flag (not Dockerfile). If the harness that runs this container doesn't already set it, a runaway page could OOM. Out of scope for this ticket, but note it in the PR description as a follow-up.
  • Playwright pins browser versions to its npm version. Lock to a specific playwright version in the Dockerfile (e.g. npm install -g playwright@1.58.x) so image rebuilds are reproducible.

In-flight work scan

Implementation prompt

Add a headless browser to Vera: in repos/anton/Dockerfile runtime stage (before the USER vera switch on line 69), install playwright@1.58.x globally via npm install -g playwright and download Chromium via PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright npx playwright install --with-deps chromium; add a new repos/anton/scripts/vera-browser.ts (bun shebang, modeled on scripts/slack-test.ts) implementing screenshot/dom/eval subcommands with the viewport presets, --no-sandbox --disable-dev-shm-usage launch args, and 30s default timeout listed in the Definition of Done above; symlink it to /usr/local/bin/vera-browser in the Dockerfile; verify end-to-end by screenshotting https://example.com at iphone-se and attaching the PNG to the PR description.

Without a skill the headless browser CLI lands dark — the agent has no
keyword-triggered guidance to use it and falls back to web_fetch + CSS
reasoning like before. This adds .vera/skills/vera-browser/SKILL.md so
'mobile layout', 'screenshot', 'render', 'visual review', etc. routes
to the new tool with concrete patterns:

- read-back flow (vera-browser screenshot → read <path>)
- mobile-audit loop (sweep iphone-se / iphone-14-pro / pixel-7 / desktop-1280)
- SPA post-hydration DOM via vera-browser dom --wait-for
- page state probing via vera-browser eval

Skill is auto-loaded by seedStateDir → loadSkillSummaries; no code
change needed. Description front-loads triggers so they survive the
120-char in-prompt truncation in buildSkillsPrompt.
@stepandel
stepandel merged commit de96ec6 into master Apr 9, 2026
2 checks passed
@stepandel
stepandel deleted the arm-142-headless-browser branch April 9, 2026 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant