watchcat

Give coding agents the ability to watch what happens. The name is the pipeline: watch something, then cat the output. watchcat turns a screen recording into timestamped prose observations using Gemini video understanding, so a coding agent (or you) can know what actually happened on screen — motion, latency, jank, dead buttons, transitions — without watching the video.

export GEMINI_API_KEY=...

watchcat watch session.mp4 --context "I clicked Add to Cart at ~0:08; the drawer should slide up smoothly"

watchcat record start --window "Chrome"  # or --screen / --sim; fallback for live sessions
# ... drive the app ...
watchcat record stop               # prints the finalized mp4

Design stance:

• Prose is the contract. No verdicts, no pass/fail, no output schema. watchcat observes; the reader judges. The consuming agent's intelligence absorbs the ambiguity — that's why this works.
• Video is the narrow waist. watchcat knows nothing about frameworks or platforms. Anything that makes pixels feeds watch; record exists only for live sessions nothing else records (frameworks that self-record bypass it entirely).
• The archive is the product's memory. Every run writes a JSON sidecar to ~/.watchcat/runs/ — prompt version, flags, raw response, token cost. Failed runs too. That corpus is how the observation prompt gets calibrated with evidence instead of vibes. The prompt is versioned data (prompts/observe.v1.txt, overridable at ~/.watchcat/prompts/observe.txt).
• Built for unattended callers. Exit codes are the machine API (0 ok · 1 caller error · 2 transient/retried · 3 permanent · 4 record state conflict · 5 internal); stderr is key=value telemetry with a heartbeat during long flex-tier waits; a delivered answer is never lost to an archive failure; a judgment is never silently re-rolled.

Cost: about $0.01 per minute of video at default settings on gemini-3.7-flash; --res low cuts tokens ~3x, --res high spends ~3.5x for verbatim small text, --tier flex halves price and waits for capacity (retries with heartbeats up to 20 minutes).

Build

go build -o watchcat .
go test ./...

macOS only for record (--screen needs ffmpeg + Screen Recording permission; --sim needs Xcode). watch is pure API and runs anywhere.

Calibration (measured, 2026-08-25)

A seeded-defect pass (dead button, never-resolving spinner, delayed response with layout shift, plus working controls) scored:

• Behavior detection 5/5 at default resolution: the dead button, stuck spinner, layout shift, and feedback timing were all caught blind, with no false alarms on the working controls.
• Quoted text 4/5 wrong at default resolution, 5/5 verbatim at --res high (~3.5x tokens). The confabulation is perceptual, not prompt-fixable: the model reads wrong text confidently, so the rule lives at the caller layer — trust what-happened-when; re-watch with --res high when exact wording matters.

Known limits

• Default sampling is 1 fps; sub-second animation detail needs --fps (≤24) and ideally --clip around the moment.
• --window requires the window to stay on screen (occluded is fine, minimized is not), and titles are only visible to a process with Screen Recording permission.

Claude Code plugin

This repo is also a Claude Code plugin: .claude-plugin/plugin.json + skills/watchcat/SKILL.md teach an agent when to reach for watchcat and how to write good --context. The binary must be on PATH (or built here).