v1.0.0 — 2026-07-12
Video skills for every AI agent, with memory. One release, seven claims,
each shipped with a measured receipt from the reference machine (8 GB
RAM, CPU-only Windows) — the benchmark tables and demo logs quoted in
the README all come from real runs there.
Everywhere — the skills library and the agent matrix
- Nine-skill library (
adapters/claude-skill/skills/):
watching-videos,asking-with-evidence,the-loop,
learning-from-mistakes,extracting-structure,video-memory,
sharing-results,configuring-vision, andrecovering-from-errors.
Each SKILL.md description is a trigger surface with
real user phrasings; each body wraps the CLI only, so the set rides
into any harness that reads skills./watchremains the direct
user-invocable entry point. - Provider-neutral setup:
setup-visionnow configures Anthropic,
OpenAI, Gemini, OpenRouter, or the optional local Ollama path. One model
can serve both tiers, or--cheap-model/--strong-modelcan route
perception and verification separately. The agent client and model
vendor are independent choices. - 20+ agents in the matrix (
docs/agents/): twelve new pages —
GitHub Copilot CLI, Kimi Code, Qwen Code, OpenCode, Goose, OpenHands,
Kilo Code, Qodo, Agent Zero, OpenClaw, Pi, Hermes-style — each written
against the agent's CURRENT official docs and graded honestly
(machine-tested / machine-configured / doc-verified). Every fenced
config block in every page is parsed by
templates/agent-adapter/validate.py, wired into the suite. - Add-your-agent funnel:
templates/agent-adapter/(walkthrough +
skeleton + validator) — one config block, one doc page, ~20 minutes. - Agent visual system: every named client guide has a logo-derived
pixel-art avatar, the README gallery covers all supported agents, and
tests prevent guide/gallery/assets from drifting apart. - Sixteen examples: a task-oriented catalog now includes a private,
offline workflow and self-contained viewer export alongside the original
fourteen demonstrations.
Remembers — the library layer (index migration v7, new tables only)
- Every watch distills notes — entities, claims, chapters, each with
(video_id, timestamp) provenance — incrementally: video N never
reprocesses the others. Works transcript+OCR-only; vision adds
material. library_synthesize(question)(MCP + CLIlibrary ask+ REST):
answers questions no single video holds, extractively and offline —
per-video timestamp citations, corroboration across videos raises
confidence, honest floor when the library does not clearly know.
Cached with automatic invalidation when the library grows.
library_overview(): what the library knows.- Live receipt: a 4-clip incident story answered across all four clips
(confidence 0.566, corroborated, repeat served from cache, ~784 tokens
saved on the meter).library rebuild-notesupgrades pre-v7 indexes.
Nearly free — the cost meter and THE COST POLICY
- Cost meter v2: every answer carries
cost_breakdown(tokens by
source: text-first / local escalation / vision call / response frames)
and a USD estimate; lifetime split behindwatch-skill stats --cost.
Prices live in a dated data file (vision/prices.json). WATCHSKILL_COST_POLICY:cheapest(default — cheapest path that
clears confidence wins),quality_first, oroffline_only(cloud
never sees a frame).benchmarks/cost/, committed from a real run: ~5,868 tokens fully
offline vs ~18,890 computed for raw-frames-into-context on the same
15-frame index — $0.00 measured, before the cache makes repeats free.
High-quality vision anywhere — perception with receipts
- OCR backend registry (
perceive/ocr_backends.py): rapidocr
default; tesseract auto-routed ONLY for the scripts rapidocr 3.9.1
genuinely lacks (Lao/Khmer/Myanmar/Tibetan — audited against its
LangRec enum); surya opt-in, never auto-routed on small machines. - Multi-script-per-frame router: each candidate script engine reads
the full frame; regions merge by overlap, gated on the engine finding
its own script there. On the committed mixed code+Arabic+CJK fixture:
98% char-hit vs 81% for the best single engine. (The first design
re-read cropped regions and measured WORSE than no routing — the bench
is why it was rebuilt.) watch-skill bench perception+ committed fixtures and results:
char-hit, latency, peak RSS per backend — including the vision rows
that show why a captioning model cannot replace OCR (moondream: 18%
on Arabic, 0% on CJK; OCR: 94–100%).- Local vision robustness: liveness-cached health check, ONE
detached restart of a dead Ollama (neverollama stop), one settled
retry on a 5xx from a fresh server, and a structured
vision.server_down(with a fix) instead of empty strings — the
kill-the-server scenario is a recorded live demo, both branches. - Opt-in retrieval upgrade:
WATCHSKILL_EMBEDDING_MODEL(bge-m3,
multilingual-e5) seeds NEW indexes; existing indexes keep their pinned
model. Big models want ~2 GB+ RAM — documented, not defaulted.
Heals itself
doctor --fixrepairs every failure class this project has hit:
dead local vision server (detached restart), corrupt cached answers
(quarantined), truncated model files (deleted; they re-download),
vanished frame directories (reindex hint), stale WAL, tight commit
headroom (with a local-model recommendation for the machine), and a
missing Playwright recording runtime (installed automatically).- Privacy controls now hold during escalation: disabling OCR also
disables dense-resample and zoom-crop OCR, so an offline or deliberately
OCR-free run never downloads a model behind the user's back. - Structured-errors audit: every raise site in
src/carries an
actionablefix— enforced forever by an AST-walking test; ten real
error paths asserted to return executable advice. 25 sites were
patched to get there.
Improves itself
lessons eval --reportreplays every stored lesson against the
CURRENT pipeline — once normally, once with the lesson suppressed —
and classifies it: still-effective (load-bearing), prunable (the
pipeline absorbed the fix), regressed (needs a human).
--pruneretires exactly the prunable ones.- The mechanics in one page:
docs/guides/how-it-improves-itself.md.
Building the live demo caught three real eval bugs (stopword terms
passed everything; the floor text leaked question words; hallucination
phrasing misclassified) — fixed and regression-tested.
Useful to everyone — the packs (docs/packs/)
- Browser-agent verification (the flagship): agents can drive real
browsers now; a screenshot shows a moment, not a flow. The pack
records the session and verdicts the RECORDING —
examples/14-browser-verification/catches a checkout total that
reads $NaN for 1.5 s mid-flow and looks perfect afterwards. Building
it exposed and fixed two real defects: grayscale phash dedup collapsed
hue-only flows to one frame (loop/monitor critiques now pin undedupable
flow cues), and "never shows nan" banned an unmatchable verb
phrase (the parser now sheds light verbs). - Monitoring/ops: monitor events now deliver to
WATCHSKILL_WEBHOOK_URL— HMAC-SHA256-signed, retried with backoff,
never fatal,events.jsonlregardless — tested against a live local
receiver. This is the piece n8n/Zapier builders were missing. - QA/bug hunting, content creators, learning/research,
meetings/lectures, agent self-verification: recipes over existing
tools, each pointing at a runnable example with recorded output.
Compatibility
- No breaking changes across the whole span: every v0.6 MCP tool
name/signature unchanged (pinned by test), CLI intact, index
migrations forward-only v5→v6→v7,~/.watch-skill/loses nothing.
v0.6 users upgrade straight to v1.0.
Foundation (built en route, first released here)
Everything below was completed and live-proven after v0.6.0 and ships
for the first time in this release.
One-command install (adapters/claude-skill/, .claude-plugin/)
- Claude Code plugin marketplace:
/plugin marketplace add oxbshw/watch-skill
→/plugin install watch-skill@watch-skill→ a working/watch, zero
manual venv steps. The bundled MCP config launches the on-PATH engine. - New
/setup-watch-skillcommand: installs the engine (uv bootstraps
its own Python), runs the self-healing doctor, registers the MCP server in
every detected agent (Claude Code/Desktop, Cursor, Codex, Windsurf, Gemini
CLI — each with a config backup), then offers a vision backend.
Vision backends (health/vision_setup.py, vision/)
watch-skill setup-vision: Anthropic, OpenAI, Gemini, OpenRouter,
or Ollama fully offline. Cloud setup accepts an existing provider
key and optional model names;--verifyruns a live probe-frame describe.- Low-RAM machines are first-class: RAM-aware model pick (moondream under
12 GB), context window sized to fit (WATCHSKILL_OLLAMA_NUM_CTX),
temperature-0 reproducible calls, keep-alive pinning, and the loop
producers unload the local model before browser captures (a resident
model and a recording browser cannot coexist in 8 GB).
THE LOOP, multiplied (loop/)
- The UI loop is now proven with real vision: broken page flagged from
actual model reads, fix verified, before/after GIF+MP4 rendered. - Pluggable loop framework: a loop type is a registry entry deciding how
the recording is produced;loop_start/loop_iterateare unchanged. - Three new loop types, each an MCP tool + CLI + runnable example:
loop_video_gen(run any generator — Manim/Remotion/ffmpeg — watch
the render, iterate until it matches the spec),loop_game(launch a
game/sim, record gameplay, catch visual/state glitches like a NaN HUD),
loop_monitor(bounded watch over a folder/stream; a described
condition becomes a structured event inevents.jsonl+ callback — the
v0.8 webhook seam). - Describe-then-judge critic: small captioning models (moondream) can't
emit the critic's JSON, but they describe frames dependably — so
deterministic rules parsed from your criteria decide (banned terms from
"never X" fail a frame; exemplar shapes from "(like $29.00)" pass the
recording; digit-generalized and whitespace-tolerant, so a misread
"ERROR 5082" still matches), with a plain PASS/FAIL judgment only where
no rule speaks.critique_recordingdegrades automatically; capable
models keep the full JSON critic.
For every agent framework (integrations/, docs/agents/frameworks.md)
- Thin native adapters — LangChain, CrewAI, OpenAI Agents SDK, LlamaIndex,
AutoGen — all wrapping the same three core calls; install via extras
(pip install "watch-skill[langchain]"). Vercel AI SDK via the REST
surface; an n8n community-node spec; REST/OpenAPI as the universal
fallback.
Structured extraction (extract/)
extract_chapters: titled chapters from scene cuts + transcript
pauses, minimum length scaled to duration.extract_bug_report: the first on-screen error — timestamp, frame,
exact OCR text, and repro steps from the preceding narration; returns
found: falseinstead of guessing.analyze_hook: the first N seconds scored on attention trigger,
pacing, visual change, and on-screen text — each with an actionable
critique.
Batch + the shareable viewer (batch.py, viewer.py)
watch_batch: one call indexes a playlist/channel URL, a folder, or a
list; one broken video never kills the batch; afterwards a single
search_videos/ask_videospans the whole set.generate_viewer: a self-contained offline HTML page per analysis —
timeline, inlined key frames, transcript, OCR, and every cached answer
with the exact evidence cited. Zero network requests; share the file as-is.
Search that actually works across scripts (index/textnorm.py)
- Thai/Lao/Khmer/Myanmar/Tibetan are now segmented (search was fully broken
for unspaced scripts); Persian/Urdu letter variants unify with Arabic;
Arabic-Indic/Persian/Devanagari/Bengali/Thai/Lao/Tibetan/Myanmar/Khmer
digits fold to ASCII ("٢٠٢٦" matches "2026"); Hebrew niqqud + final
forms, Greek final sigma + tonos, German ß/umlauts, Cyrillic ё, and
Vietnamese diacritics fold too. Forward migration v6 re-folds existing
indexes in place — nothing is lost, nothing re-processed.
The engine answers in your language (answer/localize.py)
- The honest-floor refusal, evidence labels, and the model-answer directive
follow the question's language (13 languages); the loop critic follows the
pass criteria's language. Cross-lingual answers are a tested contract, not
luck. RTL text can't mangle timestamps: they're wrapped in Unicode
isolates.