Claude Code Now Watches Videos — And It Reverse-Engineers Hooks Better Than I Do

A Claude Code skill that watches videos for you — frames + transcript together — tuned for creators reverse-engineering what makes hooks and retention beats actually work.

Hook

The pause-and-scrub frame Left: Screen recording of dragging the YouTube scrubber back and forth, opening Notes, typing "0:03 zoom in on…", scrubbing again. Sped up 4x. Timestamp counter: 00:00 → 09:42. Right: Same video URL pasted into Claude. Cursor blinks. A frame-by-frame breakdown materializes: "0:00 wide shot · 0:02 cut to phone · 0:04 text overlay 'I tried this for 30 days'…" Voiceover: "Stop scrubbing. Claude can actually watch the video now."

What this repo gives you

• A Claude Code skill (/watch) that pulls any video via yt-dlp, slices frames and audio with ffmpeg, and hands Claude the timestamped frame+transcript pair.
• Three creator-focused prompt presets: hook analysis, retention review, and bulk library ingestion.
• Three effort levels (low/medium/high) that trade frame density for cost.
• A cost summary written to disk on every run — duration, frames, transcription provider, estimated Claude tokens, total $.
• An Obsidian-ready hook library template you can point at a CSV of URLs and walk away from.

Setup

1. Clone this repo and run ./install.sh — installs yt-dlp and ffmpeg if missing and copies the skill into .claude/skills/watch/ (project-local) or ~/.claude/skills/watch/ (with --global).
2. (Optional, only for videos without native captions) Copy .env.example to .env and add either an ELEVENLABS_API_KEY (preferred, ~$0.40/hr) or a GROQ_API_KEY (Whisper Large v3 Turbo, ~$0.04/hr with a generous free tier).
3. In Claude Code, run /watch <url> — defaults to hook analysis at medium effort.
4. For batch mode, edit templates/batch-urls.csv.example and run /watch --batch templates/batch-urls.csv to populate your hook library.

Under the hood

/watch <url> is a thin Bash entrypoint (skill/watch.sh) that orchestrates four small Unix tools and writes everything to /tmp/watch/<sha-of-url>/. Claude reads the resulting frames + transcript + cost summary directly — no custom video model required, since the claude-opus-4-7 you're already running is multimodal.

URL or local file
       │
       ▼
┌──────────────────────────────────────────────────────────────────────────┐
│  skill/watch.sh   — argv parser + orchestrator                           │
└──┬────────────┬─────────────────┬──────────────────────────┬─────────────┘
   │            │                 │                          │
   ▼            ▼                 ▼                          ▼
┌──────┐  ┌─────────────┐  ┌─────────────────────┐   ┌──────────────────┐
│yt-dlp│  │ffmpeg-frames│  │captions or transcribe│  │ cost.json writer │
│      │  │             │  │  (Eleven / Groq)     │  │                  │
└──────┘  └─────────────┘  └─────────────────────┘   └──────────────────┘
   │            │                 │                          │
   ▼            ▼                 ▼                          ▼
video.mp4    frames/*.jpg     transcript.txt              cost.json
captions.txt audio.m4a        transcript.txt.provider

Step 1 — fetch (lib/ytdlp-fetch.sh)

yt-dlp does the heavy lifting. It supports YouTube, Loom, Instagram, TikTok, Twitter, Vimeo, and ~1,000 other sites, plus local file paths (which it just copies into the work dir). Native captions come for free if the platform has them — yt-dlp --write-auto-subs --sub-lang en writes video.en.vtt, which is then converted to a plain [HH:MM:SS] line format in captions.txt. No transcription cost when captions are present, which is most of YouTube.

Step 2 — slice (lib/ffmpeg-frames.sh)

ffmpeg does two things in parallel:

1. Audio extraction: -vn -ac 1 -ar 16000 -c:a aac -b:a 64k strips the audio to a clean mono 16 kHz m4a — the format ASR providers like best.
2. Frame sampling: density depends on --mode and --effort:

effort	hook mode (default)	retention/library mode
low	first 15 s @ 1 fps + body every ~15 s (cap 8)	40 frames total
medium	first 15 s @ 2 fps + body every ~5 s+ (cap 30)	100 frames total
high	first 30 s @ 2 fps + body every ~2 s+ (cap 120)	200 frames total

The hook preset is intentionally lopsided — dense around the opening (where decisions to keep watching are made), sparse after — instead of the uniform sampling most "video understanding" demos use. After extraction, every frame is renamed to HH-MM-SS.jpg so Claude can line each image up with the transcript without guessing offsets.

Step 3 — transcribe (lib/whisper-fallback.sh)

If captions.txt exists and is non-empty, it gets copied to transcript.txt and the transcribe step is skipped. Otherwise the script auto-selects a provider:

• ElevenLabs Scribe (scribe_v1 model) when ELEVENLABS_API_KEY is set. Word-level timestamps are bucketed into ~30 s segments so the output matches the YouTube caption shape: one [HH:MM:SS] line per segment.
• Groq Whisper (whisper-large-v3-turbo) when only GROQ_API_KEY is set. Native segment-level timestamps from the API.

Force a specific provider with WATCH_TRANSCRIBER=elevenlabs|groq. The provider that ran is recorded in transcript.txt.provider so the cost step can price it.

The script also accepts the common naming variants ELEVEN_LABS_KEY, ELEVEN_API_KEY, and XI_API_KEY so you don't have to rename whatever's already in your .env.

Step 4 — cost summary (in watch.sh)

Before exiting, watch.sh does the bookkeeping and writes cost.json. Three numbers come from disk (audio duration via ffprobe, frame count via ls, transcript word count via wc), the rest are estimates:

• Image tokens: frames × WATCH_TOKENS_PER_FRAME (default 1300, roughly correct for 1280×720).
• Transcript tokens: words × 1.3.
• Prompt overhead: 2,000 tokens.
• Output tokens: 1,200 (typical analysis report length; override with WATCH_ESTIMATED_OUTPUT_TOKENS).
• Claude $: (input × $15 + output × $75) / 1M — Opus 4.7 published rates, override with WATCH_RATE_CLAUDE_INPUT_PER_MTOK / WATCH_RATE_CLAUDE_OUTPUT_PER_MTOK.
• Transcription $: audio_minutes × per-min rate (Eleven default $0.0067/min, Groq default $0.0007/min, override with WATCH_RATE_ELEVEN_PER_MIN / WATCH_RATE_GROQ_PER_MIN).

The result is printed as a one-line COSTS_SUMMARY: block on stdout and saved as cost.json in the work dir. SKILL.md instructs Claude to read that JSON and surface the numbers in any report it generates, so you always see what the run cost.

Step 5 — Claude reads the work dir

The Bash side stops once everything is on disk. Claude then opens every frame in /tmp/watch/<slug>/frames/, reads transcript.txt and cost.json, applies the prompt at prompts/<mode>-analysis.md, and produces the report. Because the frames are timestamped in their filenames and the transcript is [HH:MM:SS] line–shaped, Claude can cite exact moments without needing any extra alignment metadata.

Output layout

/tmp/watch/<sha10-of-url>/
├── video.mp4                  # raw download (kept for re-runs / cache hits)
├── audio.m4a                  # 16 kHz mono, used by ASR
├── captions.txt               # native subs, if any
├── transcript.txt             # final timestamped transcript Claude reads
├── transcript.txt.provider    # one of: youtube-captions / elevenlabs-scribe / groq-whisper / none
├── cost.json                  # full cost breakdown (see above)
└── frames/
    ├── 00-00-00.jpg
    ├── 00-00-01.jpg
    └── …

Cost — what to actually expect

For a typical 30-minute YouTube video at default medium effort:

• yt-dlp + ffmpeg extraction: free (local CPU only).
• Native captions: free.
• ElevenLabs Scribe fallback: ~$0.20 for the 30 min.
• Groq Whisper fallback: ~$0.02 for the 30 min (often $0 under their free tier).
• Claude analysis: ~$1 on Opus 4.7.

Hook mode tops out around 45 frames at medium effort, so it's cheaper than the retention/library modes which sample 100 frames uniformly.

Flags

/watch <url-or-file>
  [--mode hook|retention|library]   # default: hook
  [--effort low|medium|high]        # default: medium
  [--start MM:SS] [--end MM:SS]     # trim before extraction (retention/library only)
  [--batch <csv>]                   # one URL per line; runs each through the same mode/effort