Turn any video into a self-contained, replayable slide deck — every frame and quote deep-linked back to the most relevant moment in the source.
A portable agent skill (Claude Code & other agent runtimes) that watches the video for you: it transcribes, outlines with cited evidence, curates the key frames, and renders a slide deck or navigable notes you can click straight back into the video.
The deck is just one view. Cuepoint's real output is a machine-readable outline ↔ transcript ↔ frames map of the video — a video-understanding layer that other agents and pipelines can consume to reason about footage without watching it.
English | 中文
▶ Live demos, real decks across four visual forms — slide-driven lecture · screen-recording course · architecture documentary · car review. Click any timestamp badge to jump into the source video.
A good technical video is dense — a 40-minute lecture is a slide deck, a transcript, and a dozen "look at this frame" moments, all trapped in a timeline you have to scrub. Skimming it means dragging the playhead; citing it means screenshotting by hand; turning it into notes means pausing every ten seconds.
Cuepoint collapses that into one pass. Point it at a URL or a file and it produces a deck (or notes) where:
- every page is a real point the speaker made, backed by a verbatim quote from the transcript — no hallucinated summaries;
- every frame is the right frame for that point, pulled from the video and de-duplicated, not a random thumbnail;
- every timestamp badge deep-links back to that exact second in the original video.
It is not a captioning tool and not a summarizer bolted onto YouTube. It builds a video index document — a durable, self-contained, machine-readable mapping of outline ↔ transcript ↔ frames that other agents and pipelines can consume to reason about a video without decoding it — and renders slides and notes as two human-facing views of that document.
- 📥 Any source — YouTube, Bilibili (single video and multi-part
?p=collections), or a local file (file://playback with in-page seeking). - 🗣️ Subtitles or ASR — uses existing captions when present (no API key needed); otherwise falls back to speech-to-text in three pluggable backend families (whisper-style transcription APIs, chat/base64-audio APIs, or fully-local FunASR).
- 🧭 Two-axis understanding — classifies visual form (slide-driven / screen-recording / talking-head / cinematic) and genre, reading the scene-change curve, edge density, transcript, and platform metadata.
- 🔖 Evidence-cited outline — every outline node must quote a real line from the transcript; unverifiable nodes degrade to text rather than inventing content.
- 🖼️ Smart frame curation — targeted per-node extraction in a single ffmpeg decode pass, RGB sliding-window dedup (catches A-B-A cutaways), blur/text-density scoring, and contact sheets so the host reads one image per chapter instead of hundreds.
- 📄 A real deliverable, not just a render — exports
video_index.json+frames/: a self-contained, versioned, schema-validated index document. Slides and notes are downstream views; the document is the product. - 🎞️ Two renderers — a fixed 1920×1080 HTML slide deck (via the
frontend-slidesskill) and/or navigable markdown notes (scripts/notes.py, zero extra deps). - ⏱️ Deep-links everywhere — badges jump to
youtube.com/watch?v=…&t=,bilibili.com/video/BV…?p=n&t=, or an embedded<video>overlay for local files. - 📚 Long-video aware — videos over 30 min are outlined chapter-by-chapter, then merged globally.
- ♻️ Resumable — every intermediate artifact lives under
.work/; each stage skips itself if its output is newer than its inputs (--forceto re-run). - 🌐 Degrades gracefully — a text-only (non-multimodal) host still produces the full index + notes; only the slide view needs an image-reading model.
- 🪶 Near-zero dependencies — stdlib-only Python; the only required binaries are
ffmpeg/ffprobeandyt-dlp. No PIL, OpenCV, or PaddleOCR — all image work goes through ffmpeg.
Deterministic scripts do the heavy, reproducible work; the host agent does the semantic judgment. They hand off through structured artifacts — one script's stdout feeds the agent's next decision.
YouTube · Bilibili · local file
│
┌───────▼───────┐
│ fetch.py │ yt-dlp → ~360p proxy stream
└───────┬───────┘ + subtitles · chapters · heatmap · danmaku · metadata
│
┌──────────────┴──────────────┐
▼ ▼
┌───────────────┐ ┌───────────────┐
│ transcribe.py │ │ signals.py │
│ subtitles/ASR │ │ one ffmpeg pass
└───────┬───────┘ │ → scene-score timeline
│ └───────┬───────┘
└──────────────┬───────────────┘
▼
┌─────────────────┐
│ host agent │ axis A: visual form axis B: genre
│ (analysis) │ outline where every node cites a verbatim quote
└────────┬────────┘
▼
┌───────────────┐
│ frames.py │ per-node frames · dedup · blur/text scoring
└───────┬───────┘ · contact sheets → host picks the best
▼
╔══════════════════════════════╗
║ video_index.json ║ the machine-readable contract —
║ + frames/ ║ outline ↔ transcript ↔ frames
╚═══════════════╤══════════════╝ (self-contained · versioned · schema-checked)
┌────────────────┼────────────────┐
▼ ▼ ▼
┌─────────────────┐ ┌───────────┐ ┌────────────────────┐
│ frontend-slides │ │ notes.py │ │ your agent / RAG │
│ HTML deck │ │ markdown │ │ search · tooling │
└────────┬────────┘ └─────┬─────┘ └────────────────────┘
└───────┬────────┘
▼
slides & notes deep-link back to the exact second · agents read the JSON as-is
The pipeline is zero-interruption: analysis runs start-to-finish without asking you anything. There is exactly one interaction point — after the index document is exported, before rendering — where the skill asks for output form, length, and language.
| Requirement | Why | Install |
|---|---|---|
| ffmpeg / ffprobe ≥ 5.1 | proxy stream, scene signals, frame extraction | macOS: brew install ffmpeg · Linux: distro package manager |
| yt-dlp ≥ 2026.7 | fetching online video + subtitles | macOS: brew install yt-dlp · Linux: pipx install yt-dlp |
| node or deno | only for YouTube (solves its JS challenge) | brew install node, or your distro / deno installer |
| tesseract (optional) | text-density frame scoring (falls back to an edge-density proxy if absent) | brew install tesseract |
Platform: macOS (Apple Silicon / Intel) and Linux (arm64 / x86_64). Native Windows is not supported — use WSL2 (which reports as Linux and runs fine).
Run the built-in precheck any time to see exactly what's missing, with platform-aware install hints:
python3 scripts/setup.pyCuepoint is an agent skill, not a standalone CLI — a host agent invokes it. Drop the repo into your agent's skills directory and the skill triggers on its description:
- Claude Code / Desktop — place the folder in your skills directory.
- Codex —
~/.agents/skills/cuepoint/(or a repo-local.agents/skills/). - OpenClaw —
~/.agents/skills/cuepoint/;SKILL.mdmust be a real copy (the symlink-escape guard rejects links), whilescripts//schemas//assets/may be symlinked.
Then just ask your agent, e.g. "turn this video into slides: <url>".
The slide output is rendered through the separate frontend-slides skill (a zero-dependency, anti-"AI-slop" 1920×1080 deck generator). Install it if you want decks. Without it, Cuepoint still produces the full video_index.json + navigable markdown notes — you just skip the slide view.
Videos that already have subtitles need no API key. For the rest, configure one backend in ~/.config/video2slides/.env (mode 0600, never committed):
ASR_BACKEND=funasr # fully local, no key; or: groq | openai | api | mimo | qwen
# API families also read: ASR_API_BASE / ASR_API_KEY / ASR_MODEL
# funasr reads: FUNASR_VENV (its own isolated venv, kept out of this project)Once installed, you drive it in natural language through your agent:
"Make slides from this lecture: https://www.youtube.com/watch?v=…" "Turn
/path/to/talk.mp4into notes."
The skill runs the whole pipeline unattended, then asks once:
- Form — slides · markdown notes · both · index-document only (default: slides)
- Length — short (5–10) · medium (10–20) · long (20+) — a depth dial shared by both renderers
- Language — defaults to the video's language; override at will (evidence quotes always stay verbatim)
Output lands in ~/Desktop/video2slides/<title>_<date>/ by default: a self-contained folder (video_index.json, frames/, index.html and/or notes.md) you can move or share as-is.
The first-class deliverable is video_index.json + frames/ — a single JSON embedding the full transcript (with explicit timestamp_granularity), the outline tree, and the selected proxy-resolution frames (relative paths, dedup annotations). It is validated against schemas/video_index.schema.json and carries a schema_version so consumers can check compatibility.
This — not the slides — is what makes Cuepoint interesting. video_index.json is a compact, self-contained video-understanding layer: what is said (the full transcript), what matters (a hierarchical outline where every node cites a verbatim quote), and what it looks like (the curated frames), all cross-linked by timestamp. Hand it to another agent, a RAG index, a search pipeline, or your own tooling and it can answer “what happens in this video, where, and show me the frame” — without touching the original media or re-running a vision model. Slides and notes are just the first two consumers.
Everything downstream — slides, notes, or your own tooling — reads only this document. Re-render with a different style, length, or language at zero re-analysis cost; the analysis never runs twice.
Full design rationale: docs/cuepoint-spec-v0.5.md.
| Family | Presets | Timestamps | Notes |
|---|---|---|---|
| transcriptions | groq · openai · api |
native segment | whisper-style multipart upload; api is configured via ASR_API_BASE/ASR_API_KEY/ASR_MODEL |
| chat | mimo · qwen |
45 s chunk granularity | chat/completions with base64 audio; the outline/QA layer compensates for coarse timestamps |
| funasr | (local) | sentence-level | runs in its own venv via subprocess (FUNASR_VENV); works on arm64 / x86_64 Macs, no API key |
| none | — | — | subtitled videos only; anything needing ASR stops with a clear message |
Audio is chunked at silence midpoints before upload, with per-chunk failure isolation. Bilibili subtitles are AI tracks that require a login cookie (--cookies-from-browser chrome, or an exported cookies.txt on headless hosts).
frontend-slidesby @zarazhangrui — the slide renderer this skill hands off to.yt-dlpandffmpeg— the two binaries that do all the fetching and media work.- FunASR — the local speech-recognition backend.
MIT © 2026 caomaocao.
The demo decks linked above are built from third-party videos and are shown for illustration only; the copyright in that footage belongs to the original creators.