Feed it a tutorial video. Get a skill your coding agent can actually run.
No install, no upload. The demo runs the same extraction client-side with tesseract.js — your video never leaves your machine, because there is no server for it to go to.
skillcast vite-testing-tutorial.mp4read vite-testing-tutorial.mp4 — 18s, 5 scene changes, 5 commands
skill: set-up-a-vite-project (5 steps)
1. Set up a Vite project
$ npm create vite@latest my-app -- --template react-ts
2. Install dependencies
$ cd my-app
$ npm install
3. Add Vitest
$ npm install -D vitest @testing-library/react
4. Configure the test script
5. Run the tests
$ npm run test
wrote skill/.claude/skills/set-up-a-vite-project/SKILL.md
wrote skill/.cursor/rules/set-up-a-vite-project.mdc
wrote skill/AGENTS.md
Your agent now knows how to do the thing in the video.
A developer tutorial says two different things at once.
The narration carries intent — "now we'll add the test runner". The
screen carries the part you can actually run — npm install -D vitest @testing-library/react.
Every video-to-text tool reaches for the transcript, and the transcript is the half that loses the executable truth. Nobody types package names out loud.
skillcast reads the screen instead. Scene detection finds the moments the screen changed, OCR lifts the text off those frames, and narrow heuristics decide what was typed versus what was printed. The commands in the output are the literal characters that were on screen.
Here is the same tutorial step, both ways:
| Source | What you get |
|---|---|
| Narration | "install vitest and the testing library" |
| Screen | npm install -D vitest @testing-library/react |
Only one of those runs.
pip install git+https://github.com/obeskay/skillcastThe PyPI name
skillcastis taken by an unrelated project, so the package will publish asskillcast-cli— the command staysskillcast. Until it is on PyPI, install from git as above.
Requires ffmpeg and tesseract:
brew install ffmpeg tesseract # macOS
sudo apt install ffmpeg tesseract-ocr # Debian/UbuntuNo API key. No model download. The default path is fully offline and finishes a 15-minute screencast in seconds.
skillcast https://youtube.com/watch?v=... # paste a link
skillcast demo.mp4 # or a local file
skillcast demo.mp4 # all three formats into ./skill
skillcast demo.mp4 --target claude # just the Claude Code skill
skillcast demo.mp4 --dry-run # show what was read, write nothing
skillcast demo.mp4 --json # machine-readable
skillcast demo.mp4 --threshold 0.005 # find more steps in a subtle recordingA URL works anywhere a path does — YouTube, Vimeo, Loom, or a direct .mp4.
It needs yt-dlp (pip install yt-dlp) and
downloads at 720p, which is plenty for OCR and far quicker.
YouTube blocks anonymous downloads in waves. When it does, the error says so and gives you the way through rather than looking like a broken link:
skillcast "https://youtube.com/watch?v=..." --cookies-from-browser chromeThe browser demo can only fetch URLs that allow cross-origin reads, which YouTube does not — there it hands you the CLI command instead of failing silently.
| Target | Path | Loaded by |
|---|---|---|
claude |
.claude/skills/<name>/SKILL.md |
Claude Code |
cursor |
.cursor/rules/<name>.mdc |
Cursor |
agents |
AGENTS.md |
Codex, and anything that reads AGENTS.md |
Plus skill.json — the same content structured, if you want to render your own
format.
Generating a plausible-looking SKILL.md is easy, and that is the trap. OCR
misreads one flag, the file still looks fine, and your agent fails in a way
nobody traces back to the video.
So nothing ships unverified:
| Check | Catches |
|---|---|
| Structure | frontmatter a loader would actually reject |
| Syntax | commands that do not parse as shell |
| Dashes | --template misread as —template, which no CLI accepts |
| Safety | rm -rf, curl | sh, device writes from a misread frame |
| Substance | filler text where extraction should have been |
--strict fails on warnings too. If shellcheck is installed it runs as well.
This proves the skill is loadable, runnable and not obviously dangerous. It does not prove the tutorial was right.
Measured on the fixture, degraded on purpose. Recall is exact-match against known ground truth:
| Recording | Recall |
|---|---|
| 720p terminal, large type | 5/5 |
| Re-encoded at 360p, crf 30 | 5/5 |
| Heavy noise added | 5/5 |
| IDE-style, 22px type amid code | 4/5 |
| IDE-style, 16px type amid code | 4/5 |
| IDE-style, 13px type | 2/5 |
| IDE-style, 11px type | 1/5 |
Compression barely matters; type size does. Below roughly 16px of on-screen text the OCR starts dropping commands, and no scene threshold rescues it.
If you control the recording: record at 1080p or above, and bump the terminal font. If you do not, expect to review the output rather than trust it.
Detection adapts on its own. A single command appearing in a screen already full of code scores 0.0001–0.002 — another order of magnitude below a bare terminal — so when the first pass comes back sparse for the video's length, skillcast retries ten times more sensitive, then falls back to interval sampling. It never silently returns one frame for a ten-minute tutorial.
It reads terminals and editors. That is the whole scope, and it is worth being blunt about it: a recording of hands, a phone video, slides or a talking head has nothing for it to lift. Point it at one and it says so rather than inventing steps — if not one candidate command uses a program it recognises, it refuses outright.
That check exists because it once did the wrong thing: fed a phone video, it
produced 'a and B as commands and reported "no problems found". A prompt
character followed by anything was passing straight through.
- It does not transcribe audio. The screen is the source of truth here; adding narration is planned, as enrichment rather than as the backbone.
- It does not verify that the commands work. It verifies they are well-formed.
- It does not handle videos with no visible text. If the tutorial is a talking head over slides, there is nothing to read.
- OCR is good, not perfect. Read the commands before running them — the output says so too.
Checked against live star counts, not memory:
| Project | Stars | Why it does not cover this |
|---|---|---|
| screenshot-to-code | 73.8k | UI screenshots → frontend code. Not workflows, not agent files. |
| OmniParser | 25.2k | Parses screens into UI elements. A perception layer, not an artifact. |
| screenpipe | 20.8k | Records your screen 24/7 for recall. Passive memory, not a compiler. |
| Agent-S | 12.1k | Operates a computer live. Does not consume a video offline. |
| OpenAdapt | 1.7k | Demonstration → desktop RPA. Replays clicks; brittle, and not agent-readable. |
| rulefy | 28 | Generates Cursor rules — from static code. Blind to video. |
| RuleForge | 9 | Same, for CLAUDE.md. Also blind to video. |
The two ends exist. Reading screens is solved; writing agent rules is solved. Nothing connects them.
video ─► scene detection ─► OCR ─► command heuristics ─► skill ─► verify ─► emit
One detail worth stealing if you build something similar: screencasts need a scene threshold roughly 30× lower than ordinary video. When a terminal advances, only the glyphs change and the dark background dominates the frame, so the difference score stays tiny. Measured on a terminal recording, real cuts score 0.017–0.024. The common default of 0.3–0.4 finds nothing — the tool looks broken on precisely the footage it was built for. skillcast defaults to 0.01.
python3 tests/make_fixture.py fixtures/ # build the synthetic tutorial
python3 -m unittest discover -s tests -t .The fixture is a screencast whose contents are known exactly, so "it extracted the right commands" is a measurement, not an opinion. The end-to-end tests assert full recall and zero invented commands against that ground truth.
Early. The extraction core is tested and the output has been loaded by a real agent, but it has been exercised on a narrow set of recordings so far. Issues with a link to a public video are the most useful thing you can file.
MIT — see LICENSE. Contributions welcome.