talkthru

talkthru.dev · npm · MIT

An automated feedback loop for any app. Screen-record your app on your phone and talk while you use it. Send it to your machine and your coding agent gets everything you said, attached to the screen you were looking at when you said it.

you:    *tapping through checkout* "this button is too small, I keep missing it"
                                  "and this error doesn't say what went wrong"

Claude: sees both screens, finds both components, fixes them.

Runs entirely on your machine. No accounts, no uploads, no API keys.

Tested on macOS with an iPhone. Any video with sound works — point the watcher at a folder.

---

Install

npx talkthru doctor --fix
claude mcp add --scope user talkthru -- npx talkthru-mcp

First command installs ffmpeg + whisper.cpp and downloads the speech model. Second tells Claude Code where to find your sessions — restart Claude Code afterwards so it picks it up.

---

Set up your phone

Once. Control Centre → + → Add a Control → Screen Recording. Then press and hold it and turn the microphone on.

1. tap +	2. Add a Control	3. pick Screen Recording
4. press and hold ◉	5. Microphone On	6. tap ◉ and talk

---

Use it

Leave this running:

talkthru watch

Then:

1. Record your app, talk as you go
2. Get the video onto your machine (AirDrop is easiest)
3. Ask Claude Code: "check the feedback in my last talkthru session and implement it"

Ready in about twenty seconds. Your original video is kept in ~/.talkthru/archive/.

Mute your app if it talks. Its narration lands in the transcript next to yours and whisper can't tell you apart.

---

Output

Your words, attached to the screen that was up when you said them:

## 00:34 · f11 — `frames/f11.jpg`
- [00:39] "this button is too small, I keep missing it"

## 00:52 · f13 — `frames/f13.jpg`
- [00:55] "and this error doesn't say what actually went wrong"

Plus the frames. A two-minute session is about 600 tokens.

---

Commands

talkthru watch                  # watch ~/Downloads for screen recordings
talkthru watch ~/some-folder    # watch your own folder, any video
talkthru process video.mp4      # process one file
talkthru list                   # recent sessions
talkthru show latest            # print one
talkthru doctor                 # check the setup
talkthru prune                  # delete old sessions

tt also works.

---

How it works

ffmpeg pulls the frames where your screen actually changed. whisper.cpp transcribes your voice locally, split on the real silence between sentences, so each thing you said lands on the screen you were looking at. An MCP server hands it to Claude Code.

Needs Node 20.11+.

---

Why

I built this because I test my own app on my phone every day and kept losing the feedback on the way back to my editor — screenshotting, cropping, trying to describe which screen I meant. Now I just record, talk, and Claude Code picks it up. Sharing it in case it saves you the same trip.