Natural-language search over local video files. Fully local: no cloud, no subtitles, no audio. Frames only.
vidgrep oneshot ~/Videos "a dog jumping into a lake" -k 5brew install genkio/tap/vidgrepFirst run downloads the model weights (~2 GB) to ~/.cache. Optional: brew install mpv to jump straight to search results.
# index + cut in one go, one video at a time: clips appear as each video finishes
# (-k = clips per video; for the global top-k afterwards, run vidgrep cut - index is already built)
vidgrep oneshot ~/Videos/trips/ "sunset over water" -k 5
# several descriptions in one pass, each with its own output folder
# (indexing dominates the cost, extra descriptions are nearly free)
vidgrep oneshot ~/Videos "a dog jumping into a lake" ./dog "sunset over water" ./sunset
# long unattended runs: bad files are skipped and listed at the end, re-run to retry;
# caffeinate keeps the mac awake
caffeinate -i vidgrep oneshot ~/Videos "a dog jumping into a lake" ./dog
# index one file, or a folder (recursive) - slow, one-time, resumable
vidgrep index ~/Videos/
# search - instant
vidgrep search "two people kissing in a coffee shop"
vidgrep search "a dog running on a beach" -k 20
# cut top results into ./output/*.mp4 (or several descriptions, each with its own folder)
vidgrep cut "a dog running on a beach" -k 5 --pad 1
vidgrep cut "a dog running on a beach" ./dog "sunset over water" ./sunset
# web UI: search box, click to play, save clip, fullscreen
vidgrep serve # then open the printed URLvidgrep serve starts a local web server: type a description, get a grid of matching
clips you can play, download, or watch back-to-back in an overlay player. It binds 0.0.0.0 by default so you
can reach it from your phone over LAN or Tailscale (--host 127.0.0.1 for local only,
--port to change the port). Clips and thumbnails are generated by ffmpeg on demand and
cached under ~/.vidgrep/serve-cache. Works torch-free with --encoder too.
Results are grouped by source video, showing the top -k clips per video (default 5); a
"more" button under a group loads the rest of that video's matches on demand. --all shows
every match per video with no cap (heavier on a large index). A clip's "Play all" button
opens a full-width overlay player that auto-continues to the next result when each clip
ends, crossing video groups, so you can watch every match hands-free.
Note:
0.0.0.0means anyone who can reach the port on your network can browse your library. There's no authentication; keep it to trusted networks (Tailscale) or use--host 127.0.0.1.
Indexing needs PyTorch (Apple Silicon / CUDA). Cutting only needs to turn your query into a vector, so it can run anywhere, including an Intel Mac, once you export a small encoder. Index on the capable machine, then move two things to the other machine:
# on the indexing machine (once):
vidgrep export-encoder --out ./encoder # writes an ONNX text encoder (~1.4 GB)
# copy index.db + the encoder/ folder + your videos to the other machine, then:
vidgrep cut "sunset over water" ./out --db index.db --encoder ./encoder --videos ~/Movies--encoder runs the query through ONNX Runtime instead of PyTorch. --videos remaps the
indexed source paths to local files matched by filename, so absolute paths can differ
between machines. On Intel macOS, brew install gives you exactly this torch-free subset.
Search output: score (cosine, ~0.3 = strong hit, rank matters not the number), file, time range, ready-to-paste mpv command.
Re-running vidgrep index skips already-indexed files (re-indexes if the file changed).
indexsplits each video into shots (scene-cut detection), grabs one keyframe per shot, embeds it with CLIP (image encoder), stores vector +{path, start, end}in~/.vidgrep/index.db(sqlite-vec).searchembeds your phrase with CLIP (text encoder, same vector space), runs nearest-neighbor over the stored vectors, prints timestamps.cutruns the same search, then ffmpeg cuts each hit into a standalone clip.
No training anywhere. CLIP arrives pre-trained.
What does -k mean? Number of results returned (top-K nearest matches). Default 10. For oneshot it's per video.
Can I search in other languages? No, English only: the default model was trained on English captions. For multilingual queries set MODEL_NAME = "xlm-roberta-base-ViT-B-32", PRETRAINED = "laion5b_s13b_b90k" in vidgrep/common.py and re-index.
How do I start over, or index a different set of videos? The whole index is one file. Delete ~/.vidgrep/index.db to start fresh, or keep collections side by side with --db:
vidgrep index ~/Videos/trips/ --db trips.db
vidgrep search "sunset over water" --db trips.dbWhere do clips go? ./output in the current directory by default, padded by 0.5 s of context (--pad to change). Both cut and oneshot take an output folder right after each description.
Constants at the top of vidgrep/index.py / vidgrep/common.py:
MODEL_NAME/PRETRAINED/EMBED_DIM: the CLIP model. Any open_clip model works, includinghf-hub:<repo>checkpoints; a plainViT-B-32/laion2b_s34b_b79k/512indexes fastest with weaker results. Each index is locked to the model that built it; changing models requires a fresh re-index (delete~/.vidgrep/index.dbor use a separate--db).MAX_UNIT_S/SPLIT_STEP_S: long shots get one sample per 10 s.FALLBACK_WINDOW_S: window size for footage with no scene cuts (e.g. GoPro).
Requires uv and ffmpeg.
git clone https://github.com/genkio/vidgrep && cd vidgrep
uv sync --extra index # --extra index pulls torch + the indexing stack
uv run vidgrep search "..."
uv run --extra index vidgrep serve --db ~/.vidgrep/index.db # run the web UI from sourceuv sync alone installs the torch-free core (search/cut via an exported encoder). The
index extra adds PyTorch, open_clip, OpenCV, and scene detection for indexing and
export-encoder.
- face tagging (InsightFace) -> filter by who is in the shot
- VLM re-rank of top candidates (Qwen-VL via mlx)
- web UI over the same search fn
MIT