AI Matching

AI Matching

AI matching is an optional, additional pass on top of the classic comparison. It uses neural image embeddings (DINOv2 via ONNX Runtime) to judge whether two frames show the same content, even when the pixels differ a lot. Everything runs locally on your machine; nothing is uploaded anywhere.

It consists of two independent features:

1. AI matching (whole videos and images) - finds transformed copies the classic methods miss: cropped, zoomed, mirrored or heavily re-edited versions.
2. AI partial detection - finds trimmed cuts and clips inside longer recordings by matching keyframes visually. Unlike the audio-based partial clip detection, it needs no audio track, so it also covers silent, muted and re-dubbed copies.

The easiest way to use both is the AI scan profile on the main screen. You can also enable them individually: AI matching under Settings > Matching, AI partial detection under Settings > Partial clips.

One-time component download

The AI pass needs two components that are not bundled with VDF:

• the ONNX Runtime native library (official Microsoft release, version pinned by VDF)
• the DINOv2-small embedding model (Apache-2.0 licensed, integrity-checked against a pinned hash)

Together they are roughly a 100 MB one-time download, stored next to VDF's database. Use the Download button under Settings > Matching > AI components, or just start an AI scan: if the components are missing, VDF offers to download them first. As with FFmpeg, nothing AI-related is included in normal releases; you only get it if you opt in.

How it works

Whole-video matching

For every sampled frame position (the same positions the classic comparison uses) VDF computes an embedding, a compact numerical description of what the frame shows. During comparison, a pair that fails the classic grayscale/pHash check is still reported as a duplicate when the average embedding similarity of its sampled positions reaches the AI similarity threshold (default 94%). Such pairs get an AI chip in the results list, so you always know why a group was formed.

Because it only adds matches, the AI pass never hides anything the classic comparison would have found.

AI partial detection

For clip detection, VDF samples keyframes densely across each video and looks for runs of frame pairs that match with a consistent time offset. A frame pair counts as a hit when its embedding similarity reaches the AI frame hit threshold (default 89%), and at least 4 hits must agree on the same offset before two videos are paired. Keyframe sampling runs as its own scan phase, and the embeddings are cached in the database, so subsequent scans are fast.

Matched pairs show the clip's position inside the longer video, just like audio-based partial clip detection.

Settings

Setting	Default	Description
AI matching (additional pass)	off	Enables whole-video/image AI matching.
AI similarity threshold	94%	How similar the embeddings must be. Lower to ~92% to catch more aggressively edited copies, at a slightly higher false-positive risk.
Detect partial duplicates visually (AI)	off	Enables AI clip detection.
AI frame hit threshold	89%	Minimum similarity for a single keyframe pair to count as a hit. Calibrated for re-encoded cuts; raise it if unrelated videos get paired.

The CLI exposes the same switches: --ai-matching, --ai-percent, --ai-partial and --ai-partial-hit-percent.

Performance

• Whole-video AI matching adds roughly 50 ms per file to a scan (embedding extraction happens in the same FFmpeg pass that grabs the regular frames).
• AI partial detection is more expensive because it decodes keyframes across the whole timeline; expect the first scan with it enabled to take noticeably longer. Everything is cached, so this cost is paid once per file.
• All inference runs on the CPU. No GPU is required.

AI partial detection vs. audio partial clip detection

	AI partial (visual)	Audio fingerprinting
Needs an audio track	no	yes
Works on muted/re-dubbed copies	yes	no
Works on visually re-edited clips (color grading, crops)	yes	yes (visuals are irrelevant)
First-scan cost	decodes keyframes	decodes the full audio track
Component download	~100 MB one-time	none

The two complement each other; the Deep clean profile runs both.

Troubleshooting

Unrelated videos are paired by the AI pass Raise the AI similarity threshold (whole-video matches) or the AI frame hit threshold (clip matches). Visually similar but genuinely different content, such as two recordings of the same scene, is the typical cause.

An edited copy is still not found Lower the AI similarity threshold to ~92%. If the copy is also trimmed, make sure AI partial detection is enabled, since whole-video matching compares matching timeline positions and cannot bridge large time shifts on its own.

The components fail to download The runtime comes from the official ONNX Runtime GitHub release and the model from a VDF release mirror (with an upstream fallback), so a firewall or proxy blocking GitHub is the usual culprit. The files land in the ai folder next to VDF's database; you can copy that folder from another machine if needed.