Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/bug-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ body:
attributes:
label: Affected capability
options:
- Dialogue search
- Speech search
- Scene search
- Action search (videoprism)
- Action and motion search
- Actor matching
- Multiple capabilities
- Not capability-specific or unsure
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/feature-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ body:
attributes:
label: Related capability
options:
- Dialogue search
- Speech search
- Scene search
- Action search (videoprism)
- Action and motion search
- Actor matching
- Multiple capabilities
- Not capability-specific or unsure
Expand Down
2 changes: 1 addition & 1 deletion .github/release-intro.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## Download VidXP

{release_notice}VidXP turns video into searchable dialogue, sounds, scenes, actions, people, and
{release_notice}VidXP turns video into searchable speech, sounds, scenes, actions, people, and
inspectable evidence. Choose the desktop app for the guided local setup, or use
the Python package and containers for command-line and server deployments.

Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
/venv
/chroma_data
/benchmark_runs/
/benchmarks/codex-mcp/.promptfoo/
/benchmarks/codex-mcp/results/
/benchmarks/codex-mcp/workspace/
/audio.wav
/video.mp4
/dist/
Expand Down
45 changes: 37 additions & 8 deletions INSTALLATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Approximate model downloads are:

| Feature | Download |
|---|---:|
| Dialogue search | 2.63 GiB |
| Speech search | 2.63 GiB |
| Sound event search | 0.91 GiB |
| Scene search | 1.43 GiB |
| Action search | 0.93 GiB |
Expand Down Expand Up @@ -200,8 +200,8 @@ asking for confirmation. Download only selected features when preferred:

```bash
vidxp prepare --modalities scene
vidxp prepare --modalities dialogue,actor
vidxp prepare --modalities videoprism # action search
vidxp prepare --modalities speech,actor
vidxp prepare --modalities action
vidxp prepare --modalities sound # music and environmental sounds
```

Expand Down Expand Up @@ -232,14 +232,14 @@ vidxp index create <media-id>
# Find a visual scene
vidxp search scene "a yellow taxi on a city street"

# Find an action or event (`videoprism` is the CLI name for action search)
vidxp search videoprism "a person opens a door and walks outside"
# Find an action or motion across multiple frames
vidxp search action "a person opens a door and walks outside"

# Find music or an environmental sound
vidxp search sound "an alarm ringing"

# Find something that was said
vidxp search dialogue "the bread just came out of the oven"
vidxp search speech "the bread just came out of the oven"
```

Add `--media-id <media-id>` to a search command to restrict results to one
Expand Down Expand Up @@ -313,6 +313,30 @@ clients, or public access, use the supported server deployment instead.
See [Local API and MCP server](docs/local-api.md) for authentication, uploads,
and sharing behavior.

## Optional local grounded answers

VidXP search does not require a language model. To let CLI, HTTP, or MCP
queries plan searches and draft grounded answers locally, enable **Local
grounded answers** in VidXP Desktop setup. Desktop checks for a compatible
loopback Ollama service, asks before installing Ollama through the supported
Windows or macOS package manager when needed, and explicitly downloads the
approved Qwen3.5 4B model. Linux setup links to Ollama's official installation
instructions instead of running a privileged script.

This optional feature follows Ollama's platform floor: Windows 10 22H2 or
newer, or macOS 14 or newer. VidXP Desktop itself can still run without local
grounded answers on older supported systems.

The model is an additional approximately 3.4 GB download and has no per-run
API charge or numbered hosted-model allowance. It uses local storage, memory,
compute time, and electricity. Desktop configures the private service address
for its browser, worker, API, Premiere, and generated MCP/Codex setup; there is
no URL field to fill in. A command-line-only installation remains available
for developers and custom deployments.

The complete setup and its current evidence limitations are documented under
[Enable local grounded answers](docs/local-api.md#enable-local-grounded-answers).

## Optional dependency extras

Most users should choose one of the package profiles above. The individual
Expand All @@ -321,10 +345,10 @@ assembling a custom installation:

| Extra | Adds |
|---|---|
| `dialogue` | Transcription, dialogue embeddings, and storage |
| `speech` | Transcription, speech embeddings, and storage |
| `sound` | Music and environmental-sound search and storage |
| `scene` | Scene search and storage |
| `videoprism` | Action search and storage |
| `action` | Multi-frame action and motion search plus storage |
| `actor` | Actor matching and storage |
| `all` | Every built-in search feature |
| `local-worker` | All search features plus local job processing |
Expand Down Expand Up @@ -404,6 +428,11 @@ to be indexed again. VidXP reports this instead of silently replacing a working
index. Prepare the required models, re-index the affected videos, and keep the
old repository until you have checked the replacement results.

The current public capability names are `scene`, `action`, `sound`, `speech`,
and `actor`. VidXP does not translate removed capability names.
If an older repository reports an incompatible index schema, rebuild it using
the current names.

## Data locations

VidXP stores data outside the current working directory:
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
</p>

<p align="center">
<strong>Dialogue search · Sound search · Scene search · Action search · Actor grouping</strong>
<strong>Speech search · Sound search · Scene search · Action search · Actor grouping</strong>
</p>

<p align="center">
Expand All @@ -39,7 +39,7 @@

VidXP makes one video—or an entire collection—searchable by meaning:

- **Dialogue search:** type what you remember someone saying and jump to the
- **Speech search:** type what you remember someone saying and jump to the
matching moments.
- **Scene search:** describe what appeared on screen and find the closest
visual matches.
Expand Down Expand Up @@ -122,7 +122,7 @@ See the [Coolify guide](docs/deployment/coolify.md) for the complete setup.
## What you can do today

- Build searchable libraries from individual videos or whole collections.
- Find dialogue, sound events, visual scenes, and multi-frame actions by description.
- Find speech, sound events, visual scenes, and multi-frame actions by description.
- Ask grounded questions and inspect the supporting boards, frames, or clips.
- Group recurring faces and render highlighted actor overlays.
- Keep personal, client, or project libraries separate.
Expand All @@ -144,13 +144,13 @@ vidxp index create <media-id>
vidxp search scene "a yellow taxi on a city street"

# Find an action or event
vidxp search videoprism "a person opens a door and walks outside"
vidxp search action "a person opens a door and walks outside"

# Find a sound event
vidxp search sound "a dog barking over traffic noise"

# Find something that was said
vidxp search dialogue "the bread just came out of the oven"
vidxp search speech "the bread just came out of the oven"
```

Results include the source video, timestamps, match score, and the evidence
Expand Down Expand Up @@ -208,7 +208,7 @@ approximately 3 GiB.

| Capability | Approximate model download |
|---|---:|
| Dialogue search | 2.64 GiB |
| Speech search | 2.64 GiB |
| Sound event search | 0.94 GiB |
| Scene search | 1.43 GiB |
| Action search | 0.93 GiB |
Expand Down
1 change: 1 addition & 0 deletions benchmarks/codex-mcp/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
engine-strict=true
Loading
Loading