Qwen3-VL-Embedding & Qwen3-VL-Reranker

Two-stage video search: embed videos with Qwen3-VL-Embedding into a PostgreSQL

• pgvector database, then retrieve the top-k matches and re-rank them with Qwen3-VL-Reranker for high-precision results.

---

Install

git clone https://github.com/QwenLM/Qwen3-VL-Embedding.git
cd Qwen3-VL-Embedding
bash scripts/setup_environment.sh

Download the models to ./models/:

uv pip install huggingface-hub
huggingface-cli download Qwen/Qwen3-VL-Embedding-2B --local-dir ./models/Qwen3-VL-Embedding-2B
huggingface-cli download Qwen/Qwen3-VL-Reranker-2B  --local-dir ./models/Qwen3-VL-Reranker-2B

Start PostgreSQL (required for the demo):

docker compose up -d postgres

---

Usage

Add videos (ingest)

Point --video at a file or directory. Videos are automatically sliced into overlapping 20 s chunks (hop = 15 s) and downsampled so each chunk fits the model's token budget. Each chunk becomes its own searchable row in PostgreSQL.

# Ingest all videos in a directory (e.g. the UK-Road-DashCam clips used below)
python main.py ingest --video dataset/dashcam --rechunk

# Ingest a single video file
python main.py ingest --video path/to/clip.mp4

Flags worth knowing:

flag	default	effect
--rechunk	off	re-encode chunks from scratch (also drops the table)
--batch-size N	4	chunks per forward pass — lower if you OOM (eager attn is O(L²))

Search (query)

# Top-8 recall, top-5 re-rank
python main.py query "driving through a UK roundabout" -k 8 --topk-rerank 5

# Narrower: top-3 recall, top-3 re-rank
python main.py query "car stopped at a red traffic light in rain" -k 3 --topk-rerank 3

Output includes the original video path, exact time range ([start s – end s]), chunk index, and the re-ranker relevance score.

Example session

With 4 dashcam clips from aap9002/UK-Road-DashCam (MIT, ~4.4 GB total) placed under dataset/dashcam/:

# 1. Start PostgreSQL
docker compose up -d postgres

# 2. Ingest: 4 clips → 48 chunks (12 per clip), embedded in batches, inserted into PostgreSQL
python main.py ingest --video dataset/dashcam --rechunk
# Found 4 source video(s) to ingest:
#   - dataset/dashcam/241220_125301_002_FH.MP4
#   - dataset/dashcam/241220_125601_003_FH.MP4
#   - dataset/dashcam/241220_125901_004_FH.MP4
#   - dataset/dashcam/241220_130201_005_FH.MP4
# Total chunks to embed: 48 (window=20.0s, hop=15.0s, scale=512x288@5fps)
# Inserted 48 chunk row(s) into 'videos'.

# 3. Search
python main.py query "driving through a UK roundabout" -k 8 --topk-rerank 5
# Rank 1: 241220_125601_003_FH.MP4 [45.0-65.0s, chunk 3]   (Relevance Score: 0.6641)
# Rank 2: 241220_125601_003_FH.MP4 [165.0-180.2s, chunk 11] (0.6484)
# Rank 3: 241220_125901_004_FH.MP4 [30.0-50.0s, chunk 2]   (0.6484)

python main.py query "car stopped at a red traffic light in rain" -k 6 --topk-rerank 3
# Rank 1: 241220_125601_003_FH.MP4 [0.0-20.0s, chunk 0]   (0.4707)
# Rank 2: 241220_125301_002_FH.MP4 [45.0-65.0s, chunk 3]  (0.4023)
# Rank 3: 241220_125301_002_FH.MP4 [75.0-95.0s, chunk 5]  (0.3594)

Benchmark

Measure ingestion and query throughput with pyinstrument:

python benchmark.py --batch-size 4 --num-queries 20

Reports video-seconds ingested per minute, queries per minute, and saves per-stage HTML profiles to benchmark_reports/.

---

Citation

@article{qwen3vlembedding,
  title={Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking},
  author={Li, Mingxin and Zhang, Yanzhao and Long, Dingkun and Chen, Keqin and Song, Sibo and Bai, Shuai and Yang, Zhibo and Xie, Pengjun and Yang, An and Liu, Dayiheng and Zhou, Jingren and Lin, Junyang},
  journal={arXiv},
  year={2026}
}