QUART

Query-Aware Context Routing for Streaming Video Understanding.

This repository releases the router training code. Evaluation harness and pretrained checkpoint will follow.

Install

conda create -n quart python=3.10 -y
conda activate quart
pip install -r requirements.txt
pip install flash-attn==2.6.3 --no-build-isolation

Train

bash scripts/train_router.sh \
    --base_model    Qwen/Qwen3-VL-2B-Instruct \
    --dataset_jsonl <path/to/train.jsonl> \
    --output_dir    outputs/quart_router \
    --no_freeze_vision_tower

scripts/train_router.sh is a thin torchrun wrapper around quart/train/train_router.py. See the script's CLI for all options.

The training jsonl is built by quart/data/build_routing_views.py from per-event annotations, then optionally mixed with a natural-question source via quart/data/build_train_mix.py. Each row is one streaming view of one event with frame_labels[i] ∈ {0=silence, 1=short, 2=long} as the per-frame target.

The released QUART checkpoint is trained on nine public datasets:

• ActivityNet Captions
• Charades-STA
• Charades-Ego
• COIN
• DiDeMo
• ET-Instruct-164K
• TACoS
• TVQA
• YouCook2

Each dataset is governed by its own license; download them independently.

Repo layout

quart/
├── router/                 # router token IDs + RouterModel inference helper
├── train/                  # LM-head training (collator + train_router)
└── data/                   # routing-view data construction
configs/                    # YAML recipe + DeepSpeed ZeRO-3 template
scripts/                    # torchrun launcher

TODO

• Router training code
• Paper
• Pretrained router checkpoint (HuggingFace)
• Evaluation scripts

License

Apache 2.0.