[CVPR 2026 Oral] 4D Primitive-Mâché

Glueing Primitives for Persistent 4D Scene Reconstruction

CVPR 2026 (Oral) · Project page

Installation

Requires an NVIDIA GPU (developed on an RTX 4090, CUDA 12.4 / PyTorch 2.6) and a conda-family package manager. micromamba is preferred; mamba/conda should also work.

bash install.sh

This creates the 4DPM environment and installs everything: the pinned Python deps, CUDA-matched PyTorch, lietorch, the two custom CUDA kernels, and the external models (sam2, Pi3) into third_party/. It then downloads the model checkpoints into checkpoints/ and writes config/paths.local.yaml.

To re-download just the checkpoints later: bash download_checkpoints.sh.

Model paths

The committed config/*.yaml are path-free. Machine-specific paths (checkpoints, Pi3 location) live in config/paths.local.yaml, which is overlaid onto the active config at load time. install.sh generates this file; see config/paths.local.yaml.example for the template.

Running

Activate the env first (micromamba activate 4DPM), then:

# Multi-object dataset
python run_pipeline.py --mode end2end --config ./config/franka_data.yaml --verbose

# Object-permanence demos
python run_pipeline.py --mode end2end --config ./config/drawer.yaml --verbose
python run_pipeline.py --mode end2end --config ./config/fridge.yaml  --verbose

--verbose streams visualizations to a running Rerun viewer.

By default the frontend runs silently; set silent_frontend: False in the yaml config to preview the primitive division of the video before processing.

Data

4DPM data can be accessed via this link

Evaluation

First generate the per-sequence results:

python -m evals.run_eval_sequences \
  --data-root PATH_TO_DATA \
  --config ./config/franka_data.yaml \
  --out-dir ./dump/franka_eval

Then score them and print the comparison tables:

python -m evals.eval_franka \
  --data-root PATH_TO_DATA \
  --payload-dir ./dump/franka_eval \
  --cam-id 1

Codebase organisation

• core/ the optimisation (Gauss-Newton, Hessian/Jacobians).
• frontend/ geometry and segmentation priors (Pi3, SAM 2, AllTracker).
• object_mapper/ motion extension and the components that enable 4D replay.

Citation

@inproceedings{Mazur:etal:CVPR2026,
    title={4D Primitive-Mâché: Glueing Primitives for Persistent 4D Scene Reconstruction},
    author={Kirill Mazur and Marwan Taher and Andrew Davison},
    booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2026},
}