CameraAnything: Refilming Videos with Arbitrary Camera Control (ECCV'26)

[Project Page] [Hugging Face]

Yixuan Li^1,2,*, Yanhong Zeng^2,3,*,†, Ka Leong Cheng², Jiayi Zhu², Hanlin Wang^2,5, Wen Wang^2,4, Yihao Meng^2,5,
Hao Ouyang², Qiuyu Wang², Yue Yu^2,5, ZiDong Wang¹, Yiyuan Zhang¹, Yujun Shen², Dahua Lin¹

(*Equal contribution †Project lead)

¹The Chinese University of Hong Kong, ²Ant Group, ³Tsinghua University, ⁴Zhejiang University, ⁵The Hong Kong University of Science and Technology.

Contact: liyixxxuan@gmail.com, zengyh1900@gmail.com

Strongly recommend seeing our demo page.

TL;DR

• What it is: The first unified framework for camera-controlled video editing that jointly controls extrinsic (viewpoint / trajectory / multi-shot cuts) and intrinsic (focal length / native resolution & aspect ratio) parameters in a single generation process.
• Built on: Wan2.1; we release inference and training code on this backbone.

🚀 Setup

• Clone & install:

git clone <REPO_URL>
cd camera_anything
pip install -e .

• CameraAnything checkpoint: Download from Hugging Face and place under models/ (default path used by the launcher: models/cameraanything.ckpt):

mkdir -p models
# huggingface-cli download liyixxxuan/cameraanything cameraanything.ckpt --local-dir models
# or manually move the file to models/cameraanything.ckpt

• Wan2.1 14B base weights: Needed for training / inference paths:

python download_wan2.1.py

🎥 Supported Trajectories / Tasks

We provide five mixed-control demo tasks (one launcher script each). Unless noted, camera parameters stay the same as the condition video:

• resolution_only: Extrinsics and focal unchanged; landscape → portrait (keep fx/fy).
• focal_only: Extrinsics unchanged; modify focal (HFOV).
• multi-shot-only_1: Modify extrinsics (multi-shot); focal unchanged.
• multi-shot-only_2: Modify extrinsics (another multi-shot trajectory); focal unchanged.
• multi-shot-focal-resolution: Modify extrinsics (multi-shot) and focal; landscape → portrait (keep fx/fy).

▶️ Inference

• Example inputs: Under examples/ — *.mp4 source videos, caption.json (captions / paths), camera.json (per-frame trajectories).
• Configure: Edit CKPT_PATH and OUTPUT_DIR in scripts/inference_scripts/inference_real_mixed_8gpu_common.sh.
• Run (case-parallel; one script per task):

bash scripts/inference_scripts/inference_real_mixed_resolution_only_8gpu.sh
bash scripts/inference_scripts/inference_real_mixed_focal_only_8gpu.sh
bash scripts/inference_scripts/inference_real_mixed_multi_shot_only_1_8gpu.sh
bash scripts/inference_scripts/inference_real_mixed_multi_shot_only_2_8gpu.sh
bash scripts/inference_scripts/inference_real_mixed_multi_shot_focal_resolution_8gpu.sh

• Outputs: {case}_{task}_source.mp4 / {case}_{task}_pred.mp4.

📷 Camera Visualization

Interactive 3D plot of source (blue) vs target (red) cameras for each mixed task (tools/vis_cam.py, aligned with inference.py):

# resolution_only — cond pose + cond hfov, portrait rectangle
python3 tools/vis_cam.py --case man_working --task resolution_only

# focal_only — cond pose + novel hfov, landscape
python3 tools/vis_cam.py --case man_working --task focal_only

# multi-shot-only — novel pose + cond hfov, landscape
python3 tools/vis_cam.py --case man_working --task multi-shot-only_1
python3 tools/vis_cam.py --case man_working --task multi-shot-only_2

# multi-shot-focal-resolution — novel pose + novel hfov, portrait
python3 tools/vis_cam.py --case man_working --task multi-shot-focal-resolution

Optional flags: --camera_json examples/camera.json, --stride 4, --base_xval 0.08, --cond_height 480 --cond_width 832.

🎬 Rendering (Dataset Construction)

[Rendering README]

Training data (CameraAnything-Dataset) is synthesized with Unreal Engine 5: multi-camera synchronized clips with controlled extrinsics (viewpoint / trajectory / multi-shot cuts) and intrinsics (focal length). Built on BEDLAM human animation assets.

• Pipeline: Under render/ — place SMPL-X humans in 3D scenes, sample camera trajectories (fixed / multi-shot / arc), render RGB, export per-frame camera GT (c2w + hfov), then rearrange into the training layout via render/rearrange_data.py.
• Details: Setup, human assets, scene assets, and one-click launchers (run_all_steps.sh / run_all_steps_focal.sh) — see the rendering README above.
• Sample: A single example case is provided in data_sample/ (focal_24 / scene1).
• Layout (paths used by scripts/train_scripts/train.sh / scripts/preprocess/preprocess.sh):

CameraAnything-Dataset
├── train
│   ├── focal_18.csv
│   ├── focal_24.csv
│   ├── focal_35.csv
│   ├── focal_50.csv
│   ├── focal_18_focal_18to50.csv          # focal-change metadata
│   ├── focal_24_focal_18to50.csv
│   ├── focal_35_focal_18to50.csv
│   ├── focal_50_focal_18to50.csv
│   ├── focal_18                           # fixed-focal clips
│   │   ├── scene1                         # one dynamic scene
│   │   │   ├── videos
│   │   │   │   ├── cam01.mp4
│   │   │   │   ├── ...
│   │   │   │   └── cam10.mp4              # cam01-05 single / cam06-10 multi-shot
│   │   │   └── cameras
│   │   │       └── camera_extrinsics.json # per-frame c2w + hfov
│   │   └── ...
│   ├── focal_18_focal_18to50              # same scenes, varying focal
│   │   ├── scene1
│   │   │   ├── videos
│   │   │   └── cameras
│   │   └── ...
│   ├── focal_24
│   ├── focal_24_focal_18to50
│   ├── ...
│   └── focal_50_focal_18to50
└── processed                              # caption tensors (train -> processed)
    ├── focal_18
    │   └── scene1
    │       └── videos
    │           └── cam01.mp4.tensors.pth  # prompt_emb from preprocess.sh
    └── ...

🏋️ Training

Expects the dataset layout from Rendering above (see data_sample/ for an example).

• Preprocess captions:

bash scripts/preprocess/preprocess.sh

• Train:

bash scripts/train_scripts/train.sh

🤗 Awesome Related Works

• ReCamMaster: Camera-controlled generative rendering from a single video.
• ReDirector: Creating any-length video retakes with Rotary Camera Encoding.
• TrajectoryCrafter: Redirecting camera trajectory for monocular videos via diffusion models.
• BEDLAM: A synthetic dataset of bodies exhibiting detailed lifelike animated motion.

🌟 Citation

Please leave us a star 🌟 and cite our paper if you find our work helpful.

@inproceedings{li2026cameraanything,
      title={CameraAnything: Refilming Videos with Arbitrary Camera Control},
      author={Li, Yixuan and Zeng, Yanhong and Cheng, Ka Leong and Zhu, Jiayi and Wang, Hanlin and Wang, Wen and Meng, Yihao and Ouyang, Hao and Wang, Qiuyu and Yu, Yue and Wang, ZiDong and Zhang, Yiyuan and Shen, Yujun and Lin, Dahua},
      booktitle={European Conference on Computer Vision (ECCV)},
      year={2026}
}