• Overview
• What It Produces
• Installation
• Checkpoint
• Usage
• Dataset
• Acknowledgments
• Citation
• License

Face Anything is a unified feed-forward model for high-fidelity 4D face reconstruction and dense tracking from arbitrary image sequences. The key idea is canonical facial point prediction, a representation that assigns each pixel a normalized facial coordinate in a shared canonical space. This formulation transforms dense tracking and dynamic reconstruction into a single canonical reconstruction problem, producing temporally consistent geometry and reliable correspondences.

The teaser above is a single grand tour run: one orbit that morphs through every modality. A full run writes:

Face Anything needs a CUDA GPU. Tested with Python 3.11 / PyTorch 2.9 (CUDA 12.8).

One-line setup (creates the conda env, installs everything, and downloads the checkpoint):

git clone https://github.com/kocasariumut/FaceAnything.git
cd FaceAnything

bash install.sh

Or do it manually:

git clone https://github.com/kocasariumut/FaceAnything.git
cd FaceAnything

conda create -n faceanything python=3.11 -y
conda activate faceanything

# install PyTorch matching your CUDA, then the rest
pip install torch==2.9.0 torchvision==0.24.0 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt

# install this package (exposes `faceanything` and `depth_anything_3`)
pip install -e .

pip install -e . is optional, since run_inference.py adds src/ to the path automatically, so once the dependencies are installed you can run it directly.

The architecture/config of the backbone is loaded from the public HuggingFace model depth-anything/DA3-GIANT-1.1 (downloaded automatically on first run; its weights are then overwritten by our checkpoint). To use a local cache instead of downloading, set export HF_HOME=/path/to/hf_cache.

Download the released checkpoint.pt (~15 GB) and place it at checkpoints/checkpoint.pt (where it is loaded by default). install.sh fetches it automatically; to do it manually, use either option below.

Option 1: Google Drive

pip install gdown
gdown --fuzzy "https://drive.google.com/file/d/1PdQQxzm-tU50RmJhgeoMCYVRlEiW3f8p/view?usp=sharing" \
    -O checkpoints/checkpoint.pt

Option 2: Hugging Face (UmutKocasari/FaceAnything)

huggingface-cli download UmutKocasari/FaceAnything checkpoint.pt --local-dir checkpoints

(The Hugging Face model repository becomes public when the code is released.)

Use --checkpoint /path/to/checkpoint.pt to load it from elsewhere.

python run_inference.py --input <INPUT> --output <OUT_DIR>

<INPUT> can be a single image, a folder of images, or a video file (.mp4/.mov/.avi/.mkv/...). For a single image, the reconstruction is rendered as a turntable orbit.

# image folder
python run_inference.py --input path/to/images --output output/demo
# video
python run_inference.py --input clip.mp4 --output output/clip
# single image
python run_inference.py --input face.jpg --output output/face

Backgrounds are removed automatically. If you don't provide masks, Face Anything runs Robust Video Matting to segment the foreground and saves the generated masks to <OUT_DIR>/masks. You can also supply your own masks with --mask-dir /path/to/masks, or disable background removal entirely with --no-background-removal (use the full frame).

Everything is produced by default. To produce a subset, pass a comma-separated list to --outputs (choices: pointcloud, depth, normals, canonical, tracks, grandtour, ply, raw):

python run_inference.py --input clip.mp4 --output output/clip \
    --outputs pointcloud,canonical,grandtour

--process-mode controls how frames are fed to the model, which trades off surface detail against temporal/3D consistency:

• all-at-once (default): all frames are processed jointly, giving more 3D-consistent results across the sequence but less detailed surfaces.
• one-by-one: each frame is processed independently, giving more detailed surfaces but less 3D-consistent results across frames. It also uses less memory, so it pairs well with a larger --process-res for higher-resolution, more detailed outputs.

We release the canonical maps for the selected timestamps used in Face Anything. To request access to the FaceAnything NeRSemble dataset, please fill out this form.

Face Anything builds on Depth-Anything-3. We also thank the authors of FLAME, COLMAP, and Robust Video Matting for their open-source work, which we use in training data preparation and background segmentation.

@article{kocasari2026face,
  title={Face Anything: 4D Face Reconstruction from Any Image Sequence},
  author={Kocasari, Umut and Giebenhain, Simon and Shaw, Richard and Nie{\ss}ner, Matthias},
  journal={arXiv preprint arXiv:2604.19702},
  year={2026}
}

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.