VQFace: Tokenized Facial Representation
for Affect Understanding

Minh Tran¹ · Maksim Siniukov¹ · Zhangyu Jin¹ . Mohammad Soleymani^1
1University of Southern California

Model weights

Model weights and available at:
https://huggingface.co/datasets/minhtran886688/vqface/tree/main

Environment

The environment requires:

• PyTorch
• pip install vector-quantize-pytorch
• EMOCA
  Note: for emoca please repalce the following files in EMOCA library:

File (this repo)	Target (EMOCA repo)
emoca_code/renderer.py	emoca/gdl_apps/EMOCA/demos/Renderer.py
emoca_code/DECA.py	emoca/gdl/models/DECA.py

Codebase Summary

1. Train VQ-VAE:
   python train.py

2. Run downstream evaluations:
   python analyze_<dataset name>.py

3. Visualize token dictionary:
   python render_all.py

1. Training the VQ-VAE Encoder + Codebook

Script: python train.py

Pretrain VQ-VAE model from scratch on AffectNet.

2. Downstream Analysis and Metrics

Scripts: python analyze_<dataset>.py Examples:

• analyze_affectnet.py
• analyze_chalearn.py

3. Visualizing the Codebook Tokens

Script: python render_all.py

Generates:

• Visual reconstructions of all learned quantized tokens
• Shows interpretability & qualitative analysis

Project Structure

vqface/code/
│
├── train.py # Main VQ-VAE training
├── analyze_affectnet.py # Downstream evaluation & metrics for affectnet
....
├── analyze_cremad.py # Downstream evaluation & metrics for cremad
├── render_all.py # Token visualization

Citing

Please cite our work:

@article{tran2024vqface,
  title={Discrete Facial Encoding: A Framework for Data-driven Facial Display Discovery},
  author={Tran, Minh and Siniukov, Maksim and Others},
  journal={WACV},
  year={2026}
}

License

VQFace is available under an USC Research License.

3rd-party components may have their respective licenses. Please contact their respective authors to obtain licenses.

Acknowledgments

Research was sponsored by the Army Research Office and was accomplished under Cooperative Agreement Number W911NF-25-2-0040. Work was also in part supported by the National Science Foundation under Grant IIS-2211550 and the National Institute of Mental Health of the National Institutes of Health under Award Number R61MH135407. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office, NSF, NIH, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.