Heatmap Regression without Soft-Argmax for Facial Landmark Detection

Abstract

Facial landmark detection is an important task in computer vision with numerous applications, such as head pose estimation, expression analysis, face swapping, etc. Heatmap regression-based methods have been widely used to achieve state-of-the-art results in this task. These methods involve computing the argmax over the heatmaps to predict a landmark. Since argmax is not differentiable, these methods use a differentiable approximation, Soft-argmax, to enable end-to-end training on deep-nets. In this work, we revisit this long-standing choice of using Soft-argmax and demonstrate that it is not the only way to achieve strong performance. Instead, we propose an alternative training objective based on the classic structured prediction framework. Empirically, our method achieves state-of-the-art performance on three facial landmark benchmarks (WFLW, COFW, and 300W), converging $2.2\times$ faster during training while maintaining better/competitive accuracy.

Dependencies

You can set up the environment using the provided script.

bash scripts/tools/create_env.sh

Data preparation

• We follow the data preparation from STAR (CVPR 2024) as follow.
• The default ${image_dir} is set to datasets while the default ${annot_dir} is set to data.

# the dataset directory:
|-- ${image_dir}
    |-- WFLW
        | -- WFLW_images/
    |-- 300W/
        | -- afw/
        | -- helen/
        | -- ibug/
        | -- lfpw/
    |-- COFW/
        | -- train/
        | -- test/
|-- ${annot_dir}
    |-- data_WFLW
        |-- metas/
            |-- train.tsv
            |-- test.tsv
    |-- data_300W
        |-- metas/
            |-- train.tsv
            |-- test.tsv
    |-- data_COFW
        |-- metas/
            |-- train.tsv
            |-- test.tsv

Evaluation

• Our checkpoints can be downloaded on HuggingFace.

WFLW

bash scripts/test/WFLW.sh WFLW/best_model.pkl

• NME: 3.97
• FR: 1.96
• AUC: 0.608

COFW

bash scripts/test/COFW.sh COFW/best_model.pkl

• NME 4.54
• FR 0.62
• AUC 0.547

300W

bash scripts/test/300W.sh 300W/best_model.pkl 

• NME: 2.86
• FR: 4.91
• AUC: 0.441

Demo: Comparison of convergence speed between Ours and SoftArgmax

python3 -m demo

Ours has faster convergence speed.

LICENSE

• This work is licensed under the Apache-2.0 license.
• Our project also involves the following assets from other research or projects.
  • STAR

Citation

@inproceedings{yang2025regression,
  title={Heatmap Regression without Soft-Argmax for Facial Landmark Detection},
  author={Yang, Chiao-An and Yeh, Raymond A},
  booktitle={Proc. ICCV},
  year={2025}
}

Contact

Please contact Chiao-An Yang if you have any questions.