📘 Paper PDF | 🪧 Poster | 🛝🛝 Slides
The analysis of patterns of walking is an important area of research that has numerous applications in security, healthcare, sports and human-computer interaction. Lately, walking patterns have been regarded as a unique fingerprinting method for automatic person identification at a distance. In this work, we propose a novel gait recognition architecture called Gait Pyramid Transformer (GaitPT) that leverages pose estimation skeletons to capture unique walking patterns, without relying on appearance information. GaitPT adopts a hierarchical transformer architecture that effectively extracts both spatial and temporal features of movement in an anatomically consistent manner, guided by the structure of the human skeleton. Our results show that GaitPT achieves state-of-the-art performance compared to other skeleton-based gait recognition works, in both controlled and in-the-wild scenarios. GaitPT obtains 82.6% average accuracy on CASIA-B, surpassing other works by a margin of 6%. Moreover, it obtains 52.16% Rank-1 accuracy on GREW, outperforming both skeleton-based and appearance-based approaches.
Install all dependencies with the following command:
pip install -r requirements.txt
Download the preprocessed CASIA-B [1] skeletons from here and extract them in the data/ directory. For GREW [2] and Gait3D [3], contact the authors for the 2D skeletons.
[1] Shiqi Yu, Daoliang Tan and Tieniu Tan, "A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition," 18th International Conference on Pattern Recognition (ICPR'06), Hong Kong, China, 2006, pp. 441-444, doi: 10.1109/ICPR.2006.67.
[2] Zhu, Zheng, et al. "Gait recognition in the wild: A benchmark." Proceedings of the IEEE/CVF international conference on computer vision. 2021.
[3] Jinkai Zheng, Xinchen Liu, Wu Liu, Lingxiao He, Chenggang Yan, Tao Mei, "Gait Recognition in the Wild with Dense 3D Representations and A Benchmark." (2022). IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
To train on CASIA-B run:
scripts/train_casia.sh
To train on GREW run:
scripts/train_grew.sh
To train on Gait3D run:
scripts/train_gait3d.sh
Download the pretrained weights from here and place them in the checkpoints/ directory.
To evaluate pretrained model on CASIA-B run:
scripts/evaluate_casia.sh
To evaluate pretrained model on GREW run:
scripts/evaluate_grew.sh
To evaluate pretrained model on Gait3D run:
scripts/evaluate_gait3d.sh
If you found our work useful or use our dataset in any way, please cite our paper:
@inproceedings{catruna2024gaitpt,
title={GaitPT: Skeletons Are All You Need For Gait Recognition},
author={Catruna, Andy and Cosma, Adrian and Radoi, Emilian},
booktitle={The 18th IEEE International Conference on Automatic Face and Gesture Recognition},
year={2024}
}
This work is protected by CC BY-NC-ND 4.0 License (Non-Commercial & No Derivatives).