This is the official implementation of the paper: "Multi-View Knowledge Guided Semantic Prototype Learning for Generalized Zero-Shot Action Recognition". (IEEE Transactions on Multimedia)
The Attentional Multi-view Semantic Fusion (AMSF) model is an improved version of the MSF model, incorporating action descriptions generated by GPT-4 and a multi-semantic fusion strategy based on the multi-head attention.
- Python >= 3.8.13
- Torch >= 1.12.1
- Scikit-Learn
We evaluate our method on the following datasets:
To run this code, you need to place the skeleton features in ./data/sk_feat/, the semantic features in ./data/text_feat/, and the dataset split settings in ./data/splits/. All the data used in our experiments can be downloaded here.
If you wish to use other visual features, please refer to the ./GZSSAR/dataset.py file and follow the saving format used in ./data/sk_feat/. To generate semantic features using text encoders from CLIP or Long-CLIP, please refer to the ./gen_text_feat.py file. If you want to customize the split settings, please refer to the ./gen_split.py file.
Please run this code via main.py. The arguments can be specified either through the YAML file (in ./config/) or the command line. (Arguments specified via the command line will take precedence over those defined in the YAML file.)
python main.py -c MHA.yaml
python main.py -g 0If you want to perform step-by-step training/testing, or use only the ZSL mode, please modify the mode parameter.
python main.py -m zsl@ARTICLE{Li2025AMSF,
author={Li, Ming-Zhe and Jia, Zhen and Zhang, Zhang and Li, Yaoning and Ma, Zhanyu and Wang, Liang},
journal={IEEE Transactions on Multimedia},
title={Multi-View Knowledge Guided Semantic Prototype Learning for Generalized Zero-Shot Action Recognition},
year={2025},
volume={27},
pages={9735-9748}
}