This repository stores the preprocessed data for paper:
SignDiff: Learning Diffusion Models for American Sign Language Production
Note: This work has not been accepted yet, and I am currently very busy and do not have time to submit my paper at a recent academic conference. Maybe it will be accepted at some conference in the fall and that might be a good time to open up the code.
After preprocessing How2Sign dataset, the condensed data set obtained is as follows:
It can be used in the training of ASL production models.
Note: Because I later processed more data, the link above is four times the size of the one in the paper and is the result of the full How2Sign processing.
After preprocessing Phoenix-14T dataset, the condensed data set obtained is as follows:
It can be used in the training of GSL production models.
After preprocessing How2Sign dataset, the condensed data set obtained is as follows:
It can be used for the diffusion model training of pose2video in sign language. (Based on ControlNet)
After preprocessing How2Sign dataset, the condensed data set obtained is as follows:
It can be used for the GAN model training of pose2video in sign language. (Based on Vid2Vid)
Our pre-processing tools: the data cleansing tool and the three-step 2Dto3D tool.
Stay tuned. The data above should be sufficient for the time being.
@misc{fang2024signllm,
title={SignLLM: Sign Languages Production Large Language Models},
author={Sen Fang and Lei Wang and Ce Zheng and Yapeng Tian and Chen Chen},
year={2024},
eprint={2405.10718},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{fang2023signdiff,
title={SignDiff: Learning Diffusion Models for American Sign Language Production},
author={Sen Fang and Chunyu Sui and Xuedong Zhang and Yapeng Tian},
year={2023},
eprint={2308.16082},
archivePrefix={arXiv},
primaryClass={cs.CV}
}