Skip to content

Implementation of STGAE: Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising (ISMAR 2021)

Notifications You must be signed in to change notification settings

ZhouKanglei/STGAE

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

STGAE: Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising

The implementation of STGAE: Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising in ISMAR 2021.

We propose a method for denoising hand motion data in mixed reality using a spatial-temporal graph auto-encoder. Our approach models the articulated hand structure with a partition strategy and extracts structural constraints using graph convolution and self-attention. We combine graph and temporal convolutions in an hourglass residual auto-encoder to preserve structural constraints while denoising hand motion data. Our method outperforms state-of-the-art approaches in both quantitative and qualitative experiments.

Paper Address | Download Paper | Supplementary Video | Multi-STGAE

Framework

Framework

The pipeline of the proposed method for hand motion denoising using STGAE is shown in the above figure. For more details, please refer to the supplementary video.

Experiments

  • Denoising results

Demo

More demos can be seen in our supplementary video.

Environments

  • LaTeX tool
sudo apt-get install texlive-full
  • FFmpeg
sudo apt-get install ffmpeg
  • pydot & graphviz
sudo pip3 install pydot
sudo pip3 install graphviz

App

The core code of the client and server is available, and detailed development tips can be found at the link

Citation

@inproceedings{zhou2021stgae,
  title={STGAE: Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising},
  author={Zhou, Kanglei and Cheng, Zhiyuan and Shum, Hubert PH and Li, Frederick WB and Liang, Xiaohui},
  booktitle={2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
  pages={41--49},
  year={2021},
  organization={IEEE}
}

Contact

Feel free to contact me via zhoukanglei[at]qq.com.

About

Implementation of STGAE: Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising (ISMAR 2021)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published