Skip to content
💃 Single pose estimation for iOS and android using TensorFlow 2.0
Branch: master
Clone or download

README.md

💃 Mobile Pose Estimation for TensorFlow 2.0

This repository is forked from edvardHua/PoseEstimationForMobile when the original repository was closed.
edvardHua/PoseEstimationForMobile repository is reopened! I'll maintain it separately. 👍

This repository currently implemented the Hourglass model using TensorFlow 2.0 (and Keras API). Instead of normal convolution, inverted residuals (also known as Mobilenet V2) module has been used inside the model for real-time inference.

Table of contents

Goals

  • 📚 Easy to train
  • 🏃‍ Easy to use the model on mobile device

Getting Started

Install Anaconda (~10 min)

How To Install Anaconda on Ubuntu 18.04 [Quickstart]

Create Virtual Environment (~2 min)

Create new environment.

conda create -n {env_name} python={python_version} anaconda
# in my case
# conda create -n mpe-env-tf2-alpha0 python=3.7 anaconda

Start the environment.

source activate {env_name}
# in my case
# source activate mpe-env-tf2-alpha0

Install the requirements (~1 min)

cd {tf2-mobile-pose-estimation_path}
pip install -r requirements.txt

Run The Project

In order to use the project you have to:

  1. Prepare the dataset(ai_challenger dataset) and locate the dataset on ./datasets.
  2. Run the model using:
python train.py
  1. Monitoring with TensorBoard:
tensorboard --logdir="./outputs/logs"

Results

Loss

tensorboard-1-190403

Accuracy

tensorboard-1-190403

Converting To Mobile Model

TensorFLow Lite

  1. Prepare the trained model(.hdf5) from section below Run The Project
  2. Modify trained_model_file_name variable in convert_to_tflite.py to your own trained model.
trained_model_file_name = "{model_file_name}.hdf5"
  1. Run the convert_to_tflite.py:
python convert_to_tflite.py
  1. And then, you can find the .tflite model on {PROJECT_PATH}/outputs/models/tflite/{model_file_name}.tflite.

Core ML (Preparing...)

Related issue: https://github.com/tucan9389/tf2-mobile-pose-estimation/issues/13

Details

This section will be separated to other .md file.

Folder Structure

├── train.py            - the main script file
├── data_loader.py      
├── data_augment.py     
├── data_prepare.py     
├── hourglass_model.py  
├── model_config.py     
├── network_base.py     
├── path_manage.py      
├── train_config.py     
├── requirements.txt    
├── datasets            - this folder contain the datasets of the project.
|   └── ai_challenger
|       ├── ai_challenger_train.json
|       ├── ai_challenger_valid.json
|       ├── train
|       └── valid
└── outputs             - this folder will be generated automatically when start training
    ├── models
    └── logs

TODO

  • Save model(.hdf5 or .ckpt)
  • Convert the model(.hdf5 or .ckpt) to TFLite model(.tflite)
  • Convert the model(.hdf5 or .ckpt) to Core ML model(.mlmodel)
  • Run the model on Android
  • Run the model on iOS
  • Make DEMO gif running on mobile device

Acknowledgements

This project is based on edvardHua/PoseEstimationForMobile and jwkanggist/tf-tiny-pose-estimation.

Reference

[1] Paper of Convolutional Pose Machines
[2] Paper of Stack Hourglass
[3] Paper of MobileNet V2
[4] Repository PoseEstimation-CoreML
[5] Repository of tf-pose-estimation
[6] Devlope guide of TensorFlow Lite
[7] Mace documentation

Related Projects

Other Pose Estimation Projects

Contributing

This section will be separated to other .md file.

Any contributions are welcome including improving the project.

License

Apache License 2.0

You can’t perform that action at this time.