Skip to content
CocoaPod to analyze body motion and exercise technique
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
Example
Models
Utils
Views
pose-trainer
.gitignore
.swiftlint.yml
LICENSE
README.md
TechniqueAnalysis.podspec
_Pods.xcodeproj

README.md

TechniqueAnalysis

Description

This is a CocoaPod which processes video footage of a user doing an exercise, and provides feedback on the user's form. The general idea is to capture a video and convert it to a timeseries, where each data point contains an array of body point locations in 2D space, along with a confidence level on the accuracy of each body point.

pipeline

It is already possible to estimate body points (called pose estimation), for example using the project OpenPose. tucan9389 has provided an excellent starting point for the code implemented here, based on his project which predicts body points in real-time.

To translate from pose estimation to technique analysis, we need to train a model on timeseries with correct and incorrect form for various exercises. Richard Yang and Steven Chen from Stanford University have had relative success with Dynamic Time Warping (DTW) to solve this type of problem, as described in this paper.

dtw

Additional research by Keogh et al. in their paper "Fast Time Series Classification Using Numerosity Reduction" provides a promising, efficient algorithm to train a kNN model with DTW (with k=1 and a dynamic warping window) for a use case similar to the one described here.

The ML frameworks compatible with CoreML on iOS (e.g. scikit-learn) are currently lacking a suitable model to pre-train a kNN-DTW model and bundle it into an iOS application. Because of this, the best option may be to implement the algorithm manually and ship it with the application. The kNN-DTW algorithm here is written in Swift, but writing the algo in C would provide a much faster runtime. For further explanation on kNN-DTW, Mark Regan has implemented a Python version of kNN-DTW similar to the one described by Keogh et al. It can be found here.

Example

To run the example project, clone the repo, and run pod install from the Example directory first. You should use Xcode's Legacy Build System to correctly include the videos in Bundle Resources (File → Workspace Settings → Shared Workspace Settings → Build System).

demo

Requirements

iOS 11, Swift 4.2

Installation

TechniqueAnalysis is available through CocoaPods. To install it, simply add the following line to your Podfile:

pod 'TechniqueAnalysis'

Author

trevphil, trevor.j.phillips@uconn.edu

License

TechniqueAnalysis is available under the MIT license. See the LICENSE file for more info.

You can’t perform that action at this time.