http://digitalathlete.org
Non-invasive estimation of ground and joint kinetics through deep learning
Deep learning models driven by wearable sensor accelerometers can replace captive laboratory instrumentation to facilitate biomechanical accuracy and validity anywhere By employing a new deep learning workbench for spatio-temporal data, we train convolutional neural networks (CNN) with archive biomechanics data to predict accurate multidimensional on-field analytics for complex sports movements. Using test sets from multi data-captures which include ground truth force plate or source modeling, we see strong correspondence between measured versus predicted ground reaction forces and moments, and knee joint moments. Driven by eight markers, study two GRF/M mean r>0.97, study three KJM mean r>0.88, and from five wearable sensor accelerometers, study four GRF mean r>0.87. The overarching hypothesis, whether it is possible to build deep learning models which can mimic the physics behind human movement, specifically to replace force plate derived kinetic output, is supported. William R Johnson PhD CPEng CSCS (he/him/his) bill@johnsonwr.com | July 2022 cv.billjohnson.org | videocv.billjohnson.org ![]() ![]() ![]() ![]() ![]() |
![]() |
Caution, model files are large, you may not wish to pull the complete repository.
GitHub limits file sizes to 100MB, files larger than this have been broken up using split.
Instructions to reconstitute files are given inline. |
Post-doc: Conference presentations, panels, & session chairs
ISBS-2022 ISBS |
Neural Networks Session Chair http://www.isbs2022.org/programme.html | |
---|---|---|
NSCA/PBSCCS-2021 NSCA |
NSCA Baseball and Sport Science SIG Performance Technology Roundtable Video https://vimeo.com/627746779/74b81ecfd5 | |
ISBS-2021 ISBS |
Data Science and Sports Biomechanics Panel http://www.isbs2021.org/workshops-and-panels Video https://youtu.be/A_PeEtMN92k |
Post-doc: A comparison of three neural network approaches for estimating joint angles and moments from inertial measurement units
Keywords | Machine learning · Wearable sensors · Joint kinematics · Joint kinetics |
---|---|
Sensors | https://www.mdpi.com/1424-8220/21/13/4535/pdf [12] |
Doctoral thesis: Non-invasive estimation of ground and joint kinetics through deep learning
Keywords | Biomechanics · Data science · Deep learning · Big data · Sports analytics · Computer vision · Motion capture · Wearable sensors |
---|---|
The University of Western Australia |
https://research-repository.uwa.edu.au/en/publications/non-invasive-estimation-of-ground-and-joint-kinetics-through-deep |
Study four: Multidimensional ground reaction forces and moments from wearable sensor accelerations via deep learning
Keywords | Biomechanics · Wearable sensors · Simulated accelerations · Workload exposure · Sports analytics · Deep learning | |
---|---|---|
IEEE TBME | https://ieeexplore.ieee.org/document/9130158 [11a] | |
arXiv | https://arxiv.org/abs/1903.07221 [11b] | |
Deep learning workbench for biomechanics |
Each study utilized an incremental sequence of data preparation and modeling strategies, which by study four had evolved into the "deep learning workbench for biomechanics."
Although the individual data science components had previously existed in the literature, the approach was novel and unique in this configuration and application to sports biomechanics.
![]() | |
ISB-2019 ISB/ASB |
Multidimensional ground reaction forces predicted from a single sacrum-mounted accelerometer via deep learning Abstract http://bit.ly/2M9j3rw [10] Presentation http://bit.ly/2SHcsYv | |
EMS HDR Conference 2018 |
Poster (Conference Award) http://bit.ly/2yXgdgO | |
WCB-2018 |
Abstract (Student Bursary Award) http://bit.ly/2GzYnHD [6] Presentation with commentary http://bit.ly/2tCKHTo | |
ISBS-2018 ISBS | Artificial intelligence, data analytics and sports biomechanics: A new era or a false dawn? Abstract https://commons.nmu.edu/cgi/viewcontent.cgi?article=1618&context=isbs [7] | |
MATLAB figures | https://github.com/johnsonwr/digitalathlete/tree/master/study4/figures | |
Caffe models | https://github.com/johnsonwr/digitalathlete/tree/master/study4/models (637MB)
| |
Prototxt | https://github.com/johnsonwr/digitalathlete/tree/master/study4/prototxt |
Study three: On-field player workload exposure and knee injury risk monitoring via deep learning
Keywords | Biomechanics · Wearable sensors · Computer vision · Motion capture · Sports analytics |
---|---|
Journal of Biomechanics | https://www.sciencedirect.com/science/article/abs/pii/S0021929019304427 [8a] |
arXiv | https://arxiv.org/abs/1809.08016 [8b] |
ISB-2019 ISB/ASB |
Predicting ground and joint kinetics from wearable sensor accelerations via deep learning Abstract http://bit.ly/2y7mZ3A [9] Presentation (panel) http://bit.ly/2rIh2uo |
Presentation | http://bit.ly/2HS7HCv |
Animation | Training set marker trajectories versus corresponding knee joint moments visualization (supplementary figure) http://bit.ly/2yTaX1f |
MATLAB figures | https://github.com/johnsonwr/digitalathlete/tree/master/study3/figures |
Caffe models | https://github.com/johnsonwr/digitalathlete/tree/master/study3/models (2.6GB)
|
Caffe prototxt | https://github.com/johnsonwr/digitalathlete/tree/master/study3/prototxt |
Study two: Predicting athlete ground reaction forces and moments from spatio-temporal driven CNN models
Keywords | Biomechanics · Supervised learning · Image motion analysis · Computer simulation · Pattern analysis | ![]() |
---|---|---|
IEEE TBME |
Paper https://ieeexplore.ieee.org/document/8408711 [5] Cover & Feature https://tbme.embs.org/2019/03/01/predicting-athlete-ground-reaction-forces-and-moments-from-spatio-temporal-driven-cnn-models | |
Animation | Training set marker trajectories versus corresponding ground reaction forces and moments visualization (supplementary figure) http://bit.ly/2Is3PJx | |
UWA CSSE Conference 2017 | Relative performance of Caffe deep learning models for spatio-temporal sport analytics Presentation http://bit.ly/2TCWqwM | |
ISBS-2017 ISBS | Prediction of ground reaction forces and moments via supervised learning is independent of participant sex, height and mass Abstract (Student Travel Grant) https://commons.nmu.edu/cgi/viewcontent.cgi?&article=1034&context=isbs [3] Presentation http://bit.ly/2MvqW8c | |
MATLAB figures | https://github.com/johnsonwr/digitalathlete/tree/master/study2/figures | |
Caffe models | https://github.com/johnsonwr/digitalathlete/tree/master/study2/models (1.3GB)
| |
Caffe prototxt | https://github.com/johnsonwr/digitalathlete/tree/master/study2/prototxt | |
CaffeNet reference | https://github.com/BVLC/caffe/tree/master/models/bvlc_reference_caffenet |
Study one: Predicting athlete ground reaction forces and moments from motion capture
Keywords | Action recognition · Wearable sensors · Computer simulation | |
---|---|---|
MBEC | https://link.springer.com/article/10.1007/s11517-018-1802-7 [4] | |
UWA CSSE Conference 2016 | Presentation with commentary http://bit.ly/2kcgXrw | |
ISBS-2016 ISBS | The personalised 'Digital Athlete': An evolving vision for the capture, modelling and simulation, of on-field athletic performance Abstract https://ojs.ub.uni-konstanz.de/cpa/article/download/7099/6390 [2] | |
MATLAB figures | https://github.com/johnsonwr/digitalathlete/tree/master/study1/figures | |
R models | https://github.com/johnsonwr/digitalathlete/tree/master/study1/models (1.9GB)
| |
R SPLS reference | https://cran.r-project.org/web/packages/spls/index.html |
HDR prelim: Validity of a markerless motion capture system for sporting application
The University of Western Australia |
https://bit.ly/3qLDIUB |
---|
Master's assignment: Talent identification in elite rugby union - a theoretical update to an existing predictor algorithm
Keywords | Athlete selection · Predictor variables · Algorithm · Weightings |
---|---|
JASC | https://www.strengthandconditioning.org/jasc-25-3 [1] |
Machine Learning resources
Biomechanics resources
The University of Western Australia |
Biomechanics courses SSEH2250 Biomechanics in Sport and Exercise SSEH3355 Biomechanical Principles SSEH4633 Advanced Biomechanical Methods |
---|---|
David Winter | Biomechanics and Motor Control of Human Movement https://www.amazon.com/Biomechanics-Motor-Control-Human-Movement/dp/0470398183 |
D. Gordon E. Robertson Graham E. Caldwell Joseph Hamill Gary Kamen Saunders N. Whittlesey |
Research Methods in Biomechanics https://www.amazon.com/Research-Methods-Biomechanics-Gordon-Robertson/dp/0736093400 |
Roger Bartlett | Introduction to Sports Biomechanics https://www.amazon.com/Introduction-Sports-Biomechanics-Analysing-Movement/dp/0415632439 |
Joseph Hamill Kathleen Knutzen Timothy Derrick |
Biomechanical Basis of Human Movement https://www.amazon.com/Biomechanical-Basis-Movement-Joseph-Hamill/dp/1451177305 |
Jim Richards | Biomechanics in Clinic and Research: An Interactive Teaching and Learning Course https://www.amazon.com/Comprehensive-Textbook-Clinical-Biomechanics-learning/dp/0702054895 |
Carl Peyton Adrian Burden |
Biomechanical Evaluation of Movement in Sport and Exercise https://www.amazon.com/Biomechanical-Evaluation-Movement-Exercise-Science/dp/0415632668 |
Youlian Hong Roger Bartlett |
Routledge Handbook of Biomechanics and Human Movement Science https://www.amazon.com/Routledge-Handbook-Biomechanics-International-Handbooks-ebook/dp/B001PC1ZX8 |
The Biomechanist | The Week in Biomechanics https://www.biomechanist.net/blog |
Biomch-L | Forum, sponsored by the International Society of Biomechanics (ISB) https://biomch-l.isbweb.org |
3-D Analysis of Human Movement | Technical Group of the International Society of Biomechanics (ISB) http://www.geocities.ws/3d-ahm |
Awesome Biomechanics | A curated repository of biomechanical resources https://github.com/modenaxe/awesome-biomechanics |
PhD supervisors
Jacqueline A. Alderson |
School of Human Sciences (Exercise and Sport Science), The University of Western Australia, Perth, Australia Sports Performance Research Institute New Zealand (SPRINZ), Auckland University of Technology, Auckland, New Zealand https://research-repository.uwa.edu.au/en/persons/jacqueline-alderson |
---|---|
Ajmal Mian |
Department of Computer Science and Software Engineering, The University of Western Australia, Perth, Australia https://research-repository.uwa.edu.au/en/persons/ajmal-mian Machine Intelligence Group https://www.youtube.com/channel/UCy-HDqRqdYS3UUiCIqqfFxQ/videos |
David G. Lloyd |
Menzies Health Institute Queensland, and the School of Allied Health Sciences, Griffith University, Gold Coast, Australia https://experts.griffith.edu.au/academic/david.lloyd |
Acknowledgements
This project was partially supported by the ARC Discovery Grant DP190102443 and an Australian Government Research Training Program Scholarship. NVIDIA Corporation is gratefully acknowledged for the GPU provision through its Hardware Grant Program, Eigenvector Research for the PLS_Toolbox licence, and C-Motion Inc. for the Visual3D licence. Portions of data included in this study were funded by NHMRC grant 400937. |