Non-invasive estimation of ground and joint kinetics through deep learning
http://digitalathlete.org

Deep learning models driven by wearable sensor accelerometers can replace captive laboratory instrumentation to facilitate biomechanical accuracy and validity anywhere By employing a new deep learning workbench for spatio-temporal data, we train convolutional neural networks (CNN) with archive biomechanics data to predict accurate multidimensional on-field analytics for complex sports movements. Using test sets from multi data-captures which include ground truth force plate or source modeling, we see strong correspondence between measured versus predicted ground reaction forces and moments, and knee joint moments. Driven by eight markers, study two GRF/M mean r>0.97, study three KJM mean r>0.88, and from five wearable sensor accelerometers, study four GRF mean r>0.87. The overarching hypothesis, whether it is possible to build deep learning models which can mimic the physics behind human movement, specifically to replace force plate derived kinetic output, is supported. William R Johnson PhD CPEng CSCS (he/him/his) bill@johnsonwr.com | August 2023 cv.billjohnson.org | videocv.billjohnson.org

Caution, model files are large, you may not wish to pull the complete repository. GitHub limits file sizes to 100MB, files larger than this have been broken up using split. Instructions to reconstitute files are given inline.

Post-doc: Conference presentations, panels, & session chairs

ISBS-2022 ISBS	Neural Networks Session Chair http://www.isbs2022.org/programme.html
NSCA/PBSCCS-2021 NSCA	NSCA Baseball and Sport Science SIG Performance Technology Roundtable Video https://vimeo.com/627746779/74b81ecfd5
ISBS-2021 ISBS	Data Science and Sports Biomechanics Panel http://www.isbs2021.org/workshops-and-panels Video https://youtu.be/A_PeEtMN92k

Post-doc: A comparison of three neural network approaches for estimating joint angles and moments from inertial measurement units

Keywords	Machine learning · Wearable sensors · Joint kinematics · Joint kinetics
Sensors	https://www.mdpi.com/1424-8220/21/13/4535/pdf [12]

Doctoral thesis: Non-invasive estimation of ground and joint kinetics through deep learning

Keywords	Biomechanics · Data science · Deep learning · Big data · Sports analytics · Computer vision · Motion capture · Wearable sensors
The University of Western Australia	https://research-repository.uwa.edu.au/en/publications/non-invasive-estimation-of-ground-and-joint-kinetics-through-deep

Study four: Multidimensional ground reaction forces and moments from wearable sensor accelerations via deep learning

Keywords	Biomechanics · Wearable sensors · Simulated accelerations · Workload exposure · Sports analytics · Deep learning
IEEE TBME	https://ieeexplore.ieee.org/document/9130158 [11a]
arXiv	https://arxiv.org/abs/1903.07221 [11b]
Deep learning workbench for biomechanics	Each study utilized an incremental sequence of data preparation and modeling strategies, which by study four had evolved into the "deep learning workbench for biomechanics." Although the individual data science components had previously existed in the literature, the approach was novel and unique in this configuration and application to sports biomechanics.
ISB-2019 ISB/ASB	Multidimensional ground reaction forces predicted from a single sacrum-mounted accelerometer via deep learning Abstract http://bit.ly/2M9j3rw [10] Presentation http://bit.ly/2SHcsYv
EMS HDR Conference 2018	Poster (Conference Award) http://bit.ly/2yXgdgO
WCB-2018	Abstract (Student Bursary Award) http://bit.ly/2GzYnHD [6] Presentation with commentary http://bit.ly/2tCKHTo
ISBS-2018 ISBS	Artificial intelligence, data analytics and sports biomechanics: A new era or a false dawn? Abstract https://commons.nmu.edu/cgi/viewcontent.cgi?article=1618&context=isbs [7]
MATLAB figures	https://github.com/johnsonwr/digitalathlete/tree/master/study4/figures
Caffe models	https://github.com/johnsonwr/digitalathlete/tree/master/study4/models (637MB) cat grftrain_181112104456175_mcrnet.caffemodel_* > grftrain_181112104456175_mcrnet.caffemodel # reconstitute CaffeNet donor seed model cat grftrain_181123170749181_mcrnet.caffemodel_* > grftrain_181123170749181_mcrnet.caffemodel # reconstitute CaffeNet model
Prototxt	https://github.com/johnsonwr/digitalathlete/tree/master/study4/prototxt

Study three: On-field player workload exposure and knee injury risk monitoring via deep learning

Keywords	Biomechanics · Wearable sensors · Computer vision · Motion capture · Sports analytics
Journal of Biomechanics	https://www.sciencedirect.com/science/article/abs/pii/S0021929019304427 [8a]
arXiv	https://arxiv.org/abs/1809.08016 [8b]
ISB-2019 ISB/ASB	Predicting ground and joint kinetics from wearable sensor accelerations via deep learning Abstract http://bit.ly/2y7mZ3A [9] Presentation (panel) http://bit.ly/2rIh2uo
Presentation	http://bit.ly/2HS7HCv
Animation	Training set marker trajectories versus corresponding knee joint moments visualization (supplementary figure) http://bit.ly/2yTaX1f
MATLAB figures	https://github.com/johnsonwr/digitalathlete/tree/master/study3/figures
Caffe models	https://github.com/johnsonwr/digitalathlete/tree/master/study3/models (2.6GB) cat grftrain_180612214018112_mcrnet.caffemodel_j01_* > grftrain_180612214018112_mcrnet.caffemodel_j01 # reconstitute CaffeNet donor seed model 01 cat grftrain_190215144249130_mcrnet.caffemodel_j01_* > grftrain_190215144249130_mcrnet.caffemodel_j01 # reconstitute CaffeNet model 01
Caffe prototxt	https://github.com/johnsonwr/digitalathlete/tree/master/study3/prototxt

Study two: Predicting athlete ground reaction forces and moments from spatio-temporal driven CNN models

Keywords	Biomechanics · Supervised learning · Image motion analysis · Computer simulation · Pattern analysis
IEEE TBME	Paper https://ieeexplore.ieee.org/document/8408711 [5] Cover & Feature https://tbme.embs.org/2019/03/01/predicting-athlete-ground-reaction-forces-and-moments-from-spatio-temporal-driven-cnn-models
Animation	Training set marker trajectories versus corresponding ground reaction forces and moments visualization (supplementary figure) http://bit.ly/2Is3PJx
UWA CSSE Conference 2017	Relative performance of Caffe deep learning models for spatio-temporal sport analytics Presentation http://bit.ly/2TCWqwM
ISBS-2017 ISBS	Prediction of ground reaction forces and moments via supervised learning is independent of participant sex, height and mass Abstract (Student Travel Grant) https://commons.nmu.edu/cgi/viewcontent.cgi?&article=1034&context=isbs [3] Presentation http://bit.ly/2MvqW8c
MATLAB figures	https://github.com/johnsonwr/digitalathlete/tree/master/study2/figures
Caffe models	https://github.com/johnsonwr/digitalathlete/tree/master/study2/models (1.3GB) cat grftrain_190215204017060_mcrnet.caffemodel_j01_* > grftrain_190215204017060_mcrnet.caffemodel_j01 # reconstitute CaffeNet model 01
Caffe prototxt	https://github.com/johnsonwr/digitalathlete/tree/master/study2/prototxt
CaffeNet reference	https://github.com/BVLC/caffe/tree/master/models/bvlc_reference_caffenet

Study one: Predicting athlete ground reaction forces and moments from motion capture

Keywords	Action recognition · Wearable sensors · Computer simulation
MBEC	https://link.springer.com/article/10.1007/s11517-018-1802-7 [4]
UWA CSSE Conference 2016	Presentation with commentary http://bit.ly/2kcgXrw
ISBS-2016 ISBS	The personalised 'Digital Athlete': An evolving vision for the capture, modelling and simulation, of on-field athletic performance Abstract https://ojs.ub.uni-konstanz.de/cpa/article/download/7099/6390 [2]
MATLAB figures	https://github.com/johnsonwr/digitalathlete/tree/master/study1/figures
R models	https://github.com/johnsonwr/digitalathlete/tree/master/study1/models (1.9GB) cat grftrain_171214215406095_R_predict_model_* > grftrain_171214215406095_R_predict_model.Rda # reconstitute R model
R SPLS reference	https://cran.r-project.org/web/packages/spls/index.html

HDR prelim: Validity of a markerless motion capture system for sporting application

The University of Western Australia	https://bit.ly/3qLDIUB

Master's assignment: Talent identification in elite rugby union - a theoretical update to an existing predictor algorithm

Keywords	Athlete selection · Predictor variables · Algorithm · Weightings
JASC	https://www.strengthandconditioning.org/jasc-25-3 [1]

Machine Learning resources

Andrew Ng	Machine Learning course https://www.deeplearning.ai/the-batch https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/ai-for-everyone http://cs229.stanford.edu http://www.mlyearning.org
Fei-Fei Li	Stanford Convolutional Neural Networks for Visual Recognition http://cs231n.stanford.edu http://cs231n.github.io
Ian Goodfellow	Deep Learning http://www.deeplearningbook.org https://www.amazon.com/Deep-Learning-Adaptive-Computation-Machine/dp/0262035618
Bill Lubanovic	Introducing Python https://www.amazon.com/Introducing-Python-Modern-Computing-Packages/dp/1449359361
Sebastian Raschka	Python Machine Learning https://www.amazon.com/Python-Machine-Learning-scikit-learn-TensorFlow-ebook/dp/B0742K7HYF
Aurélien Géron	Hands-On Machine Learning with Scikit-Learn & TensorFlow https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-TensorFlow-ebook/dp/B06XNKV5TS
Sean Driver	Perth Machine Learning Group https://www.meetup.com/en-AU/Perth-Machine-Learning-Group https://github.com/pmlg/pmlg.github.io https://www.fast.ai

Biomechanics resources

The University of Western Australia	Biomechanics courses SSEH2250 Biomechanics in Sport and Exercise SSEH3355 Biomechanical Principles SSEH4633 Advanced Biomechanical Methods
Clearinghouse for Sport	National 3D Motion Capture Best Practice Resources https://www.clearinghouseforsport.gov.au/networks/3d-motion-capture
David Winter	Biomechanics and Motor Control of Human Movement https://www.amazon.com/Biomechanics-Motor-Control-Human-Movement/dp/0470398183
D. Gordon E. Robertson Graham E. Caldwell Joseph Hamill Gary Kamen Saunders N. Whittlesey	Research Methods in Biomechanics https://www.amazon.com/Research-Methods-Biomechanics-Gordon-Robertson/dp/0736093400
Roger Bartlett	Introduction to Sports Biomechanics https://www.amazon.com/Introduction-Sports-Biomechanics-Analysing-Movement/dp/0415632439
Joseph Hamill Kathleen Knutzen Timothy Derrick	Biomechanical Basis of Human Movement https://www.amazon.com/Biomechanical-Basis-Movement-Joseph-Hamill/dp/1451177305
Jim Richards	Biomechanics in Clinic and Research: An Interactive Teaching and Learning Course https://www.amazon.com/Comprehensive-Textbook-Clinical-Biomechanics-learning/dp/0702054895
Carl Peyton Adrian Burden	Biomechanical Evaluation of Movement in Sport and Exercise https://www.amazon.com/Biomechanical-Evaluation-Movement-Exercise-Science/dp/0415632668
Youlian Hong Roger Bartlett	Routledge Handbook of Biomechanics and Human Movement Science https://www.amazon.com/Routledge-Handbook-Biomechanics-International-Handbooks-ebook/dp/B001PC1ZX8
The Biomechanist	The Week in Biomechanics https://www.biomechanist.net/blog
Biomch-L	Forum, sponsored by the International Society of Biomechanics (ISB) https://biomch-l.isbweb.org
3-D Analysis of Human Movement	Technical Group of the International Society of Biomechanics (ISB) http://www.geocities.ws/3d-ahm
Awesome Biomechanics	A curated repository of biomechanical resources https://github.com/modenaxe/awesome-biomechanics

PhD supervisors

Jacqueline A. Alderson	School of Human Sciences (Exercise and Sport Science), The University of Western Australia, Perth, Australia Sports Performance Research Institute New Zealand (SPRINZ), Auckland University of Technology, Auckland, New Zealand https://research-repository.uwa.edu.au/en/persons/jacqueline-alderson
Ajmal Mian	Department of Computer Science and Software Engineering, The University of Western Australia, Perth, Australia https://research-repository.uwa.edu.au/en/persons/ajmal-mian Machine Intelligence Group https://www.youtube.com/channel/UCy-HDqRqdYS3UUiCIqqfFxQ/videos
David G. Lloyd	Menzies Health Institute Queensland, and the School of Allied Health Sciences, Griffith University, Gold Coast, Australia https://experts.griffith.edu.au/academic/david.lloyd

Acknowledgements

This project was partially supported by the ARC Discovery Grant DP190102443 and an Australian Government Research Training Program Scholarship. NVIDIA Corporation is gratefully acknowledged for the GPU provision through its Hardware Grant Program, Eigenvector Research for the PLS_Toolbox licence, and C-Motion Inc. for the Visual3D licence. Portions of data included in this study were funded by NHMRC grant 400937.