Patient-Activity-Recognition-System

Patient Activity Recognition System trained with CNN LSTM on Synthetic Data Generated using Unity

Blog: https://medium.com/@parthgoel_22399/patient-monitoring-system-cnn-lstm-model-trained-on-unity-3d-simulated-data-79e52a8163ee

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Dataset and Feature Extraction
4. Contributions
5. License
6. Acknowledgments

About The Project

Hospitals and other medical centres have facilities of keeping patients under medical attention to monitor their progress. The nurse carefully monitors the patients and takes necessary action in case of any unusual activity or sense of emergency. However, sometimes patients cannot afford this service as it’s expensive. If we have an automated system in place which tracks patient’s activity and informs concerned authorities to take immediate actions in case of emergency, then many people would be benefitted. This will incur camera installation and setting up of other logistics but would be still cheaper than human monitoring. Also, there are many instances of complications or even deaths in rehab centres because of no or delayed action in case of events like collapse, heart attack or seizures. This can be mitigated by having a 24*7 surveillance in place that recognizes an abnormality in patient’s behaviour and alerts respective authorities. We introduce a novel model in this paper to monitor patient activity using the CNN-LSTM model.
For more details, please see Report

(back to top)

Built With

• TensorFlow+Keras
• NumPy
• OpenCV
• MediaPipe
• Mixamo
• Unity3D
• Flask

(back to top)

Getting Started

Prerequisites

• TensorFlow

  pip install tensorflow

• NumPy

  pip install numpy

• OpenCV

  pip install opencv-python

• Mediapipe

  pip install mediapipe

• Unity

  Follow instructions: https://unity.com/download

• Flask

  pip install Flask

Installation

1. Clone the repo

   git clone https://github.com/parthgoe1/Patient-Activity-Recognition-System.git

(back to top)

Dataset and Feature Extraction

Since it was difficult to get real data of a rehab center and then train the model for some specific events, we generated synthetic data using Unity3D. We divided the training data into two broad categories; Activities that are considered normal and those which are abnormal.
Normal activities:Sitting, Standing, Walking ,Sleeping
Abnormal activities:Falling, pain/coughing

We divide the simulations into frames for keypoint detection model. Further, the output from keypoint models are normalized to pass it on to activity detection model.

(back to top)

Contributors

1. Parth Goel
2. Prashant Kanth
3. Aditya Bhat
4. Rishika Bhanushali

(back to top)

License

Distributed under the MIT License. See LICENSE.txt for more information.

(back to top)

Acknowledgments

• S. N. Gowda, M. Rohrbach, and L. Sevilla-Lara, “Smart frame selection for action recognition,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 1451–1459, 2021.
• Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7291–7299, 2017.
• V. Bazarevsky, I. Grishchenko, K. Raveendran, T. Zhu, F. Zhang, and M. Grundmann, “Blazepose: On-device real-time body pose tracking,” arXiv preprint arXiv:2006.10204, 2020.
• T. N. Sainath, O. Vinyals, A. Senior, and H. Sak, “Convolutional, long short-term memory, fully connected deep neural networks,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 4580–4584, IEEE, 2015.

(back to top)