Remote control algorithms and schema is always changing in order to be even more practical and precise than its past evolution. As the gesture recognition algorithms based on EMG evolve, more gestures can be obtained by the usage of Artificial Neural Networks (ANN). These algorithms are designed to identify a certain position or movement in a specified region, which makes them useful for diagnostics. In this study we focus on the dynamic classification of five different possitions of the forearm based on EMG, and the possibility to embed this system into a micro-controller unit (MCU). In order to provide a solution to this project, an EMG classification system is proposed, using 2 EMG channels located on the forearm (CH_0 located in flexor group and CH_1 located in extensor group) and a classifier algorithm based on Recurrent Neural Network (RNN) to classify the combined input signals from each channel and provide a group of commands based on the possition of the forearm and the activity of each muscular group.
In order to successfuly register EMG signals, we used the Myoware system, which registers, amplifies and rectifies the EMG signal produced between its two muscle electrodes and its reference electrode. It then sends the amplified and rectified signal through the output signal SIG pin. The MCU then registers the signal in a dequeue vector, indicating a 100 ms time window, and extracting its relevant features, such as the mean, standard deviation and RMS value. These relevant features were considered for evaluation mainly because they provide high-value information regarding the overall contraction and relaxation of the selected muscular group while being features of fast computing due to them being statistical values, enabling the MCU to compute the equations of each channel and extract the features without letting the next dequeue iteration to stack and lose information.
Once the signal has been successfuly registered and its relevant features extracted, the extracted features can now be assembled as an imput matrix for the proposed dynamic RNN model. The proposed model is configured as a Long-Short-Term-Memory Recurrent Neural Network (LSTM RNN), which is used in Deep Learning (DL) to track long sequences of data (such as continuous signal), and process it aided by feedback connections.
The proposed model is presented in the following image, which presents the interconnected layers of the dynamic model, with an input layer representing the features extracted per channel by the number of channels (for this project: number of features x number of channels) and an output layer representing the different possitions considered in the project (labeled from 0 to N). The hidden layers in the model are composed by 2 LSTM layers activated using a tanh function and containing 128 nodes each, and 2 DENSE layers, the first activated by a tanh function and containing 128 nodes, while the second is activated using a softmax function and containing nodes equal to the end number of classes (in this case, 5).
In order to train the proposed model, a set of 81 dual-channel EMG signals, retrieved from 27 subjects aged 19 and 23 years of age, is used for supervised training of the model. Each signal is recorded using the Biopac system, at a 1 kHz frequency, which matches the sampling rate of the project. Each signal contains each of the considered forearm positions, diferentiated by the combined behavior of the EMG channels and separated on 5-second intervals.
Considering that the raw signal obtained in the datasets contains positive and negative voltage values based on the reference point, differing from the Myoware aquisition system, a pre-processing of the signal is needed in order to assimilate it to the Myoware signal. The sample pre-processing consists of a signal square value rectification, followed by a moving average filter. The raw signal is segmented to obtain a regular signal vectors containing only the desired raw signal label. Once the signal is segmented into separate, regular vectors, it is then processed using the method mentioned earlier.
The processed sample is then separated in a similar dequeue fashion in order to assimilate the 100 ms window for feature extraction. The features are then arranged in a feature matrix (m x n x l), which contains m: feature values, n: number of channels and l: target label. The resultant feature matrix is then scrambled, manaining the features with its correspondent label, and then applied to the model in training. The model is trained using a train/test separation rate of 70/30 and the Adam optimizer. The training of the model is determined in 100 epochs, resulting in the training chart below.
Here is a section for possible tasks and future actions for this project, even if it is already published.
Here is a list of things to do in order to complete/repair issues within the app or modules that could be added or upgraded.
- Generate a live-connection to a secondary MCU and apply the control mechanism to perform an end-action
- Migrate the dynamic classifier and MCU programs to a more powerful embedded system (eg. Teensy or Raspberry) compatible with TensorFlow or TensorFlow Lite
- Generate an electrode mechanism to implement EMG detection while ensuring wearability of the EMG module of the project
- Generate the schematics for the hardware modules of the project
- Upgrade the number of channels applied to the project in order to enable real-time classification
Here is a list of upgrades that could make a better experience for the end-user, but do not affect functionality.
- Upgrade the dynamic classifier model in order to use Neural Ordinary Differential Equations (NODE) models, as proposed by (Chen et al., 2018)
- Generate an available database with the desired EMG signals in order to use as training set for the dynamic model
- Generate a GUI app for control settings and customization