Glove Mouse and Keyboard (Homepage)
- A glove mouse that can replace your physical mouse and keyboard.
- It is able to control the cursor, left click, right click, scroll up and down, and predict the alphabets from the gesture.
- The glove mouse uses Bluetooth to pass the signals to computer
- Deep learning based alphabet classifier
The purpose of this device is to provide all users with a keyboard and mouse that can be used in their most comfortable posture. It solves the problem of poor posture caused by computer users who have long placed their hands and shoulders in line with the table and the keyboard.
| scroll up and down | right click and save image |
| 
|
| writing "rib" | writing "beef" |
| 
|
Connect the sensor on the sensor list below to the given pin and use arduino.ino to modify some parameter such as pressure sensivity.
- Collect the data from each sensor and transfer the data to computer using bluetooth.
- Collect the angular accleration of Accelerometer on the index finger
- Train the GRU model to predict the alphabet from angular accleration
- Use the data passing from glove mouse to control the mouse and input the words.
- Tranform the time domain data to frequency domain and apply dimension reduction methods(PCA/LDA) to 25 dimensions, use Support vector machine/Random forest to predict the alphabet.
Gate recurrent neural network use the angular velocity of the three-axis as input to predict the alphabets written by users.
- FSR400 : Force Sensing Resistor
- BH-37 : bending sensor
- LSM9DS0 : Accelerometer + Gyro + Magnetometer
- HC-05 : Bluetooth to Serial Port Module
## Machine learning based classifier (new) To alleviate the effort of training a GRU model, we also use PCA/LDA + Random forest/Support vector machine to classify the 26 alphabets. The model reads the frequency domain data converted from Fast Fourier transform. It achieves high accuracy (97.5% in test dataset) when PCA+SVM is used.
