Facial/Acoustic Sound Wave And Gesture Detection FASGD-III is a security application that performs the following which are:
- Facial Recognition
- Sound Recognition
- Gesture Recognition
A facial recognition device is a device that takes an image or a video of a human face and compares it to other images in a database.
When building a facial recognition system, several pictures of the subject must have been taken at different angles and stored in a folder called the images dataset.
This dataset which comprises of various images is trained on a deep learning or machine learning classifier to learn a mapping function on relationships between the face.
Once the machine learning classifier learns this mapping function, a serialized model weight file is generated and saved which is used to make live predictions on the individual faces.
Facial recognition system is a complex image-processing problem in real world applications with complex effects of illumination, occlusion and recognition techniques in image analysis. A detection application is used to find position of the faces in a given image, while a recognition algorithm is used to classify given images with known structured properties. This properties are used commonly in most of computer vision applications. Also, recognition application use standard images and detection algorithms to detect faces and extract facial features which include the eyes, eyebrows, nose, and mouth. This features make the algorithm more complicated than a single detection or recognition algorithm.
The first step for a face recognition algorithm is to get an image frame from a camera, perform a face detection and then a recognition process on the grabbed frame.
The input image which is in form of digital data is sent to the face detection classifier or algorithm which is responsible for extracting facial features in the image. Once this is achieved, the next step would be in recognition of the face from the extracted facial embeddings.
Hand gesture recognition is a topic in computer science and language technology with the goal of interpreting human hand gestures via machine learning algorithms. This approach is made using cameras and computer vision algorithms to interpret sign language.
With this camera, one can generate a depth map of what is being seen through the camera at short range, and use the data to approximate a 3D representation of what is being seen.
In the hand gesture recognition, a single camera is used to explore the vision-based application of the security system. This security system would have had the user customize various functions of a number of different hand gestures. For example, the hang gestures functions could be the in following pattern:
- Gesture A: Disable Alarm
- Gesture B: Unlock the security system
- Gesture C: Call the police or Dial 911
Thus, this functions would be executed when a physical implementation is performed.
The hand gesture recognition works when a motion sensor perceives and interprets movements as the basic source of data input.
- Firstly, the camera feeds the image data into a sensing device that is connected to the system.
- Secondly, the system identifies meaningful gestures from an established gesture library where each gesture is matched to a system command by the user.
- Thirdly, the system connects each registered hand gesture, interprets the gesture and uses the library to identify meaningful gestures that match the library. Finally, once the gesture has been interpreted, the system executes the command or function correlated to the specific gesture.
This occurs when the the cameras are closed, therefore the input returns a None type value.
Sound recognition is a technology, which is based on both traditional pattern recognition theories and audio signal analysis methods. Sound recognition technologies contain preliminary data process, feature extraction and classification algorithms. Sound recognition can classify feature vectors. Feature vectors are created as a result of preliminary data processing and linear predictive coding.
Sound recognition technologies are used for:
- Music recognition
- Speech recognition
- Automatic alarm detection and identification for surveillance, monitoring systems, based on the acoustic environment.
- Assistance to disabled or elderly people affected in their hearing capabilities.
- Identifying species of animals such as fish and mammals, e.g in acoustical oceanography
In monitoring and security, an important contribution to alarm detection and alarm verification can be supplied, using sound recognition techniques. In particular, these methods could be helpful for intrusion detection in places like offices, stores, private homes or for the supervision of public premises exposed to person aggression.
Solutions based on a sound recognition technology can offer assistance to disabled and elderly people affected in hearing capabilities, helping them to keep or recover some independence in their daily occupations.
In the security system, the facial recognition model is the first application to perform its functions, because the cameras exposed are always accepting frames from the outside world.
The cameras which are positioned at specific areas, scans the face seen in view to make predictions on if the identified persons matches the trained facec stored in its database.
Thus, if the subject face has been registered on the system like that of the owner and it runs a scan nothing really happens. However, if the subject's face has not been registered on the security system and it runs a scan, this causes the system to either sound an alarm or send notification to the owner's mobile application.
FASGD-III MOBILE: Link
FASGD-III BACKEND: Link
The following packages and modules are needed for the application to work on your system:
- Python 3.6.6
- Anaconda Package Manager
- Face-recognition module
- Flask
- Flask-SQLAlchemy
- Imutils
- Keras
- Librosa (Sound Wave Library)
- Scikit-learn
- Scipy
- Sklearn
- SoundFile
- Tensorflow
- Werkzeug
- Joblib
- FFmpeg
Download Anaconda Package manager and install it on your operating system distribution. After installation, follow the instructions below by typing the commands in your command prompt inorder to set up your conda environment.
In your project directory, you would see a file called ProjectEnvironment.yaml file. The file contains all the modules and packages needed for building your environment.
Open a command prompt or terminal in the working directory for this project and type the following commands.
Install pyaudio and portaudio for your distribution.
- Install pyaudio here: link
- Install portaudio here: link
- Other references for installation: link
- Using Anaconda to install pyaudio: Link
- Real Python explanation for speech recognition: Link
- Pulse Audio access denied: Link
- Installing PulseAudio: Link
- Installing PulseAudio On Ubuntu: Link Link-2
Linux users, also try installing the following dependencies if you get this error:
python3: src/hostapi/alsa/pa_linux_alsa.c:3641: PaAlsaStreamComponent_BeginPolling: Assertion `ret == self->nfds' failed. Aborted
- pa_linux_alsa.c:3636 Assertion ret == self-> ntds' failed: Link
-
Other Errors Occured could include the following:
Andela@team-358:~$ sudo apt-get remove libportaudio2
Andela@team-358:~$ sudo apt-get install libsdl1.2-dev libsdl-image1.2 libsdl-mixer1.2 libsdl-ttf2.0
sudo apt-get install libsdl1.2-dev libsdl-image1.2-dev libsdl-mixer1.2-dev libsdl-ttf2.0-dev
Andela@team-358:~$ sudo apt-get install libasound2-dev
Andela@team-358:~$ git clone -b alsapatch https://github.com/gglockner/portaudio
Andela@team-358:~$ cd portaudio
Andela@team-358:~/portaudio$ ./configure && make
Andela@team-358:~/portaudio$ sudo make install
Andela@team-358:~/portaudio$ sudo ldconfig
Andela@team-358:~/portaudio$ cd ..
Andela@team-358:~$ conda env create --file ProjectEnvironment.yaml
Andela@team-358:~$ conda env list
Andela@team-358:~$ conda info --envs
base * /home/Andela/anaconda3
ml /home/Andela/anaconda3/envs/ml
Andela@team-358:~$ conda activate ml
(ml)Andela@team-358:~$ pip install -r requirement.txt
After setting up your environment and installing the necessary modules and dependencies. Open a terminal window inside the root directory and execute the "app.py" file using this command
(ml)Andela@team-358:~$ python app.py
Once executed, you should see the following message displayed below if working properly.
* Serving Flask app "app" (lazy loading)
* Environment: production
* Debug mode: on
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
| Image | Fullname |
|---|---|
 |
Mbonu Chinedum (Software Engineer) |
 |
Adetunji Oluwaferanmi (Team lead) |
 |
Ugochukuwu Umeoke |
 |
Abiola Fagbemi |
- ANDELA TEAM
- Codecademy
- Google Build For SDG