This project focuses on iris estimation using deep learning techniques. The methodology involves data collection, image augmentation, model training, and real-time iris detection. Below is an overview of the process and methodology used:
- Install Dependencies and Setup
We use TensorFlow for building and training the model, OpenCV for image processing and real-time detection, and Matplotlib for visualization. GPU memory consumption is managed to avoid Out-Of-Memory (OOM) errors.
- Data Collection
Images are collected using a webcam. OpenCV is used to capture images and save them to a designated directory. The captured images serve as the dataset for training and testing the model.
- Data Annotation
The labelme tool is used to annotate the collected images. Annotations include labeling the positions of the left and right eyes in each image. The annotations are saved as JSON files, which contain the coordinates of the labeled keypoints.
- Image Augmentation
Albumentations library is used for image augmentation. This includes operations like random cropping, flipping, brightness/contrast adjustment, gamma correction, and color shifting. Augmentation helps in increasing the diversity of the dataset, which improves the robustness of the model.
- Prepare Dataset
Images and their corresponding labels are loaded and preprocessed. Images are resized and normalized. Datasets for training, validation, and testing are created and batched for efficient processing.
- Model Building and Training
A Convolutional Neural Network (CNN) is built using TensorFlow and Keras. The model architecture includes ResNet152V2 as the backbone for feature extraction, followed by convolutional layers for keypoint regression. The model is compiled with the Adam optimizer and Mean Squared Error loss function. The model is trained on the training dataset, with validation on a separate validation set to monitor performance.
- Model Evaluation
The trained model is evaluated on the test dataset. Loss plots are generated to visualize training and validation loss over epochs. Predictions are made on test images, and the predicted keypoints are visualized to assess the model's performance.
- Real-Time Detection
The trained model is used for real-time iris detection using a webcam. OpenCV captures video frames, processes them, and feeds them to the model. The model predicts the positions of the left and right eyes, which are then displayed on the video feed.