Welcome to the Convolutional Neural Networks (CNN) project in the AI Nanodegree! In this project, you will learn how to build a pipeline that can be used within a web or mobile app to process real-world, user-supplied images. Given an image of a dog, your algorithm will identify an estimate of the canine’s breed. If supplied an image of a human, the code will identify the resembling dog breed.
Along with exploring state-of-the-art CNN models for classification and localization, you will make important design decisions about the user experience for your app. Our goal is that by completing this lab, you understand the challenges involved in piecing together a series of models designed to perform various tasks in a data processing pipeline. Each model has its strengths and weaknesses, and engineering a real-world application often involves solving many problems without a perfect answer. Your imperfect solution will nonetheless create a fun user experience!
- Machine learning
- Deep learning
- Classification
- Convolutional neural networks
- Regularization
- Feedforward neural networks
- backward propagation
- Activation functions
- Hyper-parameters tuning
- Pytorch
-
Clone the repository and navigate to the downloaded folder.
git clone https://github.com/thomasxmeng/CNN-Dog-Human-detection-and-dogbreed-classification.git cd CNN-Dog-Human-detection-and-dogbreed-classification -
Download the dog dataset. Unzip the folder and place it in the repo, at location
path/to/dog-project/dogImages. ThedogImages/folder should contain 133 folders, each corresponding to a different dog breed. -
Download the human dataset. Unzip the folder and place it in the repo, at location
path/to/dog-project/lfw. If you are using a Windows machine, you are encouraged to use 7zip to extract the folder. -
Make sure you have already installed the necessary Python packages according to the README in the program repository.
-
Open a terminal window and navigate to the project folder. Open the notebook and follow the instructions.
jupyter notebook dog_app.ipynb
NOTE: While some code has already been implemented to get you started, you will need to implement additional functionality to successfully answer all of the questions included in the notebook. Unless requested, do not modify code that has already been included.
NOTE: In the notebook, you will need to train CNNs in PyTorch. If your CNN is taking too long to train, feel free to pursue one of the options under the section Accelerating the Training Process below.
(1) AUGMENT THE TRAINING DATA Augmenting the training and/or validation set might help improve model performance.
(2) TURN YOUR ALGORITHM INTO A WEB APP Turn your code into a web app using Flask!
(3) OVERLAY DOG EARS ON DETECTED HUMAN HEADS Overlay a Snapchat-like filter with dog ears on detected human heads. You can determine where to place the ears through the use of the OpenCV face detector, which returns a bounding box for the face. If you would also like to overlay a dog nose filter, some nice tutorials for facial keypoints detection exist here.
(4) ADD FUNCTIONALITY FOR DOG MUTTS Currently, if a dog appears 51% German Shephard and 49% poodle, only the German Shephard breed is returned. The algorithm is currently guaranteed to fail for every mixed breed dog. Of course, if a dog is predicted as 99.5% Labrador, it is still worthwhile to round this to 100% and return a single breed; so, you will have to find a nice balance.
(5) EXPERIMENT WITH MULTIPLE DOG/HUMAN DETECTORS Perform a systematic evaluation of various methods for detecting humans and dogs in images. Provide improved methodology for the face_detector and dog_detector functions.
If your code is taking too long to run, you will need to either reduce the complexity of your chosen CNN architecture or switch to running your code on a GPU. If you'd like to use a GPU, you can spin up an instance of your own:
You can use Amazon Web Services to launch an EC2 GPU instance.