This is the final project of the PyTorch Scholarship Challenge by Facebook and Udacity
Going forward, AI algorithms will be incorporated into more and more everyday applications. For example, you might want to include an image classifier in a smart phone app. To do this, you'd use a deep learning model trained on hundreds of thousands of images as part of the overall application architecture. A large part of software development in the future will be using these types of models as common parts of applications
In this project, you'll train an image classifier to recognize different species of flowers. You can imagine using something like this in a phone app that tells you the name of the flower your camera is looking at. In practice you'd train this classifier, then export it for use in your application.
The original repo of this challenge can be found here.
The project was done using Google Colaboratory and its code can be found at the Notebook in this repo.
Throughout the Notebook there are instructions guiding to executing the project.
As the network makes use of a sophisticated deep convolutional neural network the training process is impossible to be done by a common laptop. In order to train your models to your local machine you have three options:
- Cuda: If you have an NVIDIA GPU then you can install CUDA from here.
- Cloud Services: There are many paid cloud services that let you train your models like AWS or Google Cloud.
- Google Colab: Google Colab gives you free access to a tesla K80 GPU for 12 hours at a time. Once 12 hours have ellapsed you can just reload and continue! The only limitation is that you have to upload the data to Google Drive and if the dataset is massive you may run out of space.
Once a model is trained, then a normal CPU can be used to perform predictions and you will have an answer within some seconds.
It was used the Densenet121 pretrained model from torchvision.models using the so-called Transfer Learning. Examples on how to do this can be found at PyTorch's Documentation.
There is a wide selection of hyperparameters avaiable and different configurations.
- Model for transfer learning: densenet121
- Definition of classifier: one hidden layer, followed by a ReLU function with 25% dropout rate as shown below
Sequential(
(fc0): Linear(in_features=2208, out_features=256, bias=True)
(relu0): ReLU()
(drop0): Dropout(p=0.25)
(output): Linear(in_features=256, out_features=102, bias=True)
)
- Batch size: 32
- Criterion: Cross Entropy Loss. The criterion is the method used to evaluate the model fit.
- Optimizer: Adam. The optimizer is the optimization method used to update the weights. More info can be found here
- Scheduler: StepLR. It provides different methods for adjusting the learning rate and step
- Learning Rate: 0.001
- Epochs: 5
Using such hyperparameters, where learning_rate = 0.001, dropout = 0.25, optimizer = optim.Adam(model.classifier.parameters(), learning_rate ) and a scheduler such as lr_scheduler.StepLR(optimizer, step_size=4, gamma=0.1), which decays the learning rate by a factor of 0.1 every 4 seconds, the training process took about 11 minutes with 5 epochs, reaching an accuracy of about 94%.
- PyTorch Scholarship Challenge official website
- FAQs PyTorch Challenge
- Moving Model from Colab to Udacity's workspace
- Deep Learning with PyTorch playlist from Deep Lizard youtube channel.
- Resources library: google spreadsheet with several links to different kinds of DL resources.
- Series of 3 articles exploring the PyTorch project: Part 1, Part 2, Part 3.
- Transfer Learning with Convolutional Neural Networks in PyTorch
- Jupyter Notebook Extensions
- Numpy Arrays: Fast guide about Numpy Arrays
Deploying Deep Learning Models
Python/Machine Learning related articles/links:
- Controlling the Web with Python
- A Complete ML Project Walk-Through in Python: Part One
- Machine Learning Kaggle Competition Part One: Getting Started
- A Data Science Project Walk-Through in Python. (* Not related to PyTorch's challenge though).
- Deploying a Python Web App on AWS
This project is licensed under the terms of the MIT License © Pablo Satler 2018