Mini Course in Deep Learning with PyTorch for AIMS
The African Masters of Machine Intelligence (AMMI) is Africa's flagship program in machine intelligence led by The African Institute for Mathematical Sciences (AIMS). These lessons, developed during the course of several years while I've been teaching at Purdue and NYU, are here proposed for the AMMI (AIMS).
Prior to this course delivered for AMMI (AIMS), an earlier version of this was delivered and video-recorded for the Computational and Data Science for High Energy Physics (CoDaS-HEP) summer school at Princeton University. Please refer to this version release here.
Table of contents
T: theory (slides and animations)
P: practice (Jupyter Notebooks)
TLearning paradigms: supervised-, unsupervised-, and reinforcement-learning
PGetting started with the tools: Jupyter notebook, PyTorch tensors and autodifferentiation
T+PNeural net's forward and backward propagation for classification and regression
T+PConvolutional neural nets improve performance by exploiting data nature
T+PFoundations of Salsa
T+PRecurrent nets natively support sequential data
T+PUnsupervised learning: vanilla and variational autoencoders, generative adversarial nets
THow to create and deliver an effective presentation
T+PRegularization for neural nets
Sessions and relative material
- Time slot 1 (4h + 4h)
Topics: 1, 2, 3.
Slides: 01 - ML and spiral classification.
Notebooks: 01, 02, 03, 04, 05.
- Time slot 2 (4h + 2h)
Slides: 02 - CNN.
Notebooks: 06, 07.
- Time slot 3 (2h)
Slides: 03 - Salsa.
- Time slot 4 (4h + 4h)
Slides: 04 - RNN.
Code Readings: Word Language Model.
Assignment: HW1, HW1 Solutions.
Notebooks: 08, 09.
- Time slot 5 (4h + 4h)
Slides: 05 - Generative models.
Code Readings: GAN.
Guides: TikZ Quick Guide.
Notebooks: 10, 11.
- Time slot 6 (1h)
Slides: 06 - How to present.
Video: How to prepare a presentation.
- Time slot 7 (1h)
Slides: 07 - Regularisation.
Jupyter Notebooks are used throughout these lectures for interactive data exploration and visualisation.
We use dark styles for both GitHub and Jupyter Notebook. You should try to do the same, or they will look ugly. JupyterLab has a built-in selectable dark theme, so you only need to install something if you want to use the classic notebook interface. To see the content appropriately in the classic interface install the following:
- Jupyter Notebook dark theme;
- GitHub dark theme and comment out the
invert #fff to #181818code block.
Keeping in touch
For more educational materials you also can head to Ritchie's website.
To be able to follow the workshop exercises, you are going to need a laptop with Miniconda (a minimal version of Anaconda) and several Python packages installed. Following instruction would work as is for Mac or Ubuntu linux users, Windows users would need to install and work in the Gitbash terminal.
Download and install Miniconda
Please go to the Anaconda website. Download and install the latest Miniconda version for Python 3.6 for your operating system.
wget <http:// link to miniconda> sh <miniconda .sh>
After that, type:
and read the manual.
Check-out the git repository with the exercise
Once Miniconda is ready, checkout the course repository and and proceed with setting up the environment:
git clone https://github.com/Atcold/PyTorch-Deep-Learning-Minicourse
If you do not have git and do not wish to install it, just download the repository as zip, and unpack it:
wget https://github.com/Atcold/PyTorch-Deep-Learning-Minicourse/archive/master.zip #For Mac users: #curl -O https://github.com/Atcold/PyTorch-Deep-Learning-Minicourse/archive/master.zip unzip master.zip
Create isolated Miniconda environment
Change into the course folder, then type:
#cd PyTorch-Deep-Learning-Minicourse conda env create -f environment.yml source activate dl-minicourse
Install Autocomplete by hinterland
You have to run the following commands if you want auto-complete.
pip install jupyter_contrib_nbextensions pip install jupyter_nbextensions_configurator jupyter contrib nbextension install --user cd /usr/local/miniconda3/envs/dl-minicourse/lib/python3.6/site-packages/jupyter_contrib_nbextensions/nbextensions jupyter nbextension install hinterland jupyter nbextension enable hinterland/hinterland
Start Jupyter notebook or JupyterLab
If you are working in a JupyterLab container double click on "Files" tab in the upper right corner.
Locate first notebook, double click to open.
Do not attempt to start
jupyter from the terminal window.
If working on a laptop, start from terminal as usual:
Or, for the classic interface:
More PyTorch Resources
If you would like more PyTorch resources, head over to the global community-maintained repository of hundreds of reliable implementations and guides at the following repository created by Ritchie Ng: The Incredible PyTorch.