This is the Deep Learning part of the Unsupervised Learning Course of the Master in High-Performance Computing (SISSA/ICTP)
First Part
- Artificial neural networks
- Train, validate and test a deep learning model
- Convolutional neural networks
- Elementary aspects of unsupervised deep learning models
Second Part
TO APPEAR
- Alessio Ansuini (First Part)
- Alberto Cazzaniga (Second Part)
Institute: Research and Technology Institute, AREA Science Park)
Day 1
- The artificial neuron
- Possiblities and limitations of a single neuron
- Linear layer
- Non-linearities
- Fully connected architectures
- Softmax layer
- Cross-entropy loss and the MLE principle
Sources (see below): Michael Nielsen's online book, PyTorch Tutorials
Day 2
- Stochastic gradient descent
- Optimization
- Regularization
- Data augmentation
Sources: Michael Nielsen's online book, PyTorch Tutorials
Day 3
- Convolutional networks basics
- Transfer learning
Michael Nielsen's online book, image kernels, PyTorch Tutorials
Day 4
TO APPEAR
Day 5
TO APPEAR
TO APPEAR
There are excellent free resources to deepen your knowledge on topics such as Deep Learning, Reinforcement Learning and more in general Artificial Intelligence.
Here is a selection of very good ones.
Books for free
-
Michael Nielsen
-
The Deep Learning Book
https://www.deeplearningbook.org/
in a single pdf version
-
Information Theory, Pattern Recognition, and Neural Networks (Dave McKay)
-
Probabilistic Machine Learning (Kevin Murphy)
-
Dive into Deep Learning (Amazon group)
Courses for free
-
Fast AI (Jeremy Howards)
Invaluable resource for quickly getting your hands dirt into practical Deep Learning
-
Deep learning specialization (auditing is for free) on Coursera (Andrew Ng).
One of the best resources to learn basic and intermediate concepts.
(Check the Coursera website for other resources: auditing is sometimes for free, certificates are generally not.)
-
Deep unsupervised learning (Pieter Abbeel)
A glimpse into state-of-the-art research problems.
-
Deep reinforcement learning (Dave Silver)
The legendary course of Dave Silver on YouTube
https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLzuuYNsE1EZAXYR4FJ75jcJseBmo4KQ9-
-
Neural network class (Hugo Larochelle)
After almost 10 years still a very useful resource: crystal clear explanations of an impressive amount of topics, starting from the very basics (I used this a lot!)
https://www.youtube.com/watch?v=SGZ6BttHMPw&list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH
-
Information Theory, Pattern Recognition, and Neural Networks (Dave McKay) (See also the accompanying book)
The Lectures of Dave MacKay will accompany you to the study of its beautiful book: on of the most precious resources you will find on this topic.
Information theory is very relevant in many fields, and particularly in Unsupervised Deep Learning.
https://www.youtube.com/watch?v=BCiZc0n6COY&list=PLruBu5BI5n4aFpG32iMbdWoRVAA-Vcso6
Websites and Blogs
-
Deepmind
-
OpenAI
YouTube channels
-
Yannick Kilcher's channel
-
Two minutes papers