Deep Learning Exercise

This repo is a collection of self-learning materials for Deep Learning. It includes examples, Notebook, books, lectures and suggestions for learning Deep Learning.

* means recommended.

Roles and tasks (First of all)

* Before your started. Pls read AI Career Pathways: Put Yourself on the Right Track, recommended by Andrew NG and YC etc. It will let you know:

1. Roles and tasks in AI ecosystem
2. Your current role
3. Target role (Skill set and tasks)

中文版

Knowledge Required

Basic knowledge and Applications

1. Python and Numpy
2. Basic Neural Network components:
   • Loss functions
   • Layers & operations (FC, Relu, CNN, RNN etc)
   • Optimizations (SGD, adam)
3. Deep Learning Frameworks
   • PyTorch, Keras, TensorFlow
   • BigDL and Analytics Zoo for big data
4. Examples in different areas
   • Computer Vision (CV)
   • Natural Language Processing (NLP)
   • Reinforcement Learning (RL)
   • Generative Adversarial Networks (GAN)
5. Transfer Learning & Fine-tune
   • Computer Vision (CV)
   • Natural Language Processing (NLP)
6. Serving Trained models (Into production)
   • TensorFlow & PyTorch Serving
   • KubeFlow
   • Analytics-Zoo Web & Cluster Serving

After talking with serveral people, who are learning deep learning by themselves. I found that in this stage, learning too much without a correct direction is very inefficient. To avoid going into wrong directions, I highly recommend them to go over Standford CS231n by Feifei Li. This course will give you an overview & basic knowledge of deep learning, and let you know the mapping between problem & solutions.

Tips: Focus on applications and examples. Search with Google, and try several solutions if possible.

How to: Choose a DL framework, and learn with examples. DIY like building with LeGo. At this stage, Kaggle and Google colab will be your best playground.

Advanced knowledge

1. Machine Learning
2. Math:
   • Numerical Computation
   • Line algebra
   • Probability and Information Theory
3. Detailed deep learning knowledge in different areas
   • Computer Vision (CV)
   • Natural Language Processing (NLP)
   • Reinforcement Learning (RL)
4. Algorithms in Deep Learning
5. Efficient methods & hardware for Deep Learning
   • Distall, low-precision, compression & quantization etc
   • Methods for training & inference
   • Hardwares for training & inference

Tips: Pls focus on one of them, i.e., become expert in one area (or a few area) rather than know everything but not good at any of them. Pay more attention on Deep Learning papers rather than blogs.

How to: Choose a most interesting or relevant (with your work) area, and dig in. Then, keep tracking this area and become an expert.

Materials & Resources

Python & Numpy Exercise

1. * CS231n Numpy Tutorial
2. numpy 100 exercises
3. numpy exercises

Kaggle Notebook

1. Alien vs. Predator images Notebook
2. Chest X-Ray Images (Pneumonia) Notebook

Deep Learning Exercise & Examples

Analytics-Zoo Examples

Please refer to OpenVINO™ Toolkit - Open Model Zoo repository for more details.

1. Analytics-Zoo Streaming Object Detection & Text Classifcation Scala Example
2. Analytics-Zoo Streaming Object Detection & Text Classifcation Python Example
3. Analytics-Zoo Image Classification with Redis & Redis Streams
4. Analytics-Zoo OpenVINO ResNet_v1_50 Image Classifcation Scala Example
5. Analytics-Zoo OpenVINO ResNet_v1_50 Image Classifcation Python Example

TensorFlow & Keras Examples

1. Handwritten Recognization: minist
2. TensorFlow pre-trained Image Classification models
3. TensorFlow Image Classification Preprocessing
4. TensorFlow pre-trained Object Detection models
5. TensorFlow IMDB

OpenVINO Examples

OpenVINO optimizes and loads TensorFlow pre-trained models:

1. OpenVINO load TensorFlow resnet_v1_50
2. OpenVINO load TensorFlow inception_v3
3. OpenVINO load TensorFlow vgg_19
4. OpenVINO load TensorFlow mobilenet_v1

PyTorch Examples

1. * DEEP LEARNING WITH PYTORCH: A 60 MINUTE BLITZ
2. PyTorch Tutorials

Blog or Channels

1. Subscribe on deeplearning.ai, it will send blog and recommended paper throught email.
2. Subscribe deep learning related topics on [Medium][https://medium.com/].
3. Colah

Courses & Books

Books

1. * Deep Learning with Python by François Chollet, Notebook
2. * Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville.
3. Neural Networks and Deep Learning by Michael Nielsen.
4. Python for Data Analysis by We McKinney
5. Linear Algebra and Learning from Data by Gilbert Strang
6. Reinforcement Learning: An Introduction by RichardS Sutton
7. Mathematics for Machine Learning
8. Dive into Deep Learning

Courses

1. * Stanford CS231n: Convolutional Neural Networks for Visual Recognition, Spring 2017, Youtube Summary
2. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning, Code and Notebook
3. Google Coding TensorFlow
4. deeplearning.ai
   • Neural Networks and Deep Learning (Course 1 of the Deep Learning Specialization)
   • Improving deep neural networks: hyperparameter tuning, regularization and optimization (Course 2 of the Deep Learning Specialization)
   • Structuring Machine Learning Projects (Course 3 of the Deep Learning Specialization)
   • Convolutional Neural Networks (Course 4 of the Deep Learning Specialization)
5. Stanford CS230: Deep Learning | Autumn 2018
6. Stanford CS229: Machine Learning
7. MIT 18.065 Matrix Methods in Data Analysis, Signal Processing, and Machine Learning, Spring 2018
8. deeplearning.ai
   • Neural Networks and Deep Learning (Course 1 of the Deep Learning Specialization)
   • Improving deep neural networks: hyperparameter tuning, regularization and optimization (Course 2 of the Deep Learning Specialization)
   • Structuring Machine Learning Projects (Course 3 of the Deep Learning Specialization)
   • Convolutional Neural Networks (Course 4 of the Deep Learning Specialization)
9. Stanford CS224U: Natural Language Understanding | Spring 2019
10. Stanford CS224N: Natural Language Processing with Deep Learning | Winter 2019, Youtube
11. Stanford CS234: Reinforcement Learning | Winter 2019, Youtube
12. Essence of linear algebra

Papers

Awesome - Most Cited Deep Learning Papers

Reference

1. Analytics-Zoo
2. TensorFlow
3. OpenVINO
4. Coursea
5. Google Football
6. Apache MXNet
7. Apache Spark
8. deeplearning.ai
9. Keras
10. PyTorch
11. Kaggle
12. Google Colab
13. BigDL
14. OpenVINO Open Model Zoo
15. AI Career Pathways: Put Yourself on the Right Track

Other resources

1. Machine_Learning_Study_Path中文
2. Machine Learning Systems Design
3. Microsoft AI education materials for Chinese students, teachers and IT professionals