Machine Learning Curriculum

Machine Learning is a branch of Artificial Intelligence dedicated at making machines learn from observational data without being explicitly programmed.

Machine learning and AI are not the same. Machine learning is an instrument in the AI symphony — a component of AI. So what is Machine Learning — or ML — exactly? It’s the ability for an algorithm to learn from prior data in order to produce a behavior. ML is teaching machines to make decisions in situations they have never seen.

This curriculum is made to guide you to learn machine learning, recommend tools, and help you to embrace ML lifestyle by suggesting media to follow. I update it regularly to maintain freshness and get rid of outdated content and deprecated tools.

Machine Learning in General

Study this section to understand fundamental concepts and develop intuitions before going any deeper.

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

• Elements of AI A bunch of easy courses teaching AI and machine learning
• COMS W4995 Applied Machine Learning Spring 2020 videos and slides of an applied ML course from Columbia University.
• Opinionated Machine Learning Course by Fast.ai
• Machine Learning Crash Course with TensorFlow APIs Google's fast-paced, practical introduction to machine learning
• Artificial Intelligence, Revealed a quick introduction by Yann LeCun, mostly about Machine Learning ideas, Deep Learning, and convolutional neural network
• Intro to Machine Learning - Udacity hands on scikit-learn (python) programming learning on core ML concepts
• Machine Learning: Supervised, Unsupervised & Reinforcement - Udacity the 2 instructors are hilarious
• Machine Learning Mastery carefully laid out step-by-step guide to some particular algorithms
• Andrew Ng's Course on Coursera recommended for people who want to know the details of ML algorithms under the hood, understand enough maths to be dangerous and do coding assignments in Octave programming language (Note: Andrew said that you do not need to know Calculus beforehand but he talked about it quite often so you will regret if you don't know Calculus 😆)
• ML Recipes - YouTube Playlist a really nicely designed concrete actionable content for ML introduction
• Machine Learning is Fun Part 1 simple approach to machine learning for non-maths people
• Machine Learning with Python - YouTube Playlist
• Machine Learning Yearning by Andrew Ng
• Machine Learning Crash Course: Part 1
• Rules of Machine Learning: Best Practices for ML Engineering
• Most Shared Machine Learning Content on Twitter For The Past 7 Days
• MIT 6.S099: Artificial General Intelligence This class takes an engineering approach to exploring possible research paths toward building human-level intelligence.
• Landing A Data Job: The Course is an opinionated and practical guideline for those who wants to become data scientists quickly. For example, they suggest that knowing how decision tree works is already good enough, you don't need to know how all the models work, which is true!

Books

• Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition by Aurélien Géron is the best selling book because it's great.
• https://github.com/fastai/fastbook The fastai book, published as Jupyter Notebooks

Reinforcement Learning

Building a machine that senses the environment and then chooses the best policy (action) to do at any given state to maximize its expected long-term scalar reward is the goal of reinforcement learning.

• OpenAI Spinning Up This is an educational resource produced by OpenAI that makes it easier to learn about deep reinforcement learning (deep RL).
• Basic Reinforcement Learning An introduction series to Reinforcement Learning (RL) with comprehensive step-by-step tutorials.
• Advanced Topics: RL 2015 (COMPM050/COMPGI13) by David Silver (the guy behind AlphaGo)
• An Introduction Book by Richard S. Sutton and Andrew G. Barto
• Deep Reinforcement Learning: Pong from Pixels
• Lecture 10: Reinforcement Learning - YouTube
• A Survey Paper
• Deep Reinforcement Learning: A Tutorial - OpenAI
• CS 294: Deep Reinforcement Learning, Spring 2017

Deep Learning

Deep learning is a branch of machine learning where deep artificial neural networks (DNN) — algorithms inspired by the way neurons work in the brain — find patterns in raw data by combining multiple layers of artificial neurons. As the layers increase, so does the neural network’s ability to learn increasingly abstract concepts.

The simplest kind of DNN is a Multilayer Perceptron (MLP).

• Full Stack Deep Learning Learn Production-Level Deep Learning from Top Practitioners
• DeepLearning.ai a bunch of courses taught by Andrew Ng at Coursera; It's the sequel of Machine Learning course at Coursera.
• Advanced Machine Learning Specialization consists of 7 courses on Coursera
• A friendly introduction to Deep Learning and Neural Networks
• A Neural Network Playground Tinker with a simple neural network designed to help you visualize the learning process
• Deep Learning Demystified - Youtube explain inspiration of deep learning from real neurons to artificial neural networks
• Learn TensorFlow and deep learning, without a Ph.D. This 3-hour course (video + slides) offers developers a quick introduction to deep-learning fundamentals, with some TensorFlow thrown into the bargain.
• A Guide to Deep Learning by YN^2 a curated maths guide to Deep Learning
• Practical Deep Learning For Coders Course at Fast.ai taught by Jeremy Howard (Kaggle's #1 competitor 2 years running, and founder of Enlitic)
• Deep learning - Udacity recommended for visual learner who knows some ML, this course provides high level ideas of deep learning, dense intuitive details put in a short amount of time, you will use TensorFlow inside the course
• Deep Learning Resources (Papers, Online Courses, Books) - deeplearning4j.org
• Introduction to Deep Neural Networks - deeplearning4j.org
• NVIDIA Deep Learning Institute because GPU are efficient at training Neural Networks, NVIDIA notices this market !
• Deep Learning Book recommended for math nerds who want to understand the theoretical side, the book is crafted by our deep learning wizards (Goodfellow, Bengio and Courville)
• CS224d: Deep Learning for Natural Language Processing
• Deep Learning Summer School, Montreal 2015
• Neural networks class - YouTube Playlist
• http://deeplearning.net/
• http://neuralnetworksanddeeplearning.com/index.html a hands-on online book for deep learning maths intuition, I can say that after you finish this, you will be able to explain deep learning in a fine detail.
• https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow-i You will implement a lot of things inside TensorFlow such as Autoencoders, Convolutional neural net, Feedforward neural nets, Generative models (Generative Adversarial Networks, Recurrent networks), visualizing the network, etc. You will have lots of assignments to finish. The course director (Parag) is also approachable and active.
• 6.S094: Deep Learning for Self-Driving Cars a course at MIT
• The Neural Network Zoo a bunch of neural network models that you should know about (I know about half of them so don't worry that you don't know many because most of them are not popular or useful in the present)
• 6.S191: Introduction to Deep Learning a course for 2017
• The GAN Zoo a list of GAN papers which have their own name
• https://deeplearning.mit.edu/ MIT Deep Learning taught by Lex Fridman
• Intro to TensorFlow for Deep Learning taught at Udacity
• Advancing AI theory with a first-principles understanding of deep neural networks we use deep learning successfully for quite a long time but we don't know exactly why it works and how to really improve it from the ground up. This blog post contains a link to the paper that explains the first-principles attempt of understanding neural networks so that we can advance deep learning field further.

Convolutional Neural Networks

DNNs that work with grid data like sound waveforms, images and videos better than ordinary DNNs. They are based on the assumptions that nearby input units are more related than the distant units. They also utilize translation invariance. For example, given an image, it might be useful to detect the same kind of edges everywhere on the image. They are sometimes called convnets or CNNs.

• CS231n: Convolutional Neural Networks for Visual Recognition a course taught at Stanford university
• How Convolutional Neural Networks work - Youtube technical explanation including pooling operations, ReLU, fully connected layer, optimization using gradient descent
• Neural Network that Changes Everything - Computerphile
• A Beginner's Guide To Understanding Convolutional Neural Networks
• Deep Learning for Computer Vision (Andrej Karparthy, OpenAI) this is my most favorite video of convolutional net. Andrej explains convnet in detail answering all the curious questions that one might have. For example, most articles only talk about convolution in grayscale image, but he describe convolution in images with color channels as well. He also talks about the concerns and the assumptions that convnets make. This is a great lecture!
• Understanding Neural Networks Through Deep Visualization explains how to visualize a convnet using various techniques
• Convolutional Neural Networks Tutorial in TensorFlow gives an introduction to CNNs for beginners in TensorFlow
• Capsule Networks (CapsNets) – Tutorial CapsNets are a hot new architecture for neural networks, invented by Geoffrey Hinton, one of the godfathers of deep learning.

Recurrent Neural Networks

DNNs that have states. They also understand sequences that vary in length. They are sometimes called RNNs.

• http://karpathy.github.io/2015/05/21/rnn-effectiveness/
• http://colah.github.io/posts/2015-08-Understanding-LSTMs/
• http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/

Best Practices

• How to debug neural networks. Manual.
• An overview of gradient descent optimization algorithms

Unsupervised Domain Adaptation

Unsupervised Domain Adaptation is a type of Transfer Learning that applies a model that was trained on source dataset to do well on a target dataset without any label on the target dataset. It's one of the technique that is practically useful in the real world when the cost of labeling target dataset is high. One of the example is to train a model on synthetic data with label and try to use it on real data without label.

• https://paperswithcode.com/task/unsupervised-domain-adaptation
• https://github.com/zhaoxin94/awsome-domain-adaptation#unsupervised-da
• https://github.com/barebell/DA
• https://github.com/bbdamodaran/deepJDOT one of the easy-to-use implementation on keras of a unsupervised domain adaptation technique that I tried before and work great

Tools

Libraries and frameworks that are useful for practical machine learning

Frameworks

Machine learning building blocks

• scikit-learn (Python) general machine learning library, high level abstraction, geared towards beginners
• TensorFlow (Python); Awesome TensorFlow; computation graph framework built by Google, has nice visualization board, probably the most popular framework nowadays for doing Deep Learning
• Keras: Deep Learning library for Theano and TensorFlow (Python)
• PyTorch (Python) PyTorch is a deep learning framework that puts Python first.
• Apache MXNet (incubating) for Deep Learning Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to mix symbolic and imperative programming to maximize efficiency and productivity.
• Chainer (Python) A flexible framework of neural networks for deep learning
• DeepLearning4j (Java) Model import deployment framework for retraining models (pytorch, tensorflow,keras) deploying in JVM Micro service environments, mobile devices, iot, and Apache Spark
• Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning. There is a specific focus on reinforcement learning with several contextual bandit algorithms implemented and the online nature lending to the problem well.
• H2O is an in-memory platform for distributed, scalable machine learning.
• spektral Graph Neural Networks with Keras and Tensorflow 2.

No coding

• Lobe a drag-and-drop tool for machine learning
• Ludwig Ludwig is a toolbox that allows users to train and test deep learning models without the need to write code. It is built on top of TensorFlow.

Gradient Boosting

Models that are used heavily in competitions because of their outstanding generalization performance.

• https://github.com/dmlc/xgboost eXtreme Gradient Boosting, not actually AutoML but it is quite popular among Kaggle competitors
• https://github.com/microsoft/LightGBM lightweight alternative compared to xgboost
• https://github.com/catboost/catboost A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.
• https://github.com/tensorflow/decision-forests TensorFlow Decision Forests (TF-DF) is a collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models.
• PyTorch/TensorFlow implementation of TabNet paper. Further read: TabNet balances explainability and model performance on tabular data, but can it dethrone boosted tree models?

Time Series Inference

Time series data require unique feature extraction process for them to be usable in most machine learning models because most models require data to be in a tabular format. Or you can use special model architectures which target time series e.g. LSTM, TCN, etc.

• https://github.com/timeseriesAI/tsai Time series Timeseries Deep Learning Pytorch fastai - State-of-the-art Deep Learning with Time Series and Sequences in Pytorch / fastai. Further read: tsai — State of the Art Machine Learning for Time Series, Part 1.
• https://github.com/alan-turing-institute/sktime A unified framework for machine learning with time series
• https://github.com/sktime/sktime-dl An extension package for deep learning with Tensorflow/Keras for sktime
• https://github.com/tslearn-team/tslearn/ A machine learning toolkit dedicated to time-series data
• https://github.com/blue-yonder/tsfresh Automatic extraction of relevant features from time series
• https://github.com/johannfaouzi/pyts A Python package for time series classification
• https://github.com/facebook/prophet Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.
• https://github.com/philipperemy/keras-tcn Keras Temporal Convolutional Network
• ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels; MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification; these 2 techniques are for extracting time series features. Further read: ROCKET: Fast and Accurate Time Series Classification

Life Cycle

Libraries that help you develop/debug/deploy the model in production (MLOps). There is more to ML than training the model.

• https://github.com/allegroai/clearml Auto-Magical Suite of tools to streamline your ML workflow. Experiment Manager, ML-Ops and Data-Management
• https://github.com/quantumblacklabs/kedro A Python framework for creating reproducible, maintainable and modular data science code.
• https://github.com/determined-ai/determined Determined is an open-source deep learning training platform that makes building models fast and easy. I use it mainly for tuning hyperparameters.
• https://github.com/iterative/cml Continuous Machine Learning (CML) is an open-source library for implementing continuous integration & delivery (CI/CD) in machine learning projects. Use it to automate parts of your development workflow, including model training and evaluation, comparing ML experiments across your project history, and monitoring changing datasets.
• https://github.com/creme-ml/creme Python library for online machine learning. All the tools in the library can be updated with a single observation at a time, and can therefore be used to learn from streaming data.
• https://github.com/aimhubio/aim A super-easy way to record, search and compare 1000s of ML training runs
• https://github.com/Netflix/metaflow Metaflow is a human-friendly Python library that helps scientists and engineers build and manage real-life data science projects. Metaflow was originally developed at Netflix.
• MLflow MLflow (currently in beta) is an open source platform to manage the ML lifecycle, including experimentation, reproducibility and deployment. It currently offers three components: MLflow Tracking, MLflow Projects, MLflow Models.
• FloydHub a Heroku for Deep Learning (You focus on the model, they'll deploy)
• comet.ml Comet enables data scientists and teams to track, compare, explain and optimize experiments and models across the model's entire lifecycle. From training to production
• https://neptune.ai/ Manage all your model building metadata in a single place
• https://wandb.ai/site Build better models faster with experiment tracking, dataset versioning, and model management
• https://github.com/fastai/nbdev Create delightful python projects using Jupyter Notebooks
• https://rapids.ai/ data science on GPUs
• https://github.com/datarevenue-berlin/OpenMLOps
• https://github.com/jacopotagliabue/you-dont-need-a-bigger-boat Not really a tool, but a guide on how to compose many tools together in real-world reasonable scale business.

Data Storage

• https://github.com/activeloopai/Hub Fastest dataset optimization and management for machine and deep learning. Stream data real-time & version-control it.
• https://github.com/determined-ai/yogadl Better approach to data loading for Deep Learning. API-transparent caching to disk, GCS, or S3.
• https://github.com/google/ml_collections ML Collections is a library of Python Collections designed for ML use cases. It contains ConfigDict, a "dict-like" data structures with dot access to nested elements. It is supposed to be used as a main way of expressing configurations of experiments and models.

Data Wrangling

Data cleaning and data augmentation

• https://github.com/cgnorthcutt/cleanlab Clean labeling error of dataset
• https://github.com/aleju/imgaug Image augmentation library which supports converting keypoints to heatmaps
• https://github.com/albu/albumentations Fastest image augmentation library
• https://github.com/mdbloice/Augmentor Easy-to-use image augmentation for classification tasks (cannot augment keypoints)
• https://github.com/facebookresearch/AugLy A data augmentations library for audio, image, text, and video.

Data Orchestration

• https://github.com/PrefectHQ/prefect
• https://github.com/dagster-io/dagster
• https://github.com/ploomber/ploomber Ploomber is the fastest way to build data pipelines ⚡️. Use your favorite editor (Jupyter, VSCode, PyCharm) to develop interactively and deploy ☁️ without code changes (Kubernetes, Airflow, AWS Batch, and SLURM).
• https://github.com/orchest/orchest Build data pipelines, the easy way using user-friendly UI

Data Visualization

• https://github.com/streamlit/streamlit Streamlit turns data scripts into shareable web apps in minutes. All in Python. All for free. No front‑end experience required.
• https://github.com/oegedijk/explainerdashboard Quickly build Explainable AI dashboards that show the inner workings of so-called "blackbox" machine learning models.
• https://github.com/lux-org/lux By simply printing out a dataframe in a Jupyter notebook, Lux recommends a set of visualizations highlighting interesting trends and patterns in the dataset.
• https://github.com/slundberg/shap A game theoretic approach to explain the output of any machine learning model.

Hyperparameter Tuning

Before you begin, please read this blog post to understand the motivation of searching in general: https://www.determined.ai/blog/stop-doing-iterative-model-development

Open your eyes to search-driven development. It will change you. Main benefit is that there will be no setbacks. Only progress and improvement are allowed. Imagine working and progressing everyday, instead of regressing backwards because your new solution doesn't work. This guaranteed progress is what search-driven development will do to you. Apply it to everything in optimization, not just machine learning.

My top opinionated preferences are determined, ray tune, and optuna because of parallelization (distributed tuning on many machines), flexibility (can optimize arbitrary objectives and allow dataset parameters to be tuned), library of SOTA tuning algorithms (e.g. HyperBand, BOHB, TPE, PBT, ASHA, etc), result visualization/analysis tools, and extensive documentations/tutorials.

• https://github.com/determined-ai/determined Determined is an open-source deep learning training platform that makes building models fast and easy.
• https://docs.ray.io/en/master/tune/index.html Ray Tune is a Python library for experiment execution and hyperparameter tuning at any scale. If you are looking for distributed tuning, Ray Tune is probably the most serious framework out there.
• https://github.com/optuna/optuna an automatic hyperparameter optimization software framework (framework agnostic, define-by-run)
• https://github.com/keras-team/keras-tuner an easy-to-use, distributable hyperparameter optimization for keras; read its article here
• https://github.com/autonomio/talos Hyperparameter Optimization for Keras, TensorFlow (tf.keras) and PyTorch
• https://github.com/maxpumperla/hyperas (deprecated) Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization
• https://github.com/fmfn/BayesianOptimization A Python implementation of global optimization with gaussian processes.
• https://github.com/hyperopt/hyperopt
• https://github.com/msu-coinlab/pymoo Multi-objective Optimization in Python

AutoML

Make machines learn without the tedious task of feature engineering, model selection, and hyperparameter tuning that you have to do yourself. Let the machines perform machine learning for you!

Personally if I have a tabular dataset I would try FLAML and mljar first, especially if you want to get something working fast. If you want to try gradient boosting frameworks such as XGBoost, LightGBM, CatBoost, etc but you don't know which one works best, I suggest you to try AutoML first because internally it will try the gradient boosting frameworks mentioned previously.

• Best OpenSource AutoML frameworks in 2021 an article on medium containing a curated list of OpenSource AutoML frameworks.
• https://github.com/dabl/dabl Data Analysis Baseline Library; quickly train a simple model to be used as a performance baseline
• https://www.automl.org/ Find curated list of AutoML libraries and researches
• https://github.com/jhfjhfj1/autokeras As of writing (24 August 2018), this library is pretty premature as it can only does classification.
• https://github.com/automl/auto-sklearn/ Does not run on Windows, you need to install WSL (Windows Subsystem for Linux) to use it
• https://github.com/EpistasisLab/tpot Run thousands of machine learning pipelines and output the code for you
• https://github.com/ClimbsRocks/auto_ml Read what the author think about the comparison between tpot and auto-sklearn
• https://github.com/microsoft/FLAML Fast and Lightweight AutoML with cost-effective economical optimization algorithms.
• https://github.com/mljar/mljar-supervised an Automated Machine Learning Python package that works with tabular data. I like that it generates visualization report (in the Explain mode) and extra features for you e.g. golden features and K-means features.
• https://github.com/awslabs/autogluon AutoML for Text, Image, and Tabular Data. But it doesn't support Windows (as of 11 October 2021).
• https://github.com/AutoViML/Auto_ViML Auto_ViML was designed for building High Performance Interpretable Models with the fewest variables needed.

Model Architectures

Architectures that are state-of-the-art in its field.

• https://modelzoo.co/ Model Zoo
• https://github.com/tensorflow/models
• Magenta: Music and Art Generation with Machine Intelligence
• https://github.com/phillipi/pix2pix Image-to-image translation using conditional adversarial nets; TensorFlow port of pix2pix; Watch the presentation of this work: Learning to see without a teacher
• wav2letter Facebook AI Research's Automatic Speech Recognition Toolkit
• https://github.com/huggingface/transformers State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch

Interesting Techniques & Applications

• 2020: A Year Full of Amazing AI Papers — A Review
• https://paperswithcode.com/ A list of papers with evaluation metrics, and state of the art comparison.
• http://deeplearninggallery.com/ Deep Learning Gallery - a curated list of awesome deep learning projects
• How do GANs intuitively work? this is my article explaining GANs, I try to be as intuitive as possible, GANs are so awesome that I can't just ignore and not talk about it. You can also watch the official tutorial by Ian Goodfellow.

Nice Blogs & Vlogs to Follow

• https://www.pyimagesearch.com/ Frequently updated blog about OpenCV and deep learning
• http://colah.github.io/ nice explanations with beautiful visualizations
• https://karpathy.github.io/ He has taught some courses on Deep Nets and is working at Tesla as the AI director.
• http://ruder.io Sebastian Ruder's Deep Learning and NLP blog
• http://www.wildml.com/
• https://machinelearningmastery.com/ Contains a lot of content and beautiful website
• Sirajology's YouTube Playlists lots of dense short hilarious introduction to ML
• Two Minute Papers on Deep Learning Playlist
• Distill.pub A modern medium for presenting research in Machine Learning
• deeplearn.org Deep Learning Monitor; news about deep learning papers and tweets
• datarevenue blog about MLOps mostly

Impactful People

• Geoffrey Hinton, he has been called the godfather of deep learning by introducing 2 revolutionizing techniques (ReLU and Dropout) with his students. These techniques solve the Vanishing Gradient and Generalization problem of deep neural networks. He also taught a Neural Networks course at Coursera.
• Yann LeCun, he invented CNNs (Convolutional neural networks), the kind of network that is really popular among computer vision developers today
• Yoshua Bengio another serious professor at Deep Learning, you can watch his TEDx talk here (2017)
• Andrew Ng he discovered that GPUs make deep learning faster. He taught 2 famous online courses, Machine Learning and Deep Learning specialization at Coursera.
• Juergen Schmidhuber invented LSTM (a particular type of RNN)
• Jeff Dean, a Google Brain engineer, watch his TEDx Talk
• Ian Goodfellow, he invented GANs (Generative Adversarial Networks), is an OpenAI engineer
• David Silver this is the guy behind AlphaGo and Artari reinforcement learning game agents at DeepMind
• Demis Hassabis CEO of DeepMind, has given a lot of talks about AlphaGo and Reinforcement Learning achievements they have
• Andrej Karparthy he teaches convnet classes, wrote ConvNetJS, and produces a lot of content for DL community, he also writes a blog (see Nice Blogs & Vlogs to Follow section)
• Pedro Domingos he wrote the book The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World, watch his TEDx talk here

Cutting-Edge Research Publishers

Steal the most recent techniques introduced by smart computer scientists (could be you).

• http://www.arxiv-sanity.com/ Arxiv Sanity Preserver
• https://research.facebook.com/ai/
• http://research.google.com/pubs/MachineIntelligence.html
• https://deepmind.com/research/ Research of DeepMind company
• https://www.openai.com/
• https://www.openai.com/requests-for-research/
• State of the art performance on each ML task
• State-of-the-art result for all Machine Learning Problems

Practitioner Community

• https://www.kaggle.com
• https://gym.openai.com
• https://universe.openai.com/
• /r/MachineLearning
• https://www.facebook.com/groups/DeepNetGroup/

Thoughtful Insights for Future Research

• Why AI is Harder Than We Think
• The Consciousness Prior by Yoshua Bengio
• What Can't Deep Learning Do? a list of problems that deep learning faces
• Pedro Domingos: "The Master Algorithm" - Talks at Google
• The AI Revolution: The Road to Superintelligence
• https://ai100.stanford.edu/2016-report
• Why does Deep Learning work so well? - The Extraordinary Link Between Deep Neural Networks and the Nature of the Universe
• These are three of the biggest problems facing today's AI
• Four Questions For: Geoff Hinton Geoff Hinton is referred to as "godfather of neural networks"
• What product breakthroughs will recent advances in deep learning enable? - Quora
• Liquid Neural Networks

Uncategorized

• Artificial Intelligence: A Modern Approach (Online Book)
• The Principles of Modern Game AI
• Scipy Lecture Notes
• https://www.youtube.com/user/aicourses
• The Fundamentals of Neuroscience learn how our brain works so that you can discover new deep learning breakthrough
• Bayesian Methods for Hackers An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

Other Big Lists

• https://github.com/collections/machine-learning
• https://github.com/topics/machine-learning
• https://github.com/topics/mlops
• https://github.com/GokuMohandas/MadeWithML Join 30K+ developers in learning how to responsibly deliver value with ML.
• https://papers.labml.ai/papers/daily
• https://github.com/alirezadir/Production-Level-Deep-Learning
• https://github.com/jindongwang/transferlearning
• https://github.com/kmario23/deep-learning-drizzle
• http://aireads.top AI/ML Reads - a directory of artificial intelligence/machine learning resources
• https://github.com/ZuzooVn/machine-learning-for-software-engineers
• https://github.com/josephmisiti/awesome-machine-learning
• https://github.com/ujjwalkarn/Machine-Learning-Tutorials
• https://github.com/terryum/awesome-deep-learning-papers
• https://github.com/ChristosChristofidis/awesome-deep-learning
• https://github.com/Developer-Y/cs-video-courses#machine-learning
• Open Source Deep Learning Curriculum an open source deep learning curriculum which contains a lot of course recommendations for you to consider enrolling. (Many recommendations are the same as in this list)
• Deep Learning Resources by Jeremy D. Jackson
• https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap
• http://aimedicines.com/2017/03/17/all-ai-resources-at-one-place/
• A Large set of Machine Learning Resources for Beginners to Mavens
• Skynet Today Accessible and informed coverage of the latest AI hype and panic
• https://github.com/aikorea/awesome-rl Awesome Reinforcement Learning
• https://github.com/artix41/awesome-transfer-learning Awesome Transfer Learning

I am confused, too many links, where do I start?

If you are a beginner and want to get started with my suggestions, please read this issue: offchan42#4

Disclaimer

From now on, this list is going to be compact and opinionated towards my own real-world ML journey and I will put only content that I think are truly beneficial for me and most people. All the materials and tools that are not good enough (in any aspect) will be gradually removed, including:

• too difficult materials without much intuition; impractical content
• too much theory without real-world practice
• unstructured materials
• courses that I don't consider to enroll myself
• information overload content, unclear direction
• tools that are too specialized and not many people can use it in their works e.g. deepdream (because you can Google it if you want to use it in your work)
• tools that are beaten by other tools; not being state-of-the-art anymore
• commercial tools that look like it can die any time soon

NOTE: There is no particular rank for each link. The order in which they appear does not convey any meaning and should not be treated differently.

How to contribute to this list

1. Fork this repository, then apply your change.
2. Make a pull request and tag me if you want.
3. That's it. If your edition is useful, I'll merge it.

Or you can just submit a new issue containing the resource you want me to include if you don't have time to send a pull request.

The resource you want to include should be free to study.

---