First steps in deep learning

Here we gather definitions and links related to basic machine and deep learning ideas and technologies.

Machine and deep learning ideas

Machine learning

Machine learning deals with algorithms which are able to learn from data without being explicitly programmed. To learn more about machine learning and its specific ideas like train and test set, loss and metric, overfitting and regularization, check:

• A visual introduction to machine learning by R2D3
• Courses by OpenDataScience
• Machine learning course on Coursera by Andrew Ng
• Machine learning course on YouTube by Yaser Abu-Mostafa

Deep learning

Deep learning is, in a nutshell, machine learning based on artificial neural networks. To learn more about deep learning and its specific ideas like various kinds of neural network layers, optimizers, batch size, epoch, data augmentation, dropout and batch normalization, check:

• Deep learning course on Coursera by Andrew Ng
• CS231n course by Stanford University
• Courses by fast.ai
• Starting deep learning hands-on: image classification on CIFAR-10 by Piotr Migdał
• Other blog posts by deepsense.ai
• Feature visualization by Distill

Also, if you want to be up-to-date with the newest deep learning ideas and achievements, track posts on platforms like:

• reddit.com/r/machinelearning
• kaggle.com

Technological stack

Python

Python is the main programming language of machine and deep learning solutions worldwide. To learn more about Python, check:

• Python webpage
• Any of great Python courses available in Web

Minerva code is written in Python 3.5.

Jupyter notebook

Jupyter notebook is a web application allowing you to work with the code in an aesthetic interactive way. To learn more about Jupyter notebook, check:

• Jupyter webpage
• 28 Jupyter Notebook tips, tricks, and shortcuts by Josh Devlin
• 10 things you really should know about jupyter notebooks by Jakub Czakon

TensorFlow

TensorFlow is a deep learning framework by Google, nowadays the most popular one. To learn more about TensorFlow, check:

• TensorFlow webpage

Keras

Keras is an API on the top of TensorFlow which radically simplifies the work with TensorFlow-backend code. To learn more about Keras, check:

• Keras webpage
• Keras blog
• Blog post about differences between Keras’ sequential and functional API by Jovian Lin

PyTorch

PyTorch is a deep learning framework by Facebook, which offers creating and training neural networks in a more direct way than TensorFlow. To learn more about PyTorch, check:

• PyTorch webpage

• Blog post on differences between Keras and PyTorch by deepsense.ai

• Blog post on transfer learning with Keras and PyTorch by deepsense.ai

• Blog posts on differences between TensorFlow and PyTorch, for example these three:

  • by Kirill Dubovikov
  • by Akarsh Singh
  • by Nico Jimenez

  However, bear in mind that Pytorch develops very quickly, so even half-a-year-old posts can be a bit outdated.

Neptune

Neptune is a machine learning lab which we created to make data scientists’ life easier. To learn more about Neptune, check:

• Neptune webpage
• Blog posts by deepsense.ai
• Open starters for kaggle competitions Toxic Comment Classification Challenge and 2018 Data Science Bowl based on Neptune.