Skip to content

Implementation of Deep Learning Algorithms and useful information (courses, books, videos, etc.)

License

Notifications You must be signed in to change notification settings

ElizaLo/Deep-Learning

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Hits

This repository contains examples of deep learning algorithms implemented in Python with mathematics behind them being explained.

🎓 University Courses

🔹 Coursera Courses

List of Coursera Courses

📚 Books

List of Books

Papers

Title Description, Information
Deep Learning Papers Reading Roadmap Deep Learning papers reading roadmap for anyone who are eager to learn this amazing tech!

Videos

Other useful links

:octocat: GitHub Repositories

Title Description, Information
NVIDIA Deep Learning Examples for Tensor Cores Deep Learning Examples

Contests

Other useful links

📌 Other

Other useful links
  • Caffe – a fast open framework for deep learning;
  • Deep Learning - Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016);
  • Deep Learning от Google — короткий курс для продвинутых. Основное внимание уделяется библиотеке для глубинного обучения TensorFlow;
  • Deep Learning at Oxford (2015) – a YouTube playlist with lectures (read more);
  • awesome-deep-vision – a curated list of deep learning resources for computer vision;
  • awesome-deep-learning-papers – a curated list of the most cited deep learning papers (since 2010);
  • Deep Learning Tutorials – notes and code;
  • dl-docker – an all-in-one Docker image for deep learning. Contains all the popular DL frameworks (TensorFlow, Theano, Torch, Caffe, etc.);
  • Self-Study Courses for Deep Learning от NVDIA — self-paced classes for deep learning that feature interactive lectures, hands-on exercises, and recordings of the office hours Q&A with instructors. You’ll learn everything you need to design, train, and integrate neural network-powered artificial intelligence into your applications with widely used open-source frameworks and NVIDIA software. During the hands-on exercises, you will use GPUs and deep learning software in the cloud;
  • deep-rl-tensorflow - ensorFlow implementation of Deep Reinforcement Learning papers;
  • TensorFlow 101 – Tensorflow tutorials;
  • Introduction to Deep Learning for Image Recognition – this notebook accompanies the Introduction to Deep Learning for Image Recognition workshop to explain the core concepts of deep learning with emphasis on classifying images as the application;

Main skills required by the Deep Learning Engineer / Deep Learning Research Engineer

The research made by Faculty of Applied Sciences at UCU. Link on main article.

Deep Learning Engineer / Deep Learning Research Engineer

  1. Python3: numpy, scikit-learn, pandas, scipy.
  2. Statistics (regression, properties of distributions, statistical tests, and proper usage, etc.) and probability theory.
  3. Deep learning frameworks: Tensorflow, PyTorch; MxNet, Caffe, Keras.
  4. Deep learning architectures: VGG, ResNet, Inception, MobileNet.
  5. Deepnets, hyperparameter optimization, visualization, interpretation.
  6. Machine learning models.

Python for Deep Learning and Research

  • Basic algorithms and common tasks
  • Classical algorithms
  • Computational complexity
  • Useful Libraries and Frameworks
  • CPU vs GPU parallelization
  • Cloud and GPU Integration
  • Data Visualization
  • Vectors and Vectorization
  • Image Processing
  • Language Processing

Mathematics for Deep Learning

  • Common Notation and Core Ideas
  • Linear Algebra
  • N-dim Spaces
  • Vectors, Matrices and Operators
  • Mathematical and Function Analysis calculus
  • Derivative and Partial derivative
  • Chain Rule
  • Probability theory
  • Introduction to Statistics

Linear, Polynomial and Multivariate Regression

  • Price prediction Task
  • Linear Regression
  • Least square method
  • Loss Function
  • Optimization Task
  • Gradient Descent
  • MLE — Maximum Likelihood Estimation
  • Data Preprocessing
  • Model Visualization
  • Data Normalization
  • Polynomial Regression
  • Multivariate Regression

Introduction Computer Vision

  • Basic idea of Computer Vision
  • Classical Computer Vision
  • Deep Learning and CV
  • Core Idea of Semantic Gap
  • Classification Task
  • N-dim Spaces and Metrics
  • Common datasets
  • Mnist and Fashion-Mnist
  • Cifar10 and Cifar100
  • Cats vs Dogs
  • ImageNet and MS COCO
  • Euclidean Distance
  • Nearest Neighbour

Classification and Computer Vision

  • Image Classification
  • Cosine Similarity
  • Manhattan distance
  • KNN
  • Train / Val / Test data split
  • Logistic Regression
  • Logistic Regression and Maximum Likelihood Estimation
  • Loss function and Cross Entropy
  • Accuracy and Metrics
  • Precision, Recall and F1

Neural Networks

  • Rosenblatt’s Perceptron
  • Artificial Neuron
  • Warren McCulloch and Walter Pitts Neuron
  • Fully Connected (Linear, Dense, Affine) Layer
  • Activation Layers
  • BackPropagation Algorithm
  • Stochastic Gradient Descent
  • Biological Neuron and Analogy

Computation graphs and Deep Learning Frameworks

  • Computational graphs
  • Differentiable graphs
  • Deep Learning Frameworks
  • Custom Framework Realization
  • Linear operations and Activation Realizations
  • Main Blocks Of Deep Learning FrameWorks
  • Custom Model and Train
  • Optimizator realization
  • TensorFlow
  • Keras
  • PyTorch

Deep Learning

  • Neural Networks Problems
  • Activation Functions
  • Weights Initialization
  • Initialization Techniks
  • Overfitting and Underfitting
  • Regularization Methods
  • L1 and L2 Regularization
  • Ensemble of Models
  • Dropout
  • Hyper Parameters Search
  • Optimizations behind SGD
  • Momentum and Nesterov Momentum
  • Adagrad, RMSprop
  • Adam, Nadam
  • Batch-Normalization

Unsupervised Learning

  • Dimensionality reduction
  • Feature Learning
  • Vector Representation
  • Embeddings
  • Kernel Method
  • Clusterization
  • k-means Clusterization
  • Hierarchical Clusterization
  • Neural Networks and Unsupervised Learning
  • Autoencoders
  • Autoencoders architectures
  • Tasks for Autoencoders
  • Problem of Image Generation
  • Image Denoising Task

Introduction to Deep Learning in Computer Vision

  • Problems of Fully Connected Neural Networks
  • Towards Convolution Neural Network
  • CNN as feature extractor
  • Computer Vision tasks
  • Transfer Learning
  • Transfer Learning in Practice
  • What Next (breath: CNN Architectures, Image Detection, Segmentation, GANs)

Introduction to Natural Language Processing

  • Introduction to Natural Language Processing
  • Text classification
  • Words Preprocessing and Representation
  • Part-of-Speech tagging (PoS tagging)
  • Tokenization, Lemmatization and Stemming
  • Bag of Words
  • TF-IDF
  • Distributive semantics
  • Vector Semantics
  • Term-document matrix
  • Word context matrix
  • Dense Vectors and Embeddings
  • Word2Vec
  • What Next (breath: RNN, Seq2Seq, Attention, Transformers, Modern Language Models)