Permalink
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
518 lines (414 sloc) 38.4 KB

Deep Learning Resources

> Trying to organise the vast majority of    Deep Learning resources that I encounter.    

If you want to contribute, feel free to make a pull request.

The Readme currently gets generated based on the Lnkr API from Zero to Singularity at [https: // lnkr.zerotosingularity.com](https: // lnkr.zerotosingularity.com) which is currently not publicly available yet. Feel free to contact me at jan@zerotosingularity.com if you would like to contribute.

Table of Contents

  1. Aws
  2. Benchmarking
  3. Blogs
  4. Books
  5. Build Your Own Dl Machine
  6. Communities
  7. Conferences
  8. Datasets
  9. Docker
  10. Frameworks
  11. Gan
  12. Github Repositories
  13. Infrastructure
  14. Journalism
  15. Jupyter Notebooks
  16. Learning Rate
  17. Math & Statistics
  18. Media
  19. Miscellaneous
  20. Models
  21. Nlp
  22. Online Courses
  23. Papers
  24. Playgrounds
  25. Python Libraries
  26. Reinforcement Learning
  27. Research
  28. Resnet
  29. Resources From Courses
  30. Tips
  31. Tools
  32. Videos

Aws

  1. awslabs/ec2-spot-labs: Collection of tools and code examples to demonstrate best practices in using Amazon EC2 Spot Instances.

Benchmarking

  1. RedditSota/state-of-the-art-result-for-machine-learning-problems
  2. EpistasisLab/penn-ml-benchmarks: A large, curated repository of benchmark datasets for evaluating supervised machine learning algorithms.
  3. MLPerf
  4. u39kun/deep-learning-benchmark: Deep Learning Benchmark for comparing the performance of DL frameworks, GPUs, and single vs half precision
  5. Benchmarks: Deep Learning Nvidia P100 vs V100 GPU | Xcelerit

Blogs

  1. Zero to Singularity
  2. Google Research
  3. Microsoft ML Blog
  4. Apple ML Blog
  5. Foldl
  6. Jonas Degrave
  7. xzh
  8. Andrew Gibiansky
  9. Otoro
  10. No free hunch - The official Kaggle Blog
  11. Calculated content
  12. deeplearning.net
  13. Andrej Karpathy
  14. Colah
  15. WildML
  16. I am trask
  17. Towards Data Science
  18. Machine Learning, Data Science, Big Data, Analytics, AI
  19. Research Blog
  20. Distill — Latest articles about machine learning
  21. ML⚡️DL — AI to Hell! – Medium
  22. Jay Alammar – Visualizing machine learning one concept at a time
  23. Papers with Code : the latest in machine learning
  24. Tim Dettmers — Making deep learning accessible.

Books

  1. Deep Learning Book - Ian Goodfellow and Yoshua Bengio and Aaron Courville (11/2016)
  2. Neural Networks and Deep Learning - Michael Nielsen (12/2017)
  3. Hands-on Machine Learning with Scikit-Learn and Tensorflow - Aurélien Géron (3/2017)
  4. Manning | Grokking Deep Learning
  5. Manning | Deep Learning with R
  6. Manning | Real-World Machine Learning
  7. Manning | Deep Reinforcement Learning in Action
  8. Manning | Deep Learning and the Game of Go
  9. Manning | GANs in Action
  10. Top 10 Books on NLP and Text Analysis – sciforce – Medium

Build Your Own Dl Machine

  1. Build a Deep Learning Rig for $800
  2. Building your own deep learning box
  3. The $1700 great Deep Learning box: Assembly, setup and benchmarks
  4. Build your 1st Deep Learning Rig
  5. Which GPU(s) to Get for Deep Learning: My Experience and Advice for Using GPUs in Deep Learning (Tim Dettmers)
  6. GPU Benchmark
  7. Choosing Components for Personal Deep Learning Machine
  8. Why building your own Deep Learning computer is 10x cheaper than AWS

Communities

  1. Artificial Intelligence & Deep Learning
  2. Deep Learning
  3. AI & Deep Learning Enthusiasts Worldwide
  4. Artificial Intelligence & Deep Learning - DeepNetGroup
  5. Deep Learning / AI
  6. Self-Driving car with Deep Learning
  7. Artificial intelligence & Deep learning

Conferences

  1. NIPS
  2. ICML
  3. CVPS
  4. ECCV
  5. ICLR - International Conference on Learning Representations
  6. CVPR2018

Datasets

  1. Deeplearning.net / Datasets
  2. Google Street View House Numbers (SVHN)
  3. MNIST
  4. Tiny images
  5. One Hundred Million Creative Commons Flickr Images for Research
  6. Text REtrieval Conference (TREC) Data - English Test Questions (Topics) File List
  7. Translation Task - EMNLP 2015 Tenth Workshop on Statistical Machine Translation
  8. VGG-16
  9. Pretrained CNNs - MatConvNet
  10. Visual Geometry Group: Oxford-IIIT Pet Dataset
  11. fast.ai Datasets
  12. Tencent/tencent-ml-images: Largest multi-label image database; ResNet-101 model; 80.73% top-1 acc on ImageNet

Docker

  1. Repository configuration | nvidia-docker
  2. floydhub/dl-docker: An all-in-one Docker image for deep learning. Contains all the popular DL frameworks (TensorFlow, Theano, Torch, Caffe, etc.)
  3. Explore - Docker Hub
  4. Use nvidia-docker to create awesome Deep Learning Environments for R (or Python) PT I – Kai Lichtenberg
  5. Docker Tutorial 5: Nvidia-Docker 2.0 Installation in Ubuntu 18.04

Frameworks

  1. Tensorflow
  2. Keras
  3. Caffe
  4. Caffe2
  5. CNTK
  6. Theano
  7. PyTorch
  8. Apache MXNet
  9. Chainer
  10. Deeplearning4j
  11. Deeplearn.js
  12. Fast.ai
  13. Lore
  14. Brain.js
  15. XGBoost
  16. Libsvm
  17. SciKit Learn
  18. Gluon
  19. Knet
  20. TensorLayer
  21. Keras-Sharp
  22. Pyro
  23. GitHub - OlafenwaMoses/ImageAI
  24. Darknet: Open Source Neural Networks in C
  25. Create ML | Apple Developer Documentation
  26. apple/turicreate: Turi Create simplifies the development of custom machine learning models.
  27. xmartlabs/Bender: Easily craft fast Neural Networks on iOS! Use TensorFlow models. Metal under the hood.

Gan

  1. Progressive Growing of GANs for Improved Quality, Stability, and Variation | Research
  2. High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
  3. junyanz/CycleGAN: Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.
  4. junyanz/pytorch-CycleGAN-and-pix2pix: Image-to-image translation in PyTorch (e.g. horse2zebra, edges2cats, and more)
  5. junyanz/BicycleGAN: [NIPS 2017] Toward Multimodal Image-to-Image Translation
  6. deepfakes/faceswap: Non official project based on original /r/Deepfakes thread. Many thanks to him!
  7. faceswap-GAN/notes at master · shaoanlu/faceswap-GAN
  8. yunjey/StarGAN: PyTorch Implementation of StarGAN - CVPR 2018
  9. Meow Generator – Alexia Jolicoeur-Martineau
  10. Meow Generator – Alexia Jolicoeur-Martineau
  11. Microsoft researchers build an AI that draws what you tell it to
  12. junyanz/iGAN: Interactive Image Generation via Generative Adversarial Networks
  13. [1701.07875] Wasserstein GAN
  14. Generative Adversarial Networks — A Theoretical Walk-Through.
  15. zi2zi: Master Chinese Calligraphy with Conditional Adversarial Networks
  16. eriklindernoren/Keras-GAN: Keras implementations of Generative Adversarial Networks.
  17. eriklindernoren/PyTorch-GAN: PyTorch implementations of Generative Adversarial Networks.

Github Repositories

  1. Hands-on Machine Learning with Scikit-Learn & Tensorflow
  2. Open.ai gym
  3. deepmind/lab: A customisable 3D platform for agent-based AI research
  4. DeepMind
  5. fast.ai
  6. OpenAI
  7. deep-painterly-harmonization/README.md at master · luanfujun/deep-painterly-harmonization
  8. pytorch/examples: A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.
  9. NervanaSystems/nlp-architect: NLP Architect by Intel AI Lab: Python library for exploring the state-of-the-art deep learning topologies and techniques for natural language processing and natural language understanding
  10. Nervana
  11. zhixuhao/unet: unet for image segmentation
  12. google/python-fire: Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.
  13. google/sentencepiece: Unsupervised text tokenizer for Neural Network-based text generation.
  14. autogenerating test data along with arranging it in the directories that are required for lesson 163
  15. salesforce/awd-lstm-lm: LSTM and QRNN Language Model Toolkit for PyTorch
  16. KaimingHe/deep-residual-networks: Deep Residual Learning for Image Recognition
  17. MichalDanielDobrzanski/DeepLearningPython35: neuralnetworksanddeeplearning.com integrated scripts for Python 3.5.2 and Theano with CUDA support
  18. mnielsen/neural-networks-and-deep-learning: Code samples for my book "Neural Networks and Deep Learning"
  19. deeppomf/DeepLearningAnimePapers: A list of papers and other resources on deep learning with anime style images.
  20. FavoritePapers/image_generation.md at master · SeitaroShinagawa/FavoritePapers
  21. PeterTor/sparse_convolution: sparse convolution Implementation
  22. general-deep-image-completion/README.md at master · adamstseng/general-deep-image-completion
  23. uber/horovod: Distributed training framework for TensorFlow, Keras, and PyTorch.
  24. wookayin/gpustat: 📊 A simple command-line utility for querying and monitoring GPU status
  25. Model Zoo · BVLC/caffe Wiki
  26. ysh329/deep-learning-model-convertor: The convertor/conversion of deep learning models for different deep learning frameworks/softwares.
  27. Microsoft/MMdnn: MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.
  28. ROCm-Developer-Tools/HIP: HIP : Convert CUDA to Portable C++ Code
  29. piiswrong/deep3d: Automatic 2D-to-3D Video Conversion with CNNs
  30. Tencent/tencent-ml-images: Largest multi-label image database; ResNet-101 model; 80.73% top-1 acc on ImageNet
  31. t-SNE – Laurens van der Maaten
  32. TensorLayer Community
  33. PacktPublishing/Deep-Learning-with-Keras: Code repository for Deep Learning with Keras published by Packt
  34. PacktPublishing/Getting-Started-with-TensorFlow: Getting Started with TensorFlow, published by Packt
  35. PacktPublishing/Advanced-NLP-Projects-with-TensorFlow-2.0
  36. A gallery of interesting Jupyter Notebooks · jupyter/jupyter Wiki
  37. facebookresearch/Detectron: FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.
  38. matterport/Mask_RCNN: Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow
  39. facebookresearch/DensePose: A real-time approach for mapping all human pixels of 2D RGB images to a 3D surface-based model of the body
  40. omni-us/squeezedet-keras: Keras implementation of the Squeeze Det Object Detection Deep Learning Framework
  41. dennybritz/reinforcement-learning: Implementation of Reinforcement Learning Algorithms. Python, OpenAI Gym, Tensorflow. Exercises and Solutions to accompany Sutton's Book and David Silver's course.
  42. eriklindernoren/Keras-GAN: Keras implementations of Generative Adversarial Networks.
  43. eriklindernoren/PyTorch-GAN: PyTorch implementations of Generative Adversarial Networks.
  44. eriklindernoren/ML-From-Scratch: Machine Learning From Scratch. Bare bones Python implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from data mining to deep learning.
  45. zaidalyafeai/Notebooks: Machine learning notebooks in different subjects optimized to run in google collaboratory
  46. facebookresearch/DensePose: A real-time approach for mapping all human pixels of 2D RGB images to a 3D surface-based model of the body
  47. NVIDIA/vid2vid: Pytorch implementation of our method for high-resolution (e.g. 2048x1024) photorealistic video-to-video translation.
  48. floydhub/dl-setup: Instructions for setting up the software on your deep learning machine
  49. deep_learning_object_detection/README.md at master · hoya012/deep_learning_object_detection
  50. yunjey/pytorch-tutorial: PyTorch Tutorial for Deep Learning Researchers
  51. cvhciKIT/sloth: Sloth is a tool for labeling image and video data for computer vision research.
  52. awslabs/ec2-spot-labs: Collection of tools and code examples to demonstrate best practices in using Amazon EC2 Spot Instances.
  53. omarsar/nlp_overview: Modern Deep Learning Techniques Applied to Natural Language Processing
  54. CSAILVision/LabelMeAnnotationTool: Source code for the LabelMe annotation tool.

Infrastructure

  1. Paperspace
  2. Crestle
  3. Microsoft Azure
  4. AWS Amazon
  5. Google Compute Engine
  6. Infrastructure for Deep Learning

Journalism

  1. Hello World - Bloomberg

Jupyter Notebooks

  1. Deep Learning with Python Notebooks
  2. Jupyter Docker Deploy

Learning Rate

  1. Finding Good Learning Rate and The One Cycle Policy.
  2. [1803.09820] A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
  3. Another data science student's blog – The 1cycle policy

Math & Statistics

  1. Hands-on Machine Learning with Scikit-Learn and Tensorflow - Aurélien Géron (3/2017)
  2. Probability and Statistics for Programmers
  3. Probabilistic Programming & Bayesian Methods for Hackers
  4. Understanding Machine Learning: from Theory to Algorithms
  5. Elements of statistical learning
  6. An introduction to statistical learning
  7. Foundations of data science
  8. A programmer's guide to data Mining
  9. Mining massive datasets
  10. Machine learning yearning

Media

  1. Transforming Standard Video Into Slow Motion with AI - NVIDIA Developer News CenterNVIDIA Developer News Center
  2. NVIDIA has Open Sourced an Impressive Video to Video Translation Technique

Miscellaneous

  1. ec2-spotter
  2. Python styleguide
  3. Cycle-gan
  4. Some snippets

Models

  1. ModelDepot

Nlp

  1. The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time
  2. NLP's ImageNet moment has arrived
  3. The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time
  4. Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) – Jay Alammar – Visualizing machine learning one concept at a time
  5. Dissecting BERT Part 1: The Encoder – Dissecting BERT – Medium
  6. Dissecting BERT Appendix: The Decoder – Dissecting BERT – Medium
  7. The fall of RNN / LSTM – Towards Data Science
  8. A Brief Overview of Attention Mechanism – SyncedReview – Medium
  9. The Unreasonable Effectiveness of Recurrent Neural Networks
  10. Google AI Blog: Transformer: A Novel Neural Network Architecture for Language Understanding
  11. The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time

Online Courses

  1. Machine Learning- Coursera - Andrew Ng
  2. Deep Learning Specialization - Coursera
  3. Advanced Machine Learning Specialization - Coursera
  4. Fast.ai
  5. Deep Learning Udacity
  6. MIT 6.S094: Deep Learning for Self-Driving Cars
  7. 6.S191: Introduction to Deep Learning
  8. CS231n: Convolutional Neural Networks for Visual Recognition
  9. Google's Machine learning crash course
  10. Google.ai
  11. Machine Learning with TensorFlow on Google Cloud Platform | Coursera
  12. Machine Learning Crash Course  |  Google Developers
  13. Self-Driving Car | Udacity
  14. MIT 6.S094: Deep Learning for Self-Driving Cars
  15. AI School
  16. Reviews for Machine Learning from Coursera | Class Central
  17. Reviews for 6.S191: Introduction to Deep Learning from Massachusetts Institute of Technology | Class Central
  18. Salesforce Einstein Discovery - Easy AI and Machine Learning | Udemy
  19. Artificial Intelligence (AI) | edX
  20. Artificial Intelligence A-Z™: Learn How To Build An AI | Udemy
  21. Lecture Collection | Convolutional Neural Networks for Visual Recognition (Spring 2017) - YouTube
  22. CS 188: Introduction to Artificial Intelligence, Fall 2018
  23. MIT 6.S094: Deep Learning for Self-Driving Cars

Papers

  1. Deep Learning Papers Reading Roadmap
  2. Machine Theory of Mind - DeepMind
  3. [1801.06146] Universal Language Model Fine-tuning for Text ... - arXiv
  4. A disciplined approach to neural network hyper-parameters: Part 1 ...
  5. Visualizing and Understanding Convolutional Networks
  6. Cyclical Learning Rates for Training Neural Networks
  7. 1704.00109.pdf
  8. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
  9. Deep Video Portraits
  10. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
  11. A guide to convolution arithmetic for deep learning
  12. Generative Reversible Networks
  13. Patch-Based Image Inpainting with Generative Adversarial Networks
  14. 1712.00080.pdf
  15. NVIDIA SPLATNet Research Paper Wins a Major CVPR 2018 Award - NVIDIA Developer News CenterNVIDIA Developer News Center
  16. Adversarial Attacks on Face Detectors using Neural Net based Constrained Optimization
  17. 1803.09820.pdf
  18. Show, Attend and Tell: Neural Image CaptionGeneration with Visual Attention
  19. 1512.03385.pdf
  20. https://arxiv.org/pdf/1712.09913.pdf
  21. https://arxiv.org/pdf/1603.07285.pdf
  22. [1505.04597] U-Net: Convolutional Networks for Biomedical Image Segmentation
  23. [1603.05027] Identity Mappings in Deep Residual Networks
  24. [1712.09913] Visualizing the Loss Landscape of Neural Nets
  25. [1810.04805v1] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  26. [1803.09820] A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay

Playgrounds

  1. Tensorflow Playground
  2. ConvNetJS
  3. Google Collab
  4. Azure Notebooks
  5. Seedbank

Python Libraries

  1. Numpy
  2. Pandas: Python Data Analysis Library
  3. seaborn: statistical data visualization
  4. Scikit-Learn
  5. PDL - Python Download library
  6. Pyro

Reinforcement Learning

  1. Reinforcement Learning from scratch – Insight Data
  2. Spinning Up in Deep RL
  3. dennybritz/reinforcement-learning: Implementation of Reinforcement Learning Algorithms. Python, OpenAI Gym, Tensorflow. Exercises and Solutions to accompany Sutton's Book and David Silver's course.

Research

  1. Facebook AI | Tools | Open Source Deep Learning Tools
  2. Facebook AI Research – Facebook Research
  3. AI & Deep Learning Publications & Researchers | NVIDIA Research
  4. Berkeley Artificial Intelligence Research Lab
  5. Artificial Intelligence Archives - NVIDIA Developer News CenterNVIDIA Developer News Center

Resnet

  1. http://arxiv.org/abs/1512.03385
  2. blog
  3. code
  4. http://image-net.org/challenges/LSVRC/2015/
  5. COCO
  6. https://github.com/KaimingHe/resnet-1k-layers
  7. VGG-16
  8. Training and investigating Residual Nets
  9. Deep Residual Learning for Image Recognition
  10. ResNet training in Torch
  11. Deep Residual learning - paper implementation
  12. implementation of the deep residual network used for cifar10
  13. Residual networks in torch (MNIST 100 layers)
  14. NOAA Right Whale Recognition, Winner's Interview
  15. Using neon for Scene Recognition: Mini-Places2
  16. Matlab (MatConvNet) implementation of "Deep Residual Learning for Image Recognition"
  17. imagenet-resnet.py
  18. keras-resnet
  19. ImageNet ILSVRC classification
  20. ResNet in TensorFlow

Resources From Courses

  1. Fast.ai - Lesson 1
  1. Fast.ai - Lesson 2
  1. Fast.ai - Lesson 3
  1. Fast.ai - Lesson 4
  1. Fast.ai - Lesson 5
  1. Fast.ai - Lesson 6

Tips

  1. 28 Jupyter Notebook tips, tricks, and shortcuts
  2. Lesser known ways of using Jupyter Notebooks

Tools

  1. NVIDIA Container Runtime and Orchestrators | NVIDIA Developer
  2. Introducing Apex: PyTorch Extension with Tools to Realize the Power of Tensor Cores - NVIDIA Developer News CenterNVIDIA Developer News Center
  3. Download DeepStream SDK 2.0 Today to Develop Scalable Video Analytics Applications - NVIDIA Developer News CenterNVIDIA Developer News Center
  4. Announcing NVIDIA DALI and NVIDIA nvJPEG - NVIDIA Developer News CenterNVIDIA Developer News Center
  5. NVIDIA Releases TensorRT 4 - NVIDIA Developer News CenterNVIDIA Developer News Center
  6. lutzroeder/netron: Visualizer for deep learning and machine learning models

Videos

  1. (11) Transfer Learning for RL tasks via GANs - Road Fighter - YouTube