learning

Learning Philosophy:

• The Power of Tiny Gains
• Master Adjacent Disciplines
• T-shaped skills
• Data Scientists Should Be More End-to-End
• Just in Time Learning

Index

• Have basic business understanding
• Be able to frame an ML problem
• Be familiar with data ethics
• Be able to import data from multiple sources
• Be able to setup data annotation efficiently
• Be able to manipulate data with Numpy
• Be able to manipulate data with Pandas
• Be able to manipulate data in spreadsheets
• Be able to manipulate data in databases
• Be able to use Linux tools
• Be able to perform feature engineering
• Be able to experiment in a notebook
• Be able to visualize data
• Be able to do literature review using research papers
• Be able to model problems mathematically
• Be able to structure machine learning projects
• Be able to version control code
• Be able to version control data
• Be familiar with fundamentals of ML and DL
• Be able to implement models in scikit-learn
• Be able to implement models in Tensorflow and Keras
• Be able to implement models in PyTorch
• Be able to implement models using cloud services
• Be able to apply unsupervised learning algorithms
• Be able to implement NLP models
• Be familiar with Recommendation Systems
• Be able to implement computer vision models
• Be able to model graphs and network data
• Be able to implement models for timeseries and forecasting
• Be familiar with Reinforcement Learning
• Be able to optimize performance metric
• Be familiar with literature on model interpretability
• Be able to optimize model size for production
• Be able to write unit tests
• Be able to serve models via APIs
• Be able to build interactive UI for models
• Be able to deploy model to production
• Be able to perform load testing
• Be able to perform A/B testing
• Be proficient in Python
• Be familiar with compiled languages
• Have a general understanding of other parts of the stack
• Be familiar with fundamental Computer Science concepts
• Be able to apply proper software engineering process
• Be able to efficiently use a text editor
• Be able to communicate and collaborate well
• Be familiar with the hiring pipeline
• Broaden Perspective

Have basic business understanding

• Book: Delivering Happiness
• Book: Good to Great: Why Some Companies Make the Leap...And Others Don't
• Book: Hello, Startup: A Programmer's Guide to Building Products, Technologies, and Teams
• Book: How Google Works
• Book: Learn to Earn: A Beginner's Guide to the Basics of Investing and Business
• Book: Rework
• Book: The Airbnb Story
• Book: The Personal MBA
• Facebook: Digital marketing: get started
• Facebook: Digital marketing: go further
• Google Analytics for Beginners
• Moz: The Beginner's Guide to SEO
• Smartly: Marketing Fundamentals
• Treehouse: SEO Basics
• Udacity: App Monetization
• Udacity: App Marketing
• Udacity: Get Your Startup Started
• Udacity: How to Build a Startup
• Youtube: SEO Unlocked
  • Welcome to the SEO Unlocked 0:10:09
  • Introduction to SEO and Why It's Important 0:10:29
  • Keyword Research Part 1 0:19:20
  • Keyword Research Part 2 0:09:56
  • On-page and technical SEO Part 1 0:22:58
  • On-page and technical SEO Part 2 0:12:16
  • Mastering Technical SEO Audits 0:16:35
  • Content Marketing Part 1 0:24:09
  • Advanced Content Marketing Tactics 0:09:54
  • The 10 Commandments of Content Marketing 0:19:01
  • How to Edit Your Content For SEO 0:10:59
  • Discover Your Competitive Strategy 0:09:12
  • Over 4 Million Backlinks Built With This Simple Process 0:11:09
  • How to Get POWERFUL Backlinks for Faster Rankings 0:09:40
  • Get THOUSANDS of Backlinks On Semi-Autopilot 0:06:32
  • How To Get The Most Out Of Google Analytics 0:07:45
  • How to Setup Google Search Console 0:09:21
  • How to Use Advanced Features in Google Analytics 0:10:52
  • A Deep Dive Into Branding, Data & Experience 0:14:03
  • How To Create A Compelling Brand 0:05:52
  • Designing Your Customer Experience & Case Studies 0:07:32

Be able to frame an ML problem

• AWS: Types of Machine Learning Solutions
• Article: Apply Machine Learning to your Business
• Article: Resilience and Vibrancy: The 2020 Data & AI Landscape
• Article: Software 2.0
• Article: How Facebook uses super-efficient AI models to detect hate speech
• Article: Highlights from ICML 2020
• Article: Recent Advances in Google Translate
• Article: A Peek at Trends in Machine Learning
• Article: How to deliver on Machine Learning projects
• Article: Data Science as a Product
• Article: Customer service is full of machine learning problems
• Article: Choosing Problems in Data Science and Machine Learning
• Article: Why finance is deploying natural language processing
• Article: Cannes: How ML saves us $1.7M a year on document previews
• Article: Machine Learning @ Monzo in 2020
• Book: AI Superpowers: China, Silicon Valley, and the New World Order
• Book: A Human's Guide to Machine Intelligence
• Book: The Future Computed
• Book: Machine Learning Yearning by Andrew Ng
• Book: Prediction Machines: The Simple Economics of Artificial Intelligence
• Book: Building Machine Learning Powered Applications: Going from Idea to Product
• Coursera: AI For Everyone
• Datacamp: Case Studies in Statistical Thinking
• Datacamp: Data Science for Everyone
• Datacamp: Machine Learning with the Experts: School Budgets
• Datacamp: Machine Learning for Everyone
• Datacamp: Analyzing Police Activity with pandas
• Datacamp: HR Analytics in Python: Predicting Employee Churn
• Datacamp: Predicting Customer Churn in Python
• Datacamp: Data Science for Managers
• Facebook: Field Guide to Machine Learning
• Google: Introduction to Machine Learning Problem Framing
• Pluralsight: How to Think About Machine Learning Algorithms
• State of AI Report 2020
• Udacity: Problem Solving with Advanced Analytics
• Youtube: Vincent Warmerdam: The profession of solving (the wrong problem) | PyData Amsterdam 2019
• Youtube: Making Money from AI by Predicting Sales - Jay's Intro to AI Part 2
• Youtube: How does YouTube recommend videos? - AI EXPLAINED! 0:33:53
• Youtube: How does Google Translate's AI work? 0:15:02
• Youtube: Data Science in Finance 0:17:52
• Youtube: The Age of AI
  • How Far is Too Far? | The Age of A.I. 0:34:39
  • Healed through A.I. | The Age of A.I. 0:39:55
  • Using A.I. to build a better human | The Age of A.I. 0:44:27
  • Love, art and stories: decoded | The Age of A.I. 0:38:57
  • The 'Space Architects' of Mars | The Age of A.I. 0:30:10
  • Will a robot take my job? | The Age of A.I. 0:36:14
  • Saving the world one algorithm at a time | The Age of A.I. 0:46:37
  • How A.I. is searching for Aliens | The Age of A.I. 0:36:12
• Youtube: Gradient Dissent Podcast
  • DeepChem creator Bharath Ramsundar on using deep learning for molecules and medicine discovery 0:55:11
  • ML Research and Production Pipelines with Chip Huyen 0:43:07
  • Product Management for AI with Peter Skomoroch 1:28:14
  • Slow down and change one thing at a time - Advancing AI research with Josh Tobin 0:48:19
  • Societal Impacts of Artificial Intelligence with Miles Brundage 1:02:25
  • Deep Reinforcement Learning and Robotics with Peter Welinder 0:54:22
  • Machine learning across industries with Vicki Boykis 0:34:02
  • Designing ML models for millions of consumer robots - Angela Bassa and Danielle Dean 0:52:38
  • Building trustworthy AI systems and combating potential malicious use – A conversation w/ Jack Clark 0:55:56
  • Rachael Tatman - Conversational A.I. and Linguistics 0:36:51
  • Nicolas Koumchatzky - Machine Learning in Production for Self Driving Cars 0:44:56
  • Brandon Rohrer - Machine Learning in Production for Robots 0:34:31
• Youtube: Accuracy as a Failure
• Youtube: Using Intent Data to Optimize the Self-Solve Experience
• Youtube: Trillions of Questions, No Easy Answers: A (home) movie about how Google Search works
• Youtube: Hugging Face, Transformers | NLP Research and Open Source | Interview with Julien Chaumond
• Youtube: Vincent Warmerdam - Playing by the Rules-Based-Systems | PyData Eindhoven 2020
• Youtube: Google Machine Learning System Design Mock Interview
• Youtube: Netflix Machine Learning Mock Interview: Type-ahead Search
• Youtube: Machine Learning design: Search engine for Q&A
• Youtube: Engineering Systems for Real-Time Predictions @DoorDash

Be familiar with data ethics

• Article: How to Detect Bias in AI
• Netflix: Coded Bias
• Netflix: The Great Hack
• Netflix: The Social Dilemma
• Practical Data Ethics
  • Lesson 1: Disinformation
  • Lesson 2: Bias & Fairness
  • Lesson 3: Ethical Foundations & Practical Tools
  • Lesson 4: Privacy and surveillance
  • Lesson 4 continued: Privacy and surveillance
  • Lesson 5.1: The problem with metrics
  • Lesson 5.2: Our Ecosystem, Venture Capital, & Hypergrowth
  • Lesson 5.3: Losing the Forest for the Trees, guest lecture by Ali Alkhatib
  • Lesson 6: Algorithmic Colonialism, and Next Steps

Be able to import data from multiple sources

• Docs: Beautiful Soup Documentation
• Datacamp: Importing Data in Python (Part 2)
• Datacamp: Web Scraping in Python

Be able to setup data annotation efficiently

• Article: Create A Synthetic Image Dataset — The “What”, The “Why” and The “How”
• Article: We need Synthetic Data
• Article: Weak Supervision for Online Discussions
• Youtube: Snorkel: Dark Data and Machine Learning - Christopher Ré
• Youtube: Training a NER Model with Prodigy and Transfer Learning
• Youtube: Training a New Entity Type with Prodigy – annotation powered by active learning
• Youtube: ECCV 2020 WSL tutorial: 4. Human-in-the-loop annotations

Be able to manipulate data with Numpy

• Article: A Visual Intro to NumPy and Data Representation
• Article: NumPy Illustrated: The Visual Guide to NumPy
• Article: NumPy Fundamentals for Data Science and Machine Learning
• Datacamp: Intro to Python for Data Science
• Pluralsight: Working with Multidimensional Data Using NumPy

Be able to manipulate data with Pandas

• Article: Visualizing Pandas' Pivoting and Reshaping Functions
• Article: A Gentle Visual Intro to Data Analysis in Python Using Pandas
• Article: Comprehensive Guide to Grouping and Aggregating with Pandas
• Article: 8 Python Pandas Value_counts() tricks that make your work more efficient
• Datacamp: pandas Foundations
• Datacamp: Pandas Joins for Spreadsheet Users
• Datacamp: Manipulating DataFrames with pandas
• Datacamp: Merging DataFrames with pandas
• Datacamp: Data Manipulation with pandas
• Datacamp: Optimizing Python Code with pandas
• Datacamp: Streamlined Data Ingestion with pandas
• Datacamp: Analyzing Marketing Campaigns with pandas
• edX: Implementing Predictive Analytics with Spark in Azure HDInsight
• Article: Modern Pandas
  • Modern Pandas (Part 1)
  • Modern Pandas (Part 2)
  • Modern Pandas (Part 3)
  • Modern Pandas (Part 4)
  • Modern Pandas (Part 5)
  • Modern Pandas (Part 6)
  • Modern Pandas (Part 7)
  • Modern Pandas (Part 8)

Be able to manipulate data in spreadsheets

• Datacamp: Spreadsheet basics
• Datacamp: Data Analysis with Spreadsheets
• Datacamp: Intermediate Spreadsheets for Data Science
• Datacamp: Pivot Tables with Spreadsheets
• Datacamp: Data Visualization in Spreadsheets
• Datacamp: Introduction to Statistics in Spreadsheets
• Datacamp: Conditional Formatting in Spreadsheets
• Datacamp: Marketing Analytics in Spreadsheets
• Datacamp: Error and Uncertainty in Spreadsheets
• edX: Analyzing and Visualizing Data with Excel

Be able to manipulate data in databases

• Codecademy: SQL Track
• Datacamp: Intro to SQL for Data Science
• Datacamp: Introduction to MongoDB in Python
• Datacamp: Intermediate SQL
• Datacamp: Exploratory Data Analysis in SQL
• Datacamp: Joining Data in PostgreSQL
• Datacamp: Querying with TransactSQL
• Datacamp: Introduction to Databases in Python
• Datacamp: Reporting in SQL
• Datacamp: Applying SQL to Real-World Problems
• Datacamp: Analyzing Business Data in SQL
• Datacamp: Data-Driven Decision Making in SQL
• Datacamp: Database Design
• Udacity: SQL for Data Analysis
• Udacity: Intro to relational database
• Udacity: Database Systems Concepts & Design

Be able to use Linux tools

• Article: Streamline your projects using Makefile
• Calmcode: makefiles
• Calmcode: entr
• Codecademy: Learn the Command Line
• Datacamp: Introduction to Shell for Data Science
• Datacamp: Introduction to Bash Scripting
• Datacamp: Data Processing in Shell
• LaunchSchool: Introduction to Commandline
• Learn Enough Command Line to be dangerous
• Thoughtbot: Mastering the Shell
• Thoughtbot: tmux
• Udacity: Linux Command Line Basics
• Udacity: Shell Workshop
• Web Bos: Command Line Power User
• Youtube: GNU Parallel

Be able to perform feature engineering

• Article: Tips for Advanced Feature Engineering
• Article: Preparing data for a machine learning model
• Article: Feature selection for a machine learning model
• Article: Learning from imbalanced data
• Article: Hacker's Guide to Data Preparation for Machine Learning
• Article: Practical Guide to Handling Imbalanced Datasets
• Datacamp: Analyzing Social Media Data in Python
• Datacamp: Dimensionality Reduction in Python
• Datacamp: Preprocessing for Machine Learning in Python
• Datacamp: Data Types for Data Science
• Datacamp: Cleaning Data in Python
• Datacamp: Feature Engineering for Machine Learning in Python
• Datacamp: Importing & Managing Financial Data in Python
• Datacamp: Manipulating Time Series Data in Python
• Datacamp: Working with Geospatial Data in Python
• Datacamp: Analyzing IoT Data in Python
• Datacamp: Dealing with Missing Data in Python
• Datacamp: Exploratory Data Analysis in Python
• edX: Data Science Essentials
• Google: Feature Engineering
• Udacity: Creating an Analytical Dataset

Be able to experiment in a notebook

• Article: Securely storing configuration credentials in a Jupyter Notebook
• Article: Automatically Reload Modules with %autoreload
• Calmcode: ipywidgets
• Documentation: Jupyter Lab
• Pluralsight: Getting Started with Jupyter Notebook and Python
• Youtube: William Horton - A Brief History of Jupyter Notebooks
• Youtube: I Like Notebooks
• Youtube: I don't like notebooks.- Joel Grus (Allen Institute for Artificial Intelligence)
• Youtube: Ryan Herr - After model.fit, before you deploy| JupyterCon 2020
• Youtube: nbdev live coding with Hamel Husain
• Youtube: How to Use JupyterLab

Be able to visualize data

• Article: Creating a Catchier Word Cloud Presentation
• Article: Effectively Using Matplotlib
• Datacamp: Introduction to Data Visualization with Python
• Datacamp: Introduction to Seaborn
• Datacamp: Introduction to Matplotlib
• Datacamp: Intermediate Data Visualization with Seaborn
• Datacamp: Visualizing Time Series Data in Python
• Datacamp: Improving Your Data Visualizations in Python
• Datacamp: Visualizing Geospatial Data in Python
• Datacamp: Interactive Data Visualization with Bokeh
• Udacity: Data Visualization in Tableau
• Youtube: Jake VanderPlas - Exploratory Data Visualization with Vega, Vega-Lite, and Altair - PyCon 2018
• Youtube: Shantam Raj- Rapidly emulating professional visualizations from New York Times| PyData Global 2020
• UWData: Data Visualization Curriculum

Be able to do literature review using research papers

• Paper: A Neural Probabilistic Language Model
• Paper: Efficient Estimation of Word Representations in Vector Space
• Paper: Sequence to Sequence Learning with Neural Networks
• Paper: Neural Machine Translation by Jointly Learning to Align and Translate
• Paper: Attention Is All You Need
• Paper: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Paper: XLNet: Generalized Autoregressive Pretraining for Language Understanding
• Paper: Synonyms Based Term Weighting Scheme: An Extension to TF.IDF
• Paper: RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Paper: GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Paper: Amazon.com Recommendations Item-to-Item Collaborative Filtering
• Paper: Collaborative Filtering for Implicit Feedback Datasets
• Paper: BPR: Bayesian Personalized Ranking from Implicit Feedback
• Paper: Factorization Machines
• Paper: Wide & Deep Learning for Recommender Systems
• Paper: Neural Factorization Machines for Sparse Predictive Analytics
• Paper: Multiword Expressions: A Pain in the Neck for NLP
• Paper: PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Paper: ALBERT: A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS
• Paper: Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
• Paper: A Simple Framework for Contrastive Learning of Visual Representations
• Paper: Self-Supervised Learning of Pretext-Invariant Representations
• Paper: FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Paper: Self-Labelling via Simultalaneous Clustering and Representation Learning
• Paper: A Survey on Contextual Embeddings
• Paper: A survey on Semi-, Self- and Unsupervised Techniques in Image Classification
• Paper: Shortcut Learning in Deep Neural Networks
• Paper: Multi-document Summarization by using TextRank and Maximal Marginal Relevance for Text in Bahasa Indonesia
• Paper: Train Once, Test Anywhere: Zero-Shot Learning for Text Classification
• Paper: Zero-shot Text Classification With Generative Language Models
• Paper: How to Fine-Tune BERT for Text Classification?
• Paper: Universal Sentence Encoder
• Paper: Enriching Word Vectors with Subword Information
• Paper: Deep Learning Based Text Classification: A Comprehensive Review
• Paper: Beyond Accuracy: Behavioral Testing of NLP models with CheckList
• Paper: Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks
• Paper: Temporal Ensembling for Semi-Supervised Learning
• Paper: Boosting Self-Supervised Learning via Knowledge Transfer
• Paper: Follow-up Question Generation
• Paper: The Hardware Lottery
• Paper: Question Generation via Overgenerating Transformations and Ranking
• Paper: Good Question! Statistical Ranking for Question Generation
• Paper: Towards ML Engineering: A Brief History Of TensorFlow Extended (TFX)
• Paper: Interpolation Consistency Training for Semi-Supervised Learning
• Paper: Neural Text Generation: A Practical Guide
• Paper: Pest Management In Cotton Farms: An AI-System Case Study from the Global South
• Paper: BERT2DNN: BERT Distillation with Massive Unlabeled Data for Online E-Commerce Search
• Paper: On the surprising similarities between supervised and self-supervised models
• Paper: All-but-the-Top: Simple and Effective Postprocessing for Word Representations
• Paper: Simple and Effective Dimensionality Reduction for Word Embeddings
• Paper: A Survey on Visual Transformer
• Youtube: mixup: Beyond Empirical Risk Minimization (Paper Explained)

Be able to model problems mathematically

• 3Blue1Brown: Essence of Calculus
  • The Essence of Calculus, Chapter 1 0:17:04
  • The paradox of the derivative | Essence of calculus, chapter 2 0:17:57
  • Derivative formulas through geometry | Essence of calculus, chapter 3 0:18:43
  • Visualizing the chain rule and product rule | Essence of calculus, chapter 4 0:16:52
  • What's so special about Euler's number e? | Essence of calculus, chapter 5 0:13:50
  • Implicit differentiation, what's going on here? | Essence of calculus, chapter 6 0:15:33
  • Limits, L'Hôpital's rule, and epsilon delta definitions | Essence of calculus, chapter 7 0:18:26
  • Integration and the fundamental theorem of calculus | Essence of calculus, chapter 8 0:20:46
  • What does area have to do with slope? | Essence of calculus, chapter 9 0:12:39
  • Higher order derivatives | Essence of calculus, chapter 10 0:05:38
  • Taylor series | Essence of calculus, chapter 11 0:22:19
  • What they won't teach you in calculus 0:16:22
• 3Blue1Brown: Essence of linear algebra
  • Vectors, what even are they? | Essence of linear algebra, chapter 1 0:09:52
  • Linear combinations, span, and basis vectors | Essence of linear algebra, chapter 2 0:09:59
  • Linear transformations and matrices | Essence of linear algebra, chapter 3 0:10:58
  • Matrix multiplication as composition | Essence of linear algebra, chapter 4 0:10:03
  • Three-dimensional linear transformations | Essence of linear algebra, chapter 5 0:04:46
  • The determinant | Essence of linear algebra, chapter 6 0:10:03
  • Inverse matrices, column space and null space | Essence of linear algebra, chapter 7 0:12:08
  • Nonsquare matrices as transformations between dimensions | Essence of linear algebra, chapter 8 0:04:27
  • Dot products and duality | Essence of linear algebra, chapter 9 0:14:11
  • Cross products | Essence of linear algebra, Chapter 10 0:08:53
  • Cross products in the light of linear transformations | Essence of linear algebra chapter 11 0:13:10
  • Cramer's rule, explained geometrically | Essence of linear algebra, chapter 12 0:12:12
  • Change of basis | Essence of linear algebra, chapter 13 0:12:50
  • Eigenvectors and eigenvalues | Essence of linear algebra, chapter 14 0:17:15
  • Abstract vector spaces | Essence of linear algebra, chapter 15 0:16:46
• 3Blue1Brown: Neural networks
  • But what is a Neural Network? | Deep learning, chapter 1 0:19:13
  • Gradient descent, how neural networks learn | Deep learning, chapter 2 0:21:01
  • What is backpropagation really doing? | Deep learning, chapter 3 0:13:54
  • Backpropagation calculus | Deep learning, chapter 4 0:10:17
• Article: A Visual Tour of Backpropagation
• Article: Entropy, Cross Entropy, and KL Divergence
• Article: Introduction to Linear Algebra for Applied Machine Learning with Python
• Article: Entropy of a probability distribution — in layman’s terms
• Article: KL Divergence — in layman’s terms
• Article: Probability Distributions
• Article: Relearning Matrices as Linear Functions
• Article: You Could Have Come Up With Eigenvectors - Here's How
• Article: PageRank - How Eigenvectors Power the Algorithm Behind Google Search
• Article: Interactive Visualization of Why Eigenvectors Matter
• Article: Cross-Entropy and KL Divergence
• Article: Why Randomness Is Information?
• Article: Basic Probability Theory
• Book: Basics of Linear Algebra for Machine Learning
• Datacamp: Introduction to Statistics in Python
• Datacamp: Foundations of Probability in Python
• Datacamp: Statistical Thinking in Python (Part 1)
• Datacamp: Statistical Thinking in Python (Part 2)
• Datacamp: Statistical Simulation in Python
• edX: Essential Statistics for Data Analysis using Excel
• Computational Linear Algebra for Coders
• Khan Academy: Precalculus
• Khan Academy: Probability
• Khan Academy: Differential Calculus
• Khan Academy: Multivariable Calculus
• Khan Academy: Linear Algebra
• MIT: 18.06 Linear Algebra (Professor Strang)
  • 1. The Geometry of Linear Equations 0:39:49
  • 2. Elimination with Matrices. 0:47:41
  • 3. Multiplication and Inverse Matrices 0:46:48
  • 4. Factorization into A = LU 0:48:05
  • 5. Transposes, Permutations, Spaces R^n 0:47:41
  • 6. Column Space and Nullspace 0:46:01
  • 9. Independence, Basis, and Dimension 0:50:14
  • 10. The Four Fundamental Subspaces 0:49:20
  • 11. Matrix Spaces; Rank 1; Small World Graphs 0:45:55
  • 14. Orthogonal Vectors and Subspaces 0:49:47
  • 15. Projections onto Subspaces 0:48:51
  • 16. Projection Matrices and Least Squares 0:48:05
  • 17. Orthogonal Matrices and Gram-Schmidt 0:49:09
  • 21. Eigenvalues and Eigenvectors 0:51:22
  • 22. Diagonalization and Powers of A 0:51:50
  • 24. Markov Matrices; Fourier Series 0:51:11
  • 25. Symmetric Matrices and Positive Definiteness 0:43:52
  • 27. Positive Definite Matrices and Minima 0:50:40
  • 29. Singular Value Decomposition 0:40:28
  • 30. Linear Transformations and Their Matrices 0:49:27
  • 31. Change of Basis; Image Compression 0:50:13
  • 33. Left and Right Inverses; Pseudoinverse 0:41:52
• StatQuest: Statistics Fundamentals
  • StatQuest: Histograms, Clearly Explained 0:03:42
  • StatQuest: What is a statistical distribution? 0:05:14
  • StatQuest: The Normal Distribution, Clearly Explained!!! 0:05:12
  • Statistics Fundamentals: Population Parameters 0:14:31
  • Statistics Fundamentals: The Mean, Variance and Standard Deviation 0:14:22
  • StatQuest: What is a statistical model? 0:03:45
  • StatQuest: Sampling A Distribution 0:03:48
  • Hypothesis Testing and The Null Hypothesis 0:14:40
  • Alternative Hypotheses: Main Ideas!!! 0:09:49
  • p-values: What they are and how to interpret them 0:11:22
  • How to calculate p-values 0:25:15
  • p-hacking: What it is and how to avoid it! 0:13:44
  • Statistical Power, Clearly Explained!!! 0:08:19
  • Power Analysis, Clearly Explained!!! 0:16:44
  • Covariance and Correlation Part 1: Covariance 0:22:23
  • Covariance and Correlation Part 2: Pearson's Correlation 0:19:13
  • StatQuest: R-squared explained 0:11:01
  • The Central Limit Theorem 0:07:35
  • StatQuickie: Standard Deviation vs Standard Error 0:02:52
  • StatQuest: The standard error 0:11:43
  • StatQuest: Technical and Biological Replicates 0:05:27
  • StatQuest - Sample Size and Effective Sample Size, Clearly Explained 0:06:32
  • Bar Charts Are Better than Pie Charts 0:01:45
  • StatQuest: Boxplots, Clearly Explained 0:02:33
  • StatQuest: Logs (logarithms), clearly explained 0:15:37
  • StatQuest: Confidence Intervals 0:06:41
  • StatQuickie: Thresholds for Significance 0:06:40
  • StatQuickie: Which t test to use 0:05:10
  • StatQuest: One or Two Tailed P-Values 0:07:05
  • The Binomial Distribution and Test, Clearly Explained!!! 0:15:46
  • StatQuest: Quantiles and Percentiles, Clearly Explained!!! 0:06:30
  • StatQuest: Quantile-Quantile Plots (QQ plots), Clearly Explained 0:06:55
  • StatQuest: Quantile Normalization 0:04:51
  • StatQuest: Probability vs Likelihood 0:05:01
  • StatQuest: Maximum Likelihood, clearly explained!!! 0:06:12
  • Maximum Likelihood for the Exponential Distribution, Clearly Explained! V2.0 0:09:39
  • Why Dividing By N Underestimates the Variance 0:17:14
  • Maximum Likelihood for the Binomial Distribution, Clearly Explained!!! 0:11:24
  • Maximum Likelihood For the Normal Distribution, step-by-step! 0:19:50
  • StatQuest: Odds and Log(Odds), Clearly Explained!!! 0:11:30
  • StatQuest: Odds Ratios and Log(Odds Ratios), Clearly Explained!!! 0:16:20
  • Live 2020-04-20!!! Expected Values 0:33:00
• Udacity: Algebra Review
• Udacity: Differential Equations in Action
• Udacity: Eigenvectors and Eigenvalues
• Udacity: Linear Algebra Refresher
• Udacity: Statistics
• Udacity: Intro to Descriptive Statistics
• Udacity: Intro to Inferential Statistics
• Youtube: Visualizing Deep Learning
  • The Neural Network, A Visual Introduction | Visualizing Deep Learning, Chapter 1

Be able to structure machine learning projects

• Article: I trained a model. What is next?
• Article: pydantic
• Article: Always start with a stupid model, no exceptions.
• Article: Organizing machine learning projects: project management guidelines
• Article: Most impactful AI trends of 2018: the rise of ML Engineering
• Article: Building machine learning products: a problem well-defined is a problem half-solved.
• Article: Simple considerations for simple people building fancy neural networks
• Article: Logging and Debugging in Machine Learning - How to use Python debugger and the logging module to find errors in your AI application
• Article: How to Unit Test Deep Learning: Tests in TensorFlow, mocking and test coverage
• Article: Best practices to write Deep Learning code: Project structure, OOP, Type checking and documentation
• Article: Deep Learning in Production: Laptop set up and system design
• Article: Configuring Google Colab Like A Pro
• Article: Stop using print, start using loguru in Python
• Coursera: Structuring Machine Learning Projects
• Datacamp: Conda Essentials
• Datacamp: Conda for Building & Distributing Packages
• Datacamp: Creating Robust Python Workflows
• Datacamp: Software Engineering for Data Scientists in Python
• Datacamp: Designing Machine Learning Workflows in Python
• Datacamp: Object-Oriented Programming in Python
• Datacamp: Command Line Automation in Python
• Datacamp: Introduction to Data Engineering
• Datacamp: Experimental Design in Python
• Developing Python Packages
• Full Stack Deep Learning Bootcamp: March 2019
  • Lecture 1: Introduction to Deep Learning
  • Lecture 2: Setting Up Machine Learning Projects
  • Lecture 3: Introduction to the Text Recognizer Project
  • Lecture 4: Infrastructure and Tooling
  • Lecture 5: Tracking Experiments
  • Lecture 6: Data Management
  • Lecture 7: Machine Learning Teams
  • Lecture 9: Lukas Biewald
  • Lecture 10: Troubleshooting Deep Neural Networks
  • Lecture 11: Labs 6-9: Detection, Data Labeling, Testing and Deployment
  • Lecture 12: Testing and Deployment
  • Lecture 13: Research Directions
  • Lecture 14: Jeremy Howard
  • Lecture 15: Richard Socher
  • Guest Lecture - Chip Huyen - Machine Learning Interviews - Full Stack Deep Learning
• MIT: The Missing Semester of CS Education
  • Lecture 1: Course Overview + The Shell (2020) 0:48:16
  • Lecture 2: Shell Tools and Scripting (2020) 0:48:55
  • Lecture 3: Editors (vim) (2020) 0:48:26
  • Lecture 4: Data Wrangling (2020) 0:50:03
  • Lecture 5: Command-line Environment (2020) 0:56:06
  • Lecture 6: Version Control (git) (2020) 1:24:59
  • Lecture 7: Debugging and Profiling (2020) 0:54:13
  • Lecture 8: Metaprogramming (2020) 0:49:52
  • Lecture 9: Security and Cryptography (2020) 1:00:59
  • Lecture 10: Potpourri (2020) 0:57:54
  • Lecture 11: Q&A (2020) 0:53:52
• Treehouse: Object Oriented Python
• Treehouse: Setup Local Python Environment
• Udacity: Writing READMEs
• Youtube: Weights and Biases Tutorial
• Youtube: Integrate Weights & Biases with PyTorch
• Youtube: Log (Almost) Anything with Weights & Biases
• Youtube: MLOps Tutorials
  • MLOps Tutorial #1: Intro to Continuous Integration for ML 17:44:00
  • MLOps Tutorial #2: When data is too big for Git 10:51:00
  • MLOps Tutorial #3: Track ML models with Git & GitHub Actions 14:12:00
• Youtube: Data Engineering + ML + Software Engineering // Satish Chandra Gupta // MLOps Coffee Sessions #16
• Youtube: OO Design and Testing Patterns for Machine Learning with Chris Gerpheide
• Youtube: MLSys Seminars Fall 2020
  • Stanford MLSys Seminar Episode 0: ML + Systems 0:11:49
  • Stanford MLSys Seminar Episode 1: Marco Tulio Ribeiro 1:00:38
  • Stanford MLSys Seminar Episode 2: Matei Zaharia 0:59:44
  • Stanford MLSys Seminar Episode 3: Virginia Smith 1:00:55
  • Stanford MLSys Seminar Episode 4: Alex Ratner 1:13:34
  • Stanford MLSys Seminar Episode 5: Chip Huyen 1:06:44

Be able to version control code

• Article: Mastering Git Stash Workflow
• Article: How to Become a Master of Git Tags
• Article: How to track large files in Github / Bitbucket? Git LFS to the rescue
• Article: Keep your git directory clean with git clean and git trash
• Codecademy: Learn Git
• Code School: Git Real
• Datacamp: Introduction to Git for Data Science
• Learn enough git to be dangerous
• Learn Git Branching
• Thoughtbot: Mastering Git
• Udacity: GitHub & Collaboration
• Udacity: How to Use Git and GitHub
• Udacity: Version Control with Git
• Youtube: 045 Introduction to Git LFS

Be able to version control data

• Youtube: DVC Basics
  • Versioning Data with DVC (Hands-On Tutorial!) 0:13:04
  • Sharing Data and Models with DVC (Hands-On Data Science Tutorial!) 0:08:53
  • Machine Learning Pipelines with DVC (Hands-On Tutorial!) 0:10:04
  • Machine Learning Experiments with DVC (Hands-On Tutorial!) 0:07:29

Be familiar with fundamentals of ML and DL

• Article: Validating your Machine Learning Model
• Article: One-vs-Rest strategy for Multi-Class Classification
• Article: Multi-class Classification — One-vs-All & One-vs-One
• Article: One-vs-Rest and One-vs-One for Multi-Class Classification
• Article: The Complete Guide to AUC and Average Precision: Simulations and Visualizations
• Article: Connections: Log Likelihood, Cross Entropy, KL Divergence, Logistic Regression, and Neural Networks
• Article: Best Use of Train/Val/Test Splits, with Tips for Medical Data
• Article: Measuring Performance: AUPRC and Average Precision
• Article: Measuring Performance: AUC (AUROC)
• Article: Measuring Performance: The Confusion Matrix
• Article: Measuring Performance: Accuracy
• Article: Naive Bayes classification
• Article: Linear regression
• Article: Polynomial regression
• Article: Logistic regression
• Article: Decision trees
• Article: K-nearest neighbors
• Article: Support Vector Machines
• Article: Random forests
• Article: Boosted trees
• Article: ROC Curves: Intuition Through Visualization
• Article: Hacker's Guide to Fundamental Machine Learning Algorithms with Python
• Article: Simple Ways to Tackle Class Imbalance
• Article: MLE and MAP — in layman’s terms
• Article: The Last 5 Years In Deep Learning
• Article: Visualizing Optimization Trajectory of Neural Nets
• An overview of gradient descent optimization algorithms
• Article: Optimization for Deep Learning Highlights in 2017
• Article: Neural networks: activation functions
• Article: Neural networks: training with backpropagation
• Article: Gradient descent
• Article: Setting the learning rate of your neural network
• Article: Deep neural networks: preventing overfitting
• Article: Normalizing your data (specifically, input and batch normalization)
• Article: Batch Normalization
• Article: Are Deep Neural Networks Dramatically Overfitted?
• Article: Attention? Attention!
• Article: How to Explain the Prediction of a Machine Learning Model?
• Article: Neural Network from scratch-part 1
• Article: Neural Network from scratch-part 2
• Article: Deep Learning Algorithms - The Complete Guide
• Article: In-layer normalization techniques for training very deep neural networks
• Article: Cross-entropy for classification
• Article: Perceptron to Deep-Neural-Network
• Article: Dismantling Neural Networks to Understand the Inner Workings with Math and Pytorch
• Article: Label Smoothing Explained using Microsoft Excel
• Article: Precision, Recall, Accuracy, and F1 Score for Multi-Label Classification
• AWS: The Elements of Data Science
• AWS: Understanding Neural Networks
• Book: Approaching (Almost) Any Machine Learning Problem
• Book: Pattern Recognition and Machine Learning
• Book: Grokking Deep Learning
• Book: Make Your Own Neural Network
• Coursera: Neural Networks and Deep Learning
• Datacamp: AI Fundamentals
• Datacamp: Extreme Gradient Boosting with XGBoost
• Datacamp: Ensemble Methods in Python
• Datacamp: Foundations of Predictive Analytics in Python (Part 1)
• Datacamp: Foundations of Predictive Analytics in Python (Part 2)
• Elements of AI
• edX: Principles of Machine Learning
• edX: Data Science Essentials
• Fast.ai: Deep Learning for Coder (2020)
  • Lesson 1
  • Lesson 2
  • Lesson 3
  • Lesson 4
  • Lesson 5
  • Lesson 6
  • Lesson 7
  • Lesson 8
• Pluralsight: Deep Learning: The Big Picture
• StatQuest: Machine Learning
  • A Gentle Introduction to Machine Learning 0:12:45
  • Machine Learning Fundamentals: Cross Validation 0:06:04
  • Machine Learning Fundamentals: The Confusion Matrix 0:07:12
  • Machine Learning Fundamentals: Sensitivity and Specificity 0:11:46
  • Machine Learning Fundamentals: Bias and Variance 0:06:36
  • ROC and AUC, Clearly Explained! 0:16:26
  • StatQuest: Fitting a line to data, aka least squares, aka linear regression. 0:09:21
  • StatQuest: Linear Models Pt.1 - Linear Regression 0:27:26
  • StatQuest: Odds and Log(Odds), Clearly Explained!!! 0:11:30
  • StatQuest: Odds Ratios and Log(Odds Ratios), Clearly Explained!!! 0:16:20
  • StatQuest: Logistic Regression 0:08:47
  • Logistic Regression Details Pt1: Coefficients 0:19:02
  • Logistic Regression Details Pt 2: Maximum Likelihood 0:10:23
  • Logistic Regression Details Pt 3: R-squared and p-value 0:15:25
  • Saturated Models and Deviance 0:18:39
  • Deviance Residuals 0:06:18
  • Regularization Part 1: Ridge (L2) Regression 0:20:26
  • Regularization Part 2: Lasso (L1) Regression 0:08:19
  • Ridge vs Lasso Regression, Visualized!!! 0:09:05
  • Regularization Part 3: Elastic Net Regression 0:05:19
  • StatQuest: Principal Component Analysis (PCA), Step-by-Step 0:21:57
  • StatQuest: PCA main ideas in only 5 minutes!!! 0:06:04
  • StatQuest: PCA - Practical Tips 0:08:19
  • StatQuest: PCA in Python 0:11:37
  • StatQuest: Linear Discriminant Analysis (LDA) clearly explained. 0:15:12
  • StatQuest: MDS and PCoA 0:08:18
  • StatQuest: t-SNE, Clearly Explained 0:11:47
  • StatQuest: Hierarchical Clustering 0:11:19
  • StatQuest: K-means clustering 0:08:57
  • StatQuest: K-nearest neighbors, Clearly Explained 0:05:30
  • Naive Bayes, Clearly Explained!!! 0:15:12
  • Gaussian Naive Bayes, Clearly Explained!!! 0:09:41
  • StatQuest: Decision Trees 0:17:22
  • StatQuest: Decision Trees, Part 2 - Feature Selection and Missing Data 0:05:16
  • Regression Trees, Clearly Explained!!! 0:22:33
  • How to Prune Regression Trees, Clearly Explained!!! 0:16:15
  • StatQuest: Random Forests Part 1 - Building, Using and Evaluating 0:09:54
  • StatQuest: Random Forests Part 2: Missing data and clustering 0:11:53
  • The Chain Rule 0:18:23
  • Gradient Descent, Step-by-Step 0:23:54
  • Stochastic Gradient Descent, Clearly Explained!!! 0:10:53
  • AdaBoost, Clearly Explained 0:20:54
  • Gradient Boost Part 1: Regression Main Ideas 0:15:52
  • Gradient Boost Part 2: Regression Details 0:26:45
  • Gradient Boost Part 3: Classification 0:17:02
  • Gradient Boost Part 4: Classification Details 0:36:59
  • Support Vector Machines, Clearly Explained!!! 0:20:32
  • Support Vector Machines Part 2: The Polynomial Kernel 0:07:15
  • Support Vector Machines Part 3: The Radial (RBF) Kernel 0:15:52
  • XGBoost Part 1: Regression 0:25:46
  • XGBoost Part 2: Classification 0:25:17
  • XGBoost Part 3: Mathematical Details 0:27:24
  • XGBoost Part 4: Crazy Cool Optimizations 0:24:27
  • StatQuest: Fiitting a curve to data, aka lowess, aka loess 0:10:10
  • Statistics Fundamentals: Population Parameters 0:14:31
  • Principal Component Analysis (PCA) clearly explained (2015) 0:20:16
  • Decision Trees in Python from Start to Finish 1:06:23
• Udacity: Deep Learning
• Udacity: A Friendly Introduction to Machine Learning
• Udacity: Intro to Data Analysis
• Udacity: Intro to Data Science
• Udacity: Intro to Machine Learning
• Udacity: Classification Models
• Youtube: Applied Machine Learning 2020
  • Channel Intro - Applied Machine Learning 0:01:28
  • Applied ML 2020 - 01 Introduction 1:16:01
  • Applied ML 2020 - 02 Visualization and matplotlib 1:07:30
  • Applied ML 2020 - 03 Supervised learning and model validation 1:12:00
  • Applied ML 2020 - 04 - Preprocessing 1:07:40
  • Applied ML 2020 - 05 - Linear Models for Regression 1:06:54
  • Applied ML 2020 - 06 - Linear Models for Classification 1:07:50
  • Applied ML 2020 - 07 - Decision Trees and Random Forests 1:07:58
  • Applied ML 2020 - 08 - Gradient Boosting 1:02:12
  • Applied ML 2020 - 09 - Model Evaluation and Metrics 1:18:23
  • Applied ML 2020 - 10 - Calibration, Imbalanced data 1:16:14
  • Applied ML 2020 - 11 - Model Inspection and Feature Selection 1:15:15
  • Applied ML 2020 - 12 - AutoML (plus some feature selection) 1:25:38
  • Applied ML 2020 - 13 - Dimensionality reduction 1:30:34
  • Applied ML 2020 - 14 - Clustering and Mixture Models 1:26:33
  • Applied ML 2020 - 15 - Working with Text Data 1:27:08
  • Applied ML 2020 - 16 - Topic models for text data 1:18:34
  • Applied ML 2020 - 17 - Word vectors and document embeddings 1:03:04
  • Applied ML 2020 - 18 - Neural Networks 1:19:36
  • Applied ML 2020 - 21 - Time Series and Forecasting 1:12:36
• Youtube: Neural Networks from Scratch in Python
  • Neural Networks from Scratch - P.1 Intro and Neuron Code 0:16:59
  • Neural Networks from Scratch - P.2 Coding a Layer 0:15:06
  • Neural Networks from Scratch - P.3 The Dot Product 0:25:17
  • Neural Networks from Scratch - P.4 Batches, Layers, and Objects 0:33:46
  • Neural Networks from Scratch - P.5 Hidden Layer Activation Functions 0:40:05
• Youtube: Visualizing Deep Learning
  • The Neural Network, A Visual Introduction | Visualizing Deep Learning, Chapter 1
• Youtube: Deep Double Descent
• Youtube: How do we check if a neural network has learned a specific phenomenon?
• Youtube: What is Adversarial Machine Learning and what to do about it? – Adversarial example compilation

Be able to implement models in scikit-learn

• Article: Stacking made easy with Sklearn
• Article: A Guide to Calibration Plots in Python
• Calmcode: human-learn
• Datacamp: Supervised Learning with scikit-learn
• Datacamp: Machine Learning with Tree-Based Models in Python
• Datacamp: Introduction to Linear Modeling in Python
• Datacamp: Linear Classifiers in Python
• Datacamp: Generalized Linear Models in Python
• Notebook: scikit-learn tips
• Pluralsight: Building Machine Learning Models in Python with scikit-learn
• Video: human learn
• Youtube: dabl: Automatic Machine Learning with a Human in the Loop 00:25:43
• Youtube: Multilabel and Multioutput Classification -Machine Learning with TensorFlow & scikit-learn on Python

Be able to implement models in Tensorflow and Keras

• Coursera: Introduction to Tensorflow
• Coursera: Convolutional Neural Networks in TensorFlow
• Coursera: Getting Started With Tensorflow 2
• Coursera: Customising your models with TensorFlow 2
• Deeplizard: Keras - Python Deep Learning Neural Network API
• Book: Deep Learning with Python (Page: 276)
• Datacamp: Deep Learning in Python
• Datacamp: Convolutional Neural Networks for Image Processing
• Datacamp: Introduction to TensorFlow in Python
• Datacamp: Introduction to Deep Learning with Keras
• Datacamp: Advanced Deep Learning with Keras
• Google: Intro to Tensorflow
• Google: Machine Learning Crash Course
• Pluralsight: Deep Learning with Keras
• Udacity: Intro to TensorFlow for Deep Learning

Be able to implement models in PyTorch

• Article: Training Neural Nets on Larger Batches: Practical Tips for 1-GPU, Multi-GPU & Distributed setups
• Article: An introduction to PyTorch Lightning with comparisons to PyTorch
• Article: Scaling Logistic Regression Via Multi-GPU/TPU Training
• Article: PyTorch Lightning Bolts — From Linear, Logistic Regression on TPUs to pre-trained GANs
• Article: Keeping Up with PyTorch Lightning and Hydra
• Article: PyTorch Lightning 0.9 — synced BatchNorm, DataModules and final API!
• Article: PyTorch Lightning: Metrics
• Article: PyTorch Multi-GPU Metrics Library and More in PyTorch Lightning 0.8.1
• Article: 7 Tips To Maximize PyTorch Performance
• Article: From PyTorch to PyTorch Lightning — A gentle introduction
• Article: Converting From Keras To PyTorch Lightning
• Article: How Wadhwani AI Uses PyTorch To Empower Cotton Farmers
• Article: PyTorch Loss Functions: The Ultimate Guide
• Article: Supercharge your Training with Pytorch Lightning + Weights & Biases
• Article: Sharded: A New Technique To Double The Size Of PyTorch Models
• Article: Introducing PyTorch Lightning Sharded: Train SOTA Models, With Half The Memory
• Article: Transform your ML-model to Pytorch with Hummingbird
• Article: Distributed model training in PyTorch using DistributedDataParallel
• Article: Taming LSTMs: Variable-sized mini-batches and why PyTorch is good for your health
• Article: Pad pack sequences for Pytorch batch processing with DataLoader
• Article: Understanding Bidirectional RNN in PyTorch
• Article: PyTorch Lightning V1.2.0- DeepSpeed, Pruning, Quantization, SWA
• Article: But what are PyTorch DataLoaders really?
• Article: How to Build a Streaming DataLoader with PyTorch
• Article: Tricks for training PyTorch models to convergence more quickly
• Article: The One PyTorch Trick Which You Should Know
• Article: A developer-friendly guide to model pruning in PyTorch
• Article: A developer-friendly guide to model quantization with PyTorch
• Article: Distributed model training in PyTorch using DistributedDataParallel
• Article: A developer-friendly guide to mixed precision training with PyTorch
• Article: Fit More and Train Faster With ZeRO via DeepSpeed and FairScale
• Article: Faster Deep Learning Training with PyTorch – a 2021 Guide
• Article: EINSUM IS ALL YOU NEED - EINSTEIN SUMMATION IN DEEP LEARNING
• Notebook: Tensor Arithmetic
• Notebook: Autograd
• Notebook: Optimization
• Notebook: Network modules
• Notebook: Datasets and Dataloaders
• Documentation: Pytorch Lightning
• Datacamp: Introduction to Deep Learning with PyTorch
• Deeplizard: Neural Network Programming - Deep Learning with PyTorch
• Udacity: Intro to Deep Learning with PyTorch
• Youtube: PyTorch Lightning 101
  • Training a classification model on MNIST with PyTorch
  • From PyTorch to PyTorch Lightning
  • Lightning Data Modules
  • PyTorch Dropout, Batch size and interactive debugging
  • Episode 4: Implementing a PyTorch Trainer: PyTorch Lightning Trainer and callbacks under-the-hood
• Youtube: SimCLR with PyTorch Lightning
• Youtube: PyTorch Performance Tuning Guide 26:41:00
• Youtube: Skin Cancer Detection with PyTorch
  • [PART 1] Skin Cancer Detection with PyTorch 0:10:21
  • [PART 2] Skin Cancer Detection with PyTorch 0:21:57
  • [PART 3] Skin Cancer Detection with PyTorch 0:22:24
• Youtube: Learn with Lightning
• Youtube: PyTorch Tutorial - RNN & LSTM & GRU - Recurrent Neural Nets 00:15:51
• Youtube: Pytorch Zero to All
  • PyTorch Lecture 01: Overview 0:10:18
  • PyTorch Lecture 02: Linear Model 0:12:52
  • PyTorch Lecture 03: Gradient Descent 0:08:24
  • PyTorch Lecture 04: Back-propagation and Autograd 0:15:25
  • PyTorch Lecture 05: Linear Regression in the PyTorch way 0:11:50
  • PyTorch Lecture 06: Logistic Regression 0:10:41
  • PyTorch Lecture 07: Wide and Deep 0:10:37
  • PyTorch Lecture 08: PyTorch DataLoader 0:06:41
  • PyTorch Lecture 09: Softmax Classifier 0:18:47
  • PyTorch Lecture 10: Basic CNN 0:15:52
  • PyTorch Lecture 11: Advanced CNN 0:12:58
  • PyTorch Lecture 12: RNN1 - Basics 0:28:47
  • PyTorch Lecture 13: RNN 2 - Classification 0:17:22
• PyTorch Developer Day 2020 | Full Livestream
• Youtube: Lightning Chat: How a Grandmaster Won a Kaggle Competition Using Pytorch Lightning

Be able to implement models using cloud services

• AWS: Amazon Transcribe Deep Dive: Using Feedback Loops to Improve Confidence Level of Transcription
• AWS: Build a Text Classification Model with AWS Glue and Amazon SageMaker
• AWS: Deep Dive on Amazon Rekognition: Building Computer Visions Based Smart Applications
• AWS: Hands-on Rekognition: Automated Video Editing
• AWS: Introduction to Amazon Comprehend
• AWS: Introduction to Amazon Comprehend Medical
• AWS: Introduction to Amazon Elastic Inference
• AWS: Introduction to Amazon Forecast
• AWS: Introduction to Amazon Lex
• AWS: Introduction to Amazon Personalize
• AWS: Introduction to Amazon Polly
• AWS: Introduction to Amazon SageMaker Ground Truth
• AWS: Introduction to Amazon SageMaker Neo
• AWS: Introduction to Amazon Transcribe
• AWS: Introduction to Amazon Translate
• AWS: Introduction to AWS Marketplace - Machine Learning Category
• AWS: Machine Learning Exam Basics
• AWS: Neural Machine Translation with Sockeye
• AWS: Process Model: CRISP-DM on the AWS Stack
• AWS: Satellite Image Classification in SageMaker
• Datacamp: Introduction to AWS Boto in Python
• edX: Amazon SageMaker: Simplifying Machine Learning Application Development

Be able to apply unsupervised learning algorithms

• Article: From Research to Production with Deep Semi-Supervised Learning
• Article: RecSys 2020 - Takeaways and Notable Papers
• Article: Paper Summary: DeViSE: A Deep Visual-Semantic Embedding Model
• Article: An overview of proxy-label approaches for semi-supervised learning
• Article: Create, Visualize and Interpret Customer Segments
• Article: A gentle introduction to HDBSCAN and density-based clustering
• Article: Grouping data points with k-means clustering
• Article: Soft clustering with Gaussian mixed models (EM)
• Article: Introduction to autoencoders
• Article: Variational autoencoders
• Article: Principal components analysis (PCA)
• Article: Deep Inside Autoencoders
• Article: Build a simple Image Retrieval System with an Autoencoder
• Article: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Article: A Framework For Contrastive Self-Supervised Learning And Designing A New Approach
• Article: Self-supervised learning: The dark matter of intelligence
• Article: Contrastive Self-Supervised Learning
• Article: Understanding self-supervised and contrastive learning with "Bootstrap Your Own Latent" (BYOL)
• Article: Affinity Propagation Algorithm Explained
• Article: Algorithm Breakdown: Affinity Propagation
• Article: From Autoencoder to Beta-VAE
• Article: Self-Supervised Representation Learning
• Article: GANs in computer vision - Introduction to generative learning
• Article: GANs in computer vision - self-supervised adversarial training and high-resolution image synthesis with style incorporation
• Article: GANs in computer vision - semantic image synthesis and learning a generative model from a single image
• Article: GANs in computer vision - Improved training with Wasserstein distance, game theory control and progressively growing schemes
• Article: GANs in computer vision - Conditional image synthesis and 3D object generation
• Article: Decrypt Generative Adversarial Networks (GAN)
• Article: How to Generate Images using Autoencoders
• Article: Deepfakes: Face synthesis with GANs and Autoencoders
• Article: EfficientDet Meets Pytorch-Lightning
• Berkeley: Deep Unsupervised Learning Spring 2020
  • L1 Introduction -- CS294-158-SP20 Deep Unsupervised Learning -- UC Berkeley, Spring 2020 1:10:02
  • L2 Autoregressive Models -- CS294-158-SP20 Deep Unsupervised Learning -- UC Berkeley, Spring 2020 2:27:23
  • L3 Flow Models -- CS294-158-SP20 Deep Unsupervised Learning -- UC Berkeley -- Spring 2020 1:56:53
  • L4 Latent Variable Models (VAE) -- CS294-158-SP20 Deep Unsupervised Learning -- UC Berkeley 2:19:33
  • Lecture 5 Implicit Models -- GANs Part I --- UC Berkeley, Spring 2020 2:32:32
  • Lecture 6 Implicit Models / GANs part II --- CS294-158-SP20 Deep Unsupervised Learning -- Berkeley 2:09:14
  • Lecture 7 Self-Supervised Learning -- UC Berkeley Spring 2020 - CS294-158 Deep Unsupervised Learning 2:20:41
  • L8 Round-up of Strengths and Weaknesses of Unsupervised Learning Methods -- UC Berkeley SP20 0:41:51
  • L9 Semi-Supervised Learning and Unsupervised Distribution Alignment -- CS294-158-SP20 UC Berkeley 2:16:00
  • L10 Compression -- UC Berkeley, Spring 2020, CS294-158 Deep Unsupervised Learning 3:09:49
  • L11 Language Models -- guest instructor: Alec Radford (OpenAI) --- Deep Unsupervised Learning SP20 2:38:19
  • L12 Representation Learning for Reinforcement Learning --- CS294-158 UC Berkeley Spring 2020 2:01:56
• Datacamp: Customer Segmentation in Python
• Datacamp: Unsupervised Learning in Python
• Deck: Demystifying Self-Supervised Learning for Visual Recognition
• DeepMind: Inefficient Data Efficiency
• Google: Clustering
• Udacity: Segmentation and Clustering
• Wandb: Unsupervised Visual Representation Learning with SwAV
• Youtube: BYOL: Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (Paper Explained)
• Youtube: A critical analysis of self-supervision, or what we can learn from a single image (Paper Explained)
• Youtube: Week 10 – Lecture: Self-supervised learning (SSL) in computer vision (CV)
• Youtube: CVPR 2020 Tutorial: Towards Annotation-Efficient Learning
• Youtube: Yuki Asano | Self-Supervision | Self-Labelling | Labelling Unlabelled videos | CV | CTDS.Show #81
• Youtube: Contrastive Clustering with SwAV
• Youtube: Variational Autoencoders - EXPLAINED! 0:17:36
• Youtube: OptaProAnalyticsForum– Learning to watch football: Self-supervised representations for tracking data
• Youtube: Can a Neural Net tell if an image is mirrored? – Visual Chirality
• Youtube: Deep InfoMax: Learning deep representations by mutual information estimation and maximization
• Deep Learning Lecture Summer 2020
  • Deep Learning: Unsupervised Learning - Part 1
  • Deep Learning: Unsupervised Learning - Part 2
  • Deep Learning: Unsupervised Learning - Part 3
  • Deep Learning: Unsupervised Learning - Part 4
  • Deep Learning: Unsupervised Learning - Part 5
  • Deep Learning: Weakly and Self-Supervised Learning - Part 1
  • Deep Learning: Weakly and Self-Supervised Learning - Part 2
  • Deep Learning: Weakly and Self-Supervised Learning - Part 3
  • Deep Learning: Weakly and Self-Supervised Learning - Part 4
• ECCV 2020: New Frontiers for Learning with Limited Labels or Data
  • Introduction to New Frontiers on Learning with Limited Labels or Data
  • Self-Supervised Part and Viewpoint Discovery from Image Collections
  • Learning Visual Correspondences across Instances and Video Frames
  • Limitless Labels in a Labelless World: Weak Supervision with Noisy Labels
  • Inverting Neural Networks for Data-free Knowledge Transfer
  • Learning Efficiently with Biologically Inspired Feedback
• Youtube: Self-Supervised Learning - What is Next? - Workshop at ECCV 2020, August 28th
  • Next Challenges for Self-Supervised Learning - Aäron van den Oord 0:20:13
  • Perspectives on Unsupervised Representation Learning - Paolo Favaro 0:42:41
  • Learning and Transferring Visual Representations with Few Labels - Carl Doersch 0:32:53
  • Multi-view Invariance and Grouping for Self-Supervised Learning - Ishan Misra 0:36:31
  • Representation Learning beyond Instance Discrimination and Semantic Categorization - Stella Yu 0:43:09
  • Self-Supervision as a Path to a Post-Dataset Era - Alexei Alyosha Efros 0:38:06
  • Self-Supervision & Modularity: Cornerstones for Generalization in Embodied Agents - Deepak Pathak 0:41:56
• Youtube: Marco Cuturi - A Primer on Optimal Transport
• Youtube: Sebastian Ruder: Neural Semi-supervised Learning under Domain Shift
• Youtube: Clustering Algorithms
• Youtube: Beyond supervised learning

Be able to implement NLP models

• Article: Understanding ARPA and Language Models
• Article: Transformer-based Encoder-Decoder Models
• Article: Zero-Shot Learning in Modern NLP
• Article: Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)
• Article: Text Data Cleanup - Dynamic Embedding Visualisation
• Article: The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
• Article: Intuition & Use-Cases of Embeddings in NLP & beyond
• Article: The Illustrated GPT-2 (Visualizing Transformer Language Models)
• Article: The Illustrated Word2vec
• Article: All Our N-gram are Belong to You
• Article: Introducing spaCy
• Article: DialogRPT with Huggingface Transformers
• Article: How to Outperform GPT-3 by Combining Task Descriptions With Supervised Learning
• Article: How we used Universal Sentence Encoder and FAISS to make our search 10x smarter
• Article: Porting fairseq wmt19 translation system to transformers
• Article: NLP: Pre-trained Sentiment Analysis
• Article: How to build a State-of-the-Art Conversational AI with Transfer Learning
• Article: The Current Best of Universal Word Embeddings and Sentence Embeddings
• Article: Long Short-Term Memory: From Zero to Hero with PyTorch
• Article: Ten trends in Deep learning NLP
• Article: Google mT5 multilingual text-to-text transformer: A Brief Paper Analysis
• Article: String Matching with BERT, TF-IDF, and more!
• Article: Keyword Extraction with BERT
• Article: Creating a class-based TF-IDF with Scikit-Learn
• Article: Topic Modeling with BERT
• Article: NLP Year In Review
• Article: Using an NLP Q&A System To Study Climate Hazards and Nature-Based Solutions
• Article: Building a sentence embedding index with fastText and BM25
• Article: ML and NLP Research Highlights of 2020
• Article: Hugging Face Reads - 01/2021 - Sparsity and Pruning
• Article: Recent Advances in Language Model Fine-tuning
• Article: Understanding building blocks of ULMFIT
• Article: Swiss army knife for unsupervised task solving
• Article: Multi-Label Text Classification
• Article: Key topics extraction and contextual sentiment of users reviews
• Article: Understanding Convolutional Neural Networks for NLP
• Article: Attention and Memory in Deep Learning and NLP
• Article: Unsupervised synonym harvesting
• Article: Hugging Face Reads, Feb. 2021 - Long-range Transformers
• Article: Maximizing BERT model performance
• Article: Unsupervised creation of interpretable sentence representations
• Article: Document search with fragment embeddings
• Article: Unsupervised NER using BERT
• Article: Examining BERT’s raw embeddings
• Article: T5 — a model that explores the limits of transfer learning
• Article: T5 — XLNet — a clever language modeling solution
• Article: A review of BERT based models
• Article: Deconstructing BERT
• Article: Text classification from few training examples
• Article: Improved Few-Shot Text classification
• Article: Solving NER with BERT for any entity type with very little training data (compared to past approaches)
• Article: Trends in input representation for state-of-art NLP models (2019)
• Article: Brief review of word embedding families
• Article: GPT-2 A nascent transfer learning method that could eliminate supervised learning in some NLP tasks
• Article: Semantic search using BERT embeddings
• Article: A Survey of Long-Term Context in Transformers
• Article: A Deep Dive into the Reformer
• Article: Representation Learning and Retrieval
• Article: Large Memory Layers with Product Keys
• Article: Pattern-Exploiting Training
• Article: Optimal Transport and the Sinkhorn Transformer
• Article: Talking-Heads Attention
• Article: Rebuilding the most popular spellchecker. Part 1
• Article: Rebuilding the spellchecker, pt.2: Just look in the dictionary, they said!
• Article: Rebuilding the spellchecker, pt.3: Lookup—compounds and solutions
• Article: Rebuilding the spellchecker, pt.4: Introduction to suggest algorithm
• Article: Rebuilding the spellchecker: Hunspell and the order of edits
• Article: Performers: The Kernel Trick, Random Fourier Features, and Attention
• Article: Advance BERT model via transferring knowledge from Cross-Encoders to Bi-Encoders
• Article: Interactive Topic Modeling with BERTopic
• Article: Commonsense Reasoning for Natural Language Processing
• Article: Language Models
• Article: Zero shot NER using RoBERTA
• Article: Understanding Climate Change Domains through Topic Modeling
• Article: Simple PyTorch Transformer Example with Greedy Decoding
• Article: Topic Modeling for Keyword Extraction
• Article: Poor man’s GPT-3: Few shot text generation with T5 Transformer
• Article: Paraphrasing
• Article: Spelling Correction: How to make an accurate and fast corrector
• Article: Text Generation
• Article: How to steal modern NLP systems with gibberish?
• Article: Part of Speech Tagging with Hidden Markov Chain Models
• Article: Building RNNs is Fun with PyTorch and Google Colab
• Article: Automatically Summarize Trump’s State of the Union Address
• Article: Evaluation Metrics for Language Modeling
• Article: Does GPT-2 Know Your Phone Number?
• Article: ColumnTransformer Meets Natural Language Processing
• Article: pyLDAvis: Topic Modelling Exploration Tool That Every NLP Data Scientist Should Know
• Article: Explain NLP models with LIME & SHAP
• Article: When Topic Modeling is Part of the Text Pre-processing
• Article: Automatic Topic Labeling in 2018: History and Trends
• Article: The Annotated Transformer
• Article: Question Classification using Self-Attention Transformer — Part 1
• Article: Question Classification using Self-Attention Transformer — Part 1.1
• Article: Question Classification using Self-Attention Transformer — Part 2
• Article: Question Classification using Self-Attention Transformer — Part 3
• Article: Attention? An Other Perspective!: Part 1
• Article: Attention? An Other Perspective!: Part 2
• Article: Attention? An Other Perspective!: Part 3
• Article: Attention? An Other Perspective!: Part 4
• Article: Attention? An Other Perspective!: Part 5
• Article: On word embeddings - Part 1
• Article: On word embeddings - Part 2: Approximating the Softmax
• Article: On word embeddings - Part 3: The secret ingredients of word2vec
• Article: A survey of cross-lingual word embedding models
• Article: Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models
• Article: Semantic search using BERT embeddings
• Article: GPT-2 A nascent transfer learning method that could eliminate supervised learning in some NLP tasks
• Article: Brief review of word embedding families (2019)
• Article: Trends in input representation for state-of-art NLP models (2019)
• Article: An Overview of Multi-Task Learning in Deep Neural Networks
• Article: Deep Learning for NLP Best Practices
• Article: Learning to select data for transfer learning
• Article: Multi-Task Learning Objectives for Natural Language Processing
• Article: Word embeddings in 2017: Trends and future direction
• Article: Tracking the Progress in Natural Language Processing
• Article: NLP's ImageNet moment has arrived
• Article: A Review of the Neural History of Natural Language Processing
• Article: 10 Exciting Ideas of 2018 in NLP
• Article: The 4 Biggest Open Problems in NLP
• Article: Neural Transfer Learning for Natural Language Processing
• Article: The State of Transfer Learning in NLP
• Article: Unsupervised Cross-lingual Representation Learning
• Article: Why You Should Do NLP Beyond English
• Article: A review of BERT based models
• Article: Character level language model RNN
• Article: How To Create Natural Language Semantic Search For Arbitrary Objects With Deep Learning
• Article: How To Create Data Products That Are Magical Using Sequence-to-Sequence Models
• Article: State-of-the-Art Language Models in 2020
• Article: UNDERSTANDING WORD2VEC THROUGH CULTURAL DIMENSIONS
• Article: How to solve 90% of NLP problems: a step-by-step guide
• Article: The Unreasonable Effectiveness of Recurrent Neural Networks
• Article: How to Apply BERT to Arabic and Other Languages
• Article: The Illustrated Transformer
• Article: Under the Hood of RNNs
• Article: Gaussian Mixture Models for Clustering
• Article: What Semantic Search Can do for You
• Article: The Annotated GPT-2
• Article: How to Use n-gram Models to Detect Format Errors in Datasets
• Article: How to Use n-gram Models to Detect Format Errors in Datasets
• Article: Shrinking fastText embeddings so that it fits Google Colab
• Article: Perplexity Intuition (and its derivation)
• Article: How To Do Things With Words. And Counters
• Article: ML and NLP Publications in 2020
• Article: How GPT3 Works
• Article: Explaining RNNs without neural networks
• Article: Implementing Bengio’s Neural Probabilistic Language Model (NPLM) using Pytorch
• Doc: Huggingface Summary of the models
• Doc: Summary of the tokenizers
• Article: OpenAI's GPT-3 Language Model: A Technical Overview
• Article: Adapting Text Augmentation to Industry problems
• Article: How I Used Deep Learning To Train A Chatbot To Talk Like Me (Sorta)
• Article: Semantic Entailment
• Article: Feature-based Approach with BERT
• Article: Introduction to recurrent neural networks
• Article: Aspect-Based Opinion Mining (NLP with Python)
• Article: The Transformer Explained
• Article: Controlling Text Generation with Plug and Play Language Models
• Article: What makes a good conversation?
• Article: NLP for Supervised Learning - A Brief Survey
• Article: Generating Questions Using Transformers
• Article: Neural Language Models as Domain-Specific Knowledge Bases
• Article: Understanding BERT’s Semantic Interpretations
• Article: Using NLP (BERT) to improve OCR accuracy
• Article: Hyperparameter Optimization for 🤗Transformers: A guide
• Article: Faster and smaller quantized NLP with Hugging Face and ONNX Runtime
• Article: Text Similarities : Estimate the degree of similarity between two texts
• Article: Reducing Toxicity in Language Models
• Article: Learning Word Embedding
• Article: The Transformer Family
• Article: Generalized Language Models
• Article: Document clustering
• Article: The Unreasonable Effectiveness of Recurrent Neural Networks
• Article: LSTM Primer With Real Life Application( DeepMind Kidney Injury Prediction )*
• Article: Making sense of LSTMs by example
• Article: 3 subword algorithms help to improve your NLP model performance
• Article: Exploring LSTMs
• Article: Understanding LSTM Networks
• Article: 74 Summaries of Machine Learning and NLP Research
• Article: Smart Batching Tutorial - Speed Up BERT Training
• Article: GPU Benchmarks for Fine-Tuning BERT
• Article: Existing Tools for Named Entity Recognition
• Article: Domain-Specific BERT Models
• Article: Search (Pt 1) — A Gentle Introduction
• Article: Search (Pt 2) — A Semantic Horse Race
• Article: Search (Pt 3) — Elastic Transformers
• Article: How to Implement a Beam Search Decoder for Natural Language Processing
• Article: Speller100: Zero-shot spelling correction at scale for 100-plus languages
• Article: Breaking the spell of the spelling check
• Article: DaCy: New Fast and Efficient State-of-the-Art in Danish NLP!
• A friendly introduction to Recurrent Neural Networks
• Book: Embeddings in Natural Language Processing
• Book: Linguistic Fundamentals for Natural Language Processing: 100 Essentials from Morphology and Syntax
• Coursera: Sequence Models
• Coursera: Natural Language Processing in TensorFlow
• CMU: Low-resource NLP Bootcamp 2020
  • CMU Low resource NLP Bootcamp 2020 (1): NLP Tasks 1:46:06
  • CMU Low resource NLP Bootcamp 2020 (2): Linguistics - Phonology and Morphology 1:24:08
  • CMU Low resource NLP Bootcamp 2020 (3): Machine Translation 1:55:59
  • CMU Low resource NLP Bootcamp 2020 (4): Linguistics - Syntax and Morphosyntax 2:00:21
  • CMU Low resource NLP Bootcamp 2020 (5): Neural Representation Learning 1:19:57
  • CMU Low resource NLP Bootcamp 2020 (6): Multilingual NLP 2:04:34
  • CMU Low resource NLP Bootcamp 2020 (7): Speech Synthesis 2:22:14
• CMU: Neural Nets for NLP 2020
  • CMU Neural Nets for NLP 2020 (1): Introduction 1:11:38
  • CMU Neural Nets for NLP 2020 (2): Language Modeling, Efficiency/Training Tricks 1:18:31
  • CMU Neural Nets for NLP 2020 (3): Convolutional Neural Networks for Text 0:54:45
  • CMU Neural Nets for NLP 2020 (4): Recurrent Neural Networks 1:11:28
  • CMU Neural Nets for NLP 2020 (5): Efficiency Tricks for Neural Nets 1:05:37
  • CMU Neural Nets for NLP 2020 (6): Conditioned Generation 1:07:13
  • CMU Neural Nets for NLP 2020 (7): Attention 1:05:26
  • CMU Neural Nets for NLP 2020 (8): Distributional Semantics and Word Vectors 1:10:45
  • CMU Neural Nets for NLP 2020 (9): Sentence and Contextual Word Representations 1:16:19
  • CMU Neural Nets for NLP 2020 (10): Debugging Neural Nets (for NLP) 1:15:26
  • CMU Neural Nets for NLP 2020 (11): Structured Prediction with Local Independence Assumptions 1:08:38
  • CMU Neural Nets for NLP 2020 (12): Generating Trees Incrementally 1:14:13
  • CMU Neural Nets for NLP 2020 (13): Generating Trees Incrementally 0:51:58
  • CMU Neural Nets for NLP 2020 (14): Search-based Structured Prediction 1:06:19
  • CMU Neural Nets for NLP 2020 (15): Minimum Risk Training and Reinforcement Learning 1:09:16
  • CMU Neural Nets for NLP 2020 (16): Advanced Search Algorithms 1:03:02
  • CMU Neural Nets for NLP 2020 (17): Adversarial Methods 1:14:55
  • CMU Neural Nets for NLP 2020 (18): Models w/ Latent Random Variables 1:13:16
  • CMU Neural Nets for NLP 2020 (19): Unsupervised and Semi-supervised Learning of Structure 1:12:47
  • CMU Neural Nets for NLP 2020 (20): Multitask and Multilingual Learning 1:02:46
  • CMU Neural Nets for NLP 2020 (21): Document Level Models 0:52:04
  • CMU Neural Nets for NLP 2020 (22): Neural Nets + Knowledge Bases 1:18:39
  • CMU Neural Nets for NLP 2020 (23): Machine Reading w/ Neural Nets 1:06:11
  • CMU Neural Nets for NLP 2020 (24): Natural Language Generation 1:21:48
  • CMU Neural Nets for NLP 2020 (25): Model Interpretation 1:04:11
• CMU Multilingual NLP 2020
  • CMU Multilingual NLP 2020 (1): Introduction 1:17:29
  • CMU Multilingual NLP 2020 (2): Typology - The Space of Language 0:37:13
  • CMU Multilingual NLP 2020 (3): Words, Parts of Speech, Morphology 0:38:58
  • CMU Multilingual NLP 2020 (4): Text Classification and Sequence Labeling 0:45:56
  • CMU Multilingual NLP 2020 (5): Advanced Text Classification/Labeling 0:49:40
  • CMU Multilingual NLP 2020 (6): Translation, Evaluation, and Datasets 0:46:17
  • CMU Multilingual NLP 2020 (7): Machine Translation/Sequence-to-sequence Models 0:43:51
  • CMU Multilingual NLP 2020 (8): Data Augmentation for Machine Translation 0:24:42
  • CMU Multilingual NLP 2020 (9): Language Contact and Similarity Across Languages 0:30:25
  • CMU Multilingual NLP 2020 (10): Multilingual Training and Cross-lingual Transfer 0:39:58
  • CMU Multilingual NLP 2020 (11): Unsupervised Translation 0:51:17
  • CMU Multilingual NLP 2020 (12): Code Switching, Pidgins, and Creoles 0:46:37
  • CMU Multilingual NLP 2020 (13): Speech 0:41:16
  • CMU Multilingual NLP 2020 (14): Automatic Speech Recognition 0:39:33
  • CMU Multilingual NLP 2020 (15): Low Resource ASR 0:43:38
  • CMU Multilingual NLP 2020 (16): Text to Speech 0:39:00
  • CMU Multilingual NLP 2020 (17): Morphological Analysis and Inflection 0:45:22
  • CMU Multilingual NLP 2020 (18): Dependency Parsing 0:38:15
  • CMU Multilingual NLP 2020 (19): Data Annotation 0:53:08
  • CMU Multilingual NLP 2020 (20): Active Learning 0:28:37
  • CMU Multilingual NLP 2020 (21): Information Extraction 0:41:00
  • CMU Multilingual NLP 2020 (22): Multilingual NLP for Indigenous Languages 1:21:58
  • CMU Multilingual NLP 2020 (23): Universal Translation at Scale 1:27:33
• CMU: MultiModal Machine Learning Fall 2020
  • Lecture 1.1: Course Introduction
  • Lecture 1.2: Multimodal applications and datasets
  • Lecture 2.1: Basic concepts: neural networks
  • Lecture 2.2: Basic concepts: network optimization
  • Lecture 3.1: Visual unimodal representations
  • Lecture 3.2: Language unimodal representations
  • Lecture 4.1: Multimodal representation learning
  • Lecture 4.2: Coordinated representations
  • Lecture 5.1: Multimodal alignment
  • Lecture 5.2: Alignment and representation
  • Lecture 7.1: Alignment and translation
  • Lecture 7.2: Probabilistic graphical models
  • Lecture 8.1: Discriminative graphical models
  • Lecture 8.2: Deep Generative Models
  • Lecture 9.1: Reinforcement learning
  • Lecture 9.2: Multimodal RL
  • Lecture 10.1: Fusion and co-learning
  • Lecture 10.2: New research directions
• Datacamp: Advanced NLP with spaCy
• Datacamp: Building Chatbots in Python
• Datacamp: Clustering Methods with SciPy
• Datacamp: Feature Engineering for NLP in Python
• Datacamp: Machine Translation in Python
• Datacamp: Natural Language Processing Fundamentals in Python
• Datacamp: Natural Language Generation in Python
• Datacamp: RNN for Language Modeling
• Datacamp: Regular Expressions in Python
• Datacamp: Sentiment Analysis in Python
• Datacamp: Spoken Language Processing in Python
• Notebook: NNLM - Predict Next Word
• Notebook: Word2Vec
• Notebook: FastText Sentence Classification
• Notebook: TextCNN - Binary Sentiment Classification
• Notebook: TextRNN - Predict Next Step
• Notebook: TextLSTM - Autocomplete
• Notebook: Bi-LSTM - Predict Next Word in Long Sentence
• Notebook: SeqSeq - Change Word
• Notebook: Seq2Seq with Attention - Translate
• Notebook: Bi-LSTM with Attention - Binary Sentiment Classification
• Notebook: The Transformer - Translate
• Notebook: The Transformer - Greedy Decoder
• Notebook: BERT - NSP and MLM
• RNN and LSTM
• Spacy Tutorial
• Stanford CS224U: Natural Language Understanding | Spring 2019
  • Lecture 1 – Course Overview | Stanford CS224U: Natural Language Understanding | Spring 2019 1:12:59
  • Lecture 2 – Word Vectors 1 | Stanford CS224U: Natural Language Understanding | Spring 2019 1:17:10
  • Lecture 3 – Word Vectors 2 | Stanford CS224U: Natural Language Understanding | Spring 2019 1:16:52
  • Lecture 4 – Word Vectors 3 | Stanford CS224U: Natural Language Understanding | Spring 2019 0:38:20
  • Lecture 5 – Sentiment Analysis 1 | Stanford CS224U: Natural Language Understanding | Spring 2019 1:10:44
  • Lecture 6 – Sentiment Analysis 2 | Stanford CS224U: Natural Language Understanding | Spring 2019 1:03:23
  • Lecture 7 – Relation Extraction | Stanford CS224U: Natural Language Understanding | Spring 2019 1:19:04
  • Lecture 8 – NLI 1 | Stanford CS224U: Natural Language Understanding | Spring 2019 1:15:02
  • Lecture 9 – NLI 2 | Stanford CS224U: Natural Language Understanding | Spring 2019 1:15:35
  • Lecture 10 – Grounding | Stanford CS224U: Natural Language Understanding | Spring 2019 1:23:15
  • Lecture 11 – Semantic Parsing | Stanford CS224U: Natural Language Understanding | Spring 2019 1:07:05
  • Lecture 12 – Evaluation Methods | Stanford CS224U: Natural Language Understanding | Spring 2019 1:18:32
  • Lecture 13 – Evaluation Metrics | Stanford CS224U: Natural Language Understanding | Spring 2019 1:11:32
  • Lecture 14 – Contextual Vectors | Stanford CS224U: Natural Language Understanding | Spring 2019 1:14:33
  • Lecture 15 – Presenting Your Work | Stanford CS224U: Natural Language Understanding | Spring 2019 1:15:11
• Stanford CS224N: Stanford CS224N: NLP with Deep Learning | Winter 2019
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 1 – Introduction and Word Vectors 1:21:52
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 2 – Word Vectors and Word Senses 1:20:43
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 3 – Neural Networks 1:18:50
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 4 – Backpropagation 1:22:15
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 5 – Dependency Parsing 1:20:22
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 6 – Language Models and RNNs 1:08:25
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 7 – Vanishing Gradients, Fancy RNNs 1:13:23
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 8 – Translation, Seq2Seq, Attention 1:16:56
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 9 – Practical Tips for Projects 1:22:39
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 10 – Question Answering 1:21:01
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 11 – Convolutional Networks for NLP 1:20:18
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 12 – Subword Models 1:15:30
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 13 – Contextual Word Embeddings 1:20:18
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 14 – Transformers and Self-Attention 0:53:48
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 15 – Natural Language Generation 1:19:37
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 16 – Coreference Resolution 1:19:20
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 17 – Multitask Learning 1:11:54
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 18 – Constituency Parsing, TreeRNNs 1:20:37
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 19 – Bias in AI 0:56:03
  • Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 20 – Future of NLP + Deep Learning 1:19:15
  • Stanford CS224N: NLP with Deep Learning | Winter 2020 | Low Resource Machine Translation 1:15:45
  • Stanford CS224N: NLP with Deep Learning | Winter 2020 | BERT and Other Pre-trained Language Models 0:54:28
• Stanford: Natural Language Processing | Dan Jurafsky, Christopher Manning
  • Course Introduction 0:12:52
  • Regular Expressions 0:11:25
  • Regular Expressions in Practical NLP 0:06:05
  • Word Tokenization 0:14:26
  • Word Normalization and Stemming 0:11:48
  • Sentence Segmentation 0:05:34
  • Defining Minimum Edit Distance 0:07:05
  • Computing Minimum Edit Distance 0:05:55
  • Backtrace for Computing Alignments 0:05:56
  • Weighted Minimum Edit Distance 0:02:48
  • Minimum Edit Distance in Computational Biology 0:09:30
  • Introduction to N grams 0:08:41
  • Estimating N gram Probabilities 0:09:38
  • Evaluation and Perplexity 0:11:09
  • Generalization and Zeros 0:05:15
  • Smoothing Add One 0:06:31
  • Interpolation 0:10:25
  • Good Turing Smoothing 0:15:35
  • Kneser Ney Smoothing 0:08:59
  • The Spelling Correction Task 0:05:40
  • The Noisy Channel Model of Spelling 0:19:31
  • Real Word Spelling Correction 0:09:20
  • State of the Art Systems 0:07:10
  • What is Text Classification 0:08:12
  • Naive Bayes 0:03:20
  • Formalizing the Naive Bayes Classifier 0:09:29
  • Naive Bayes Learning 0:05:23
  • Naive Bayes Relationship to Language Modeling 0:04:36
  • Multinomial Naive Bayes A Worked Example 0:08:59
  • Precision, Recall, and the F measure 0:16:17
  • Text Classification Evaluation 0:07:17
  • Practical Issues in Text Classification 0:05:57
  • What is Sentiment Analysis 0:07:18
  • Sentiment Analysis A baseline algorithm 0:13:27
  • Sentiment Lexicons 0:08:38
  • Learning Sentiment Lexicons 0:14:46
  • Other Sentiment Tasks 0:11:02
  • Generative vs Discriminative Models 0:07:50
  • Making features from text for discriminative NLP models 0:18:12
  • Feature Based Linear Classifiers 0:13:35
  • Building a Maxent Model The Nuts and Bolts 0:08:05
  • Generative vs Discriminative models 0:12:10
  • Maximizing the Likelihood 0:10:30
  • Introduction to Information Extraction 0:09:19
  • Evaluation of Named Entity Recognition 0:06:35
  • Sequence Models for Named Entity Recognition 0:15:06
  • Maximum Entropy Sequence Models 0:13:02
  • What is Relation Extraction 0:09:47
  • Using Patterns to Extract Relations 0:06:17
  • Supervised Relation Extraction 0:10:51
  • Semi Supervised and Unsupervised Relation Extraction 0:09:53
  • The Maximum Entropy Model Presentation 0:12:14
  • Feature Overlap Feature Interaction 0:12:52
  • Conditional Maxent Models for Classification 0:04:11
  • Smoothing Regularization Priors for Maxent Models 0:29:24
  • An Intro to Parts of Speech and POS Tagging 0:13:19
  • Some Methods and Results on Sequence Models for POS Tagging 0:13:04
  • Syntactic Structure Constituency vs Dependency 0:08:46
  • Empirical Data Driven Approach to Parsing 0:07:11
  • The Exponential Problem in Parsing 0:14:31
  • Instructor Chat 0:09:03
  • CFGs and PCFGs 0:15:30
  • Grammar Transforms 0:12:06
  • CKY Parsing 0:23:26
  • CKY Example 0:21:25
  • Constituency Parser Evaluation 0:09:46
  • Lexicalization of PCFGs 0:07:03
  • Charniak's Model 0:18:24
  • PCFG Independence Assumptions 0:09:44
  • The Return of Unlexicalized PCFGs 0:20:53
  • Latent Variable PCFGs 0:12:08
  • Dependency Parsing Introduction 0:10:25
  • Greedy Transition Based Parsing 0:31:05
  • Dependencies Encode Relational Structure 0:07:21
  • Introduction to Information Retrieval 0:09:16
  • Term Document Incidence Matrices 0:08:59
  • The Inverted Index 0:10:43
  • Query Processing with the Inverted Index 0:06:44
  • Phrase Queries and Positional Indexes 0:19:46
  • Introducing Ranked Retrieval 0:04:27
  • Scoring with the Jaccard Coefficient 0:05:07
  • Term Frequency Weighting 0:06:00
  • Inverse Document Frequency Weighting 0:10:17
  • TF IDF Weighting 0:03:42
  • The Vector Space Model 0:16:23
  • Calculating TF IDF Cosine Scores 0:12:48
  • Evaluating Search Engines 0:09:03
  • Word Senses and Word Relations 0:11:50
  • WordNet and Other Online Thesauri 0:06:23
  • Word Similarity and Thesaurus Methods 0:16:18
  • Word Similarity Distributional Similarity I 0:13:15
  • Word Similarity Distributional Similarity II 0:08:16
  • What is Question Answering 0:07:29
  • Answer Types and Query Formulation 0:08:48
  • Passage Retrieval and Answer Extraction 0:06:38
  • Using Knowledge in QA 0:04:25
  • Advanced Answering Complex Questions 0:04:53
  • Introduction to Summarization 0:04:46
  • Generating Snippets 0:07:35
  • Evaluating Summaries ROUGE 0:05:03
  • Summarizing Multiple Documents 0:10:42
  • Instructor Chat II 0:05:24
• TextBlob Tutorial Series
  • Natural Language Processing Tutorial With TextBlob -Tokens,Translation and Ngrams 0:11:01
  • NLP Tutorial With TextBlob and Python - Parts of Speech Tagging 0:05:59
  • NLP Tutorial With TextBlob & Python - Lemmatizating 0:06:32
  • NLP Tutorial with TextBlob & Python - Sentiment Analysis(Polarity,Subjectivity) 0:06:31
  • Building a NLP-based Flask App with TextBlob 0:37:30
  • Natural Language Processing with Polyglot - Installation & Intro 0:12:49
• Youtube: fast.ai Code-First Intro to Natural Language Processing
  • What is NLP? (NLP video 1) 0:22:42
  • Topic Modeling with SVD & NMF (NLP video 2) 1:06:39
  • Topic Modeling & SVD revisited (NLP video 3) 0:33:05
  • Sentiment Classification with Naive Bayes (NLP video 4) 0:58:20
  • Sentiment Classification with Naive Bayes & Logistic Regression, contd. (NLP video 5) 0:51:29
  • Derivation of Naive Bayes & Numerical Stability (NLP video 6) 0:23:56
  • Revisiting Naive Bayes, and Regex (NLP video 7) 0:37:33
  • Intro to Language Modeling (NLP video 8) 0:40:58
  • Transfer learning (NLP video 9) 1:35:16
  • ULMFit for non-English Languages (NLP Video 10) 1:49:22
  • Understanding RNNs (NLP video 11) 0:33:16
  • Seq2Seq Translation (NLP video 12) 0:59:42
  • Word embeddings quantify 100 years of gender & ethnic stereotypes-- Nikhil Garg (NLP video 13) 0:47:17
  • Text generation algorithms (NLP video 14) 0:25:39
  • Implementing a GRU (NLP video 15) 0:23:13
  • Algorithmic Bias (NLP video 16) 1:26:17
  • Introduction to the Transformer (NLP video 17) 0:22:54
  • The Transformer for language translation (NLP video 18) 0:55:17
  • What you need to know about Disinformation (NLP video 19) 0:51:21
  • Article: Zero to Hero with fastai - Beginner
  • Article: Zero to Hero with fastai - Intermediate
• NLP Course | For You
  • Word Embeddings
  • Text Classification
  • Language Modeling
  • Seq2seq and Attention
• Youtube: BERT Research Series
• YouTube: Intro to NLP with Spacy
• Talk: Practical NLP for the Real World
• YouTube: Level 3 AI Assistant Conference 2020
  • Youtube: Conversation Analysis Theory in Chatbots | Michael Szul
  • Youtube: Designing Practical NLP Solutions | Ines Montani
  • Youtube: Effective Copywriting for Chatbots | Hans Van Dam
  • Youtube: Distilling BERT | Sam Sucik
  • Youtube: Transformer Policies that improve Dialogues: A Live Demo by Vincent Warmerdam
  • Youtube: From Research to Production – Our Process at Rasa | Tanja Bunk
  • Youtube: Keynote: Perspective on the 5 Levels of Conversational AI | Alan Nichol
• Youtube: RASA Algorithm Whiteboard
  • Introducing The Algorithm Whiteboard 0:01:16
  • Rasa Algorithm Whiteboard - Diet Architecture 1: How it Works 0:23:27
  • Rasa Algorithm Whiteboard - Diet Architecture 2: Design Decisions 0:15:06
  • Rasa Algorithm Whiteboard - Diet Architecture 3: Benchmarking 0:22:34
  • Rasa Algorithm Whiteboard - Embeddings 1: Just Letters 0:13:48
  • Rasa Algorithm Whiteboard - Embeddings 2: CBOW and Skip Gram 0:19:24
  • Rasa Algorithm Whiteboard - Embeddings 3: GloVe 0:19:12
  • Rasa Algorithm Whiteboard - Embeddings 4: Whatlies 0:14:03
  • Rasa Algorithm Whiteboard - Attention 1: Self Attention 0:14:32
  • Rasa Algorithm Whiteboard - Attention 2: Keys, Values, Queries 0:12:26
  • Rasa Algorithm Whiteboard - Attention 3: Multi Head Attention 0:10:55
  • Rasa Algorithm Whiteboard: Attention 4 - Transformers 0:14:34
  • Rasa Algorithm Whiteboard - StarSpace 0:11:46
  • Rasa Algorithm Whiteboard - TED Policy 0:16:10
  • Rasa Algorithm Whiteboard - TED in Practice 0:14:54
  • Rasa Algorithm Whiteboard - Response Selection 0:12:07
  • Rasa Algorithm Whiteboard - Response Selection: Implementation 0:09:25
  • Rasa Algorithm Whiteboard - Countvectors 0:13:32
  • Rasa Algorithm Whiteboard - Subword Embeddings 0:11:58
  • Rasa Algorithm Whiteboard - Implementation of Subword Embeddings 0:10:01
  • Rasa Algorithm Whiteboard - BytePair Embeddings 0:12:44
• Youtube: A brief history of the Transformer architecture in NLP
• Youtube: The Transformer neural network architecture explained. “Attention is all you need” (NLP)
• Youtube: How does a Transformer architecture combine Vision and Language? ViLBERT - NLP meets Computer Vision
• Youtube: Strategies for pre-training the BERT-based Transformer architecture – language (and vision)
• Youtube: Ilya Sutskever - GPT-2
• Youtube: NLP Masterclass | Modeling Fallacies in NLP
• Youtube: What is GPT-3? Showcase, possibilities, and implications
• Youtube: TextAttack: A Framework for Data Augmentation and Adversarial Training in NLP
• Article: How the Embedding Layers in BERT Were Implemented
• Youtube: Easy Data Augmentation for Text Classification
• Youtube: Webinar: Special NLP Session with Hugging Face
• Youtube: Spacy IRL 2019
  • Sebastian Ruder: Transfer Learning in Open-Source Natural Language Processing (spaCy IRL 2019) 0:31:24
  • Giannis Daras: Improving sparse transformer models for efficient self-attention (spaCy IRL 2019) 0:20:13
  • Peter Baumgartner: Applied NLP: Lessons from the Field (spaCy IRL 2019) 0:18:44
  • Justina Petraitytė: Lessons learned in helping ship conversational AI assistants (spaCy IRL 2019) 0:23:48
  • Yoav Goldberg: The missing elements in NLP (spaCy IRL 2019) 0:30:27
  • Sofie Van Landeghem: Entity linking functionality in spaCy (spaCy IRL 2019) 0:20:08
  • Guadalupe Romero: Rethinking rule-based lemmatization (spaCy IRL 2019) 0:14:49
  • Mark Neumann: ScispaCy: A spaCy pipeline & models for scientific & biomedical text (spaCy IRL 2019) 0:18:59
  • Patrick Harrison: Financial NLP at S&P Global (spaCy IRL 2019) 0:21:42
  • McKenzie Marshall: NLP in Asset Management (spaCy IRL 2019) 0:20:32
  • David Dodson: spaCy in the News: Quartz's NLP pipeline (spaCy IRL 2019) 0:20:56
  • Matthew Honnibal & Ines Montani: spaCy and Explosion: past, present & future (spaCy IRL 2019) 0:54:13
• Youtube: The Future of Natural Language Processing
• Youtube: Sentiment Analysis: Key Milestones, Challenges and New Directions
• Youtube: Simple and Efficient Deep Learning for Natural Language Processing, with Moshe Wasserblat, Intel AI
• Youtube: Why not solve biological problems with a Transformer? BERTology meets Biology
• Youtube: Self-attention step-by-step | How to get meaning from text
• Youtube: Chat Bot with PyTorch
• Youtube: NLP with Friends Talks
  • NLP with Friends, Featured Friend: Tom McCoy 0:36:48
  • NLP with Friends, Featured Friend: Maarten Sap 0:36:11
  • NLP with Friends, featured friend: Nitika Mathur 1:01:42
  • NLP with Friends, Featured Friend: Sabrina J Mielke 1:01:28
• Youtube: Insincere Question Classification with PyTorch
  • [PART 1] Insincere Question Classification with PyTorch 0:22:34
  • [PART 2] Insincere Question Classification with PyTorch 0:33:53
  • [PART 3] Insincere Question Classification with PyTorch 0:23:49
  • [PART 4] Insincere Question Classification with PyTorch 0:14:25
• Crash Course: Linguistics
  • Crash Course Linguistics Preview 0:02:50
  • What is Linguistics?: Crash Course Linguistics #1 0:11:11
• Youtube: Recent Advances in Language Pretraining and Generation
• Youtube: Talks # 3: Lorenzo Ampil - Introduction to T5 for Sentiment Span Extraction
• Youtube: Frontiers in ML: Learning from Limited Labeled Data: Challenges and Opportunities for NLP
• Youtube: DeepLearning.ai NLP talk: Chris Manning
• Youtube: DeepLearning.ai NLP talk: Oren Etzioni
• Youtube: DeepLearning.ai NLP talk: Quoc Le
• Youtube: What can MIR learn from transfer learning in NLP?
• Youtube: The Narrated Transformer Language Model
• Youtube: spaCy v3.0: Bringing State-of-the-art NLP from Prototype to Production 00:22:40
• Youtube: Conversational AI with Transformers and Rule-Based Systems 1:53:24
• Talk: High Performance Natural Language Processing
• Talk: EmoTag1200: Understanding the Association between Emojis and Emotions
• Youtube: Research Paper Walkthrough
  • Simple Unsupervised Keyphrase Extraction using Sentence Embeddings | Research Paper Walkthrough 0:21:23
  • Leveraging BERT for Extractive Text Summarization on Lectures | Research Paper Walkthrough 0:20:10
  • Data Augmentation Techniques for Text Classification in NLP | Research Paper Walkthrough 0:14:33
  • CRIM at SemEval-2018 Task 9: A Hybrid Approach to Hypernym Discovery | Research Paper Walkthrough 0:23:47
  • Data Augmentation using Pre-trained Transformer Model (BERT, GPT2, etc) | Research Paper Walkthrough 0:17:43
  • A Supervised Approach to Extractive Summarisation of Scientific Papers | Research Paper Walkthrough 0:19:01
  • BLEURT: Learning Robust Metrics for Text Generation | Research Paper Walkthrough 0:13:38
  • DeepWalk: Online Learning of Social Representations | ML with Graphs | Research Paper Walkthrough 0:17:44
  • LSBert: A Simple Framework for Lexical Simplification | Research Paper Walkthrough 0:20:27
  • SpanBERT: Improving Pre-training by Representing and Predicting Spans | Research Paper Walkthrough 0:14:21
  • Text Summarization of COVID-19 Medical Articles using BERT and GPT-2 | Research Paper Walkthrough 0:21:52
  • Extractive & Abstractive Summarization with Transformer Language Models | Research Paper Walkthrough 0:16:58
  • Unsupervised Multi-Document Summarization using Neural Document Model | Research Paper Walkthrough 0:15:11
  • SummPip: Multi-Document Summarization with Sentence Graph Compression | Research Paper Walkthrough 0:16:54
  • Combining BERT with Static Word Embedding for Categorizing Social Media | Research Paper Walkthrough 0:13:51
  • Reformulating Unsupervised Style Transfer as Paraphrase Generation | Research Paper Walkthrough 0:19:41
  • PEGASUS: Pre-training with Gap-Sentences for Abstractive Summarization | Research Paper Walkthrough 0:15:04
  • Evaluation of Text Generation: A Survey | Human-Centric Evaluations | Research Paper Walkthrough 0:15:54
  • TOD-BERT: Pre-trained Transformers for Task-Oriented Dialogue Systems (Research Paper Walkthrough) 0:15:25
  • TextRank: Bringing Order into Texts (Research Paper Walkthrough) 0:14:34
  • Node2Vec: Scalable Feature Learning for Networks | ML with Graphs (Research Paper Walkthrough) 0:14:33
  • HARP: Hierarchical Representation Learning for Network | ML with Graphs (Research Paper Walkthrough) 0:15:10
  • URL2Video: Automatic Video Creation From a Web Page | AI and Creativity (Research Paper Walkthrough) 0:15:21
  • On Generating Extended Summaries of Long Documents (Research Paper Walkthrough) 0:14:24
  • Nucleus Sampling: The Curious Case of Neural Text Degeneration (Research Paper Walkthrough) 0:12:48
  • T5: Exploring Limits of Transfer Learning with Text-to-Text Transformer (Research Paper Walkthrough) 0:12:47
  • DialoGPT: Generative Training for Conversational Response Generation (Research Paper Walkthrough) 0:13:17
  • Hierarchical Transformers for Long Document Classification (Research Paper Walkthrough) 0:12:46
  • Beyond Accuracy: Behavioral Testing of NLP Models with CheckList (Best Paper ACL 2020) 0:14:00
• NLP Summit 2020
  • The 2020 Trends for Applied Natural Language Processing | NLP Summit 2020 0:21:10
  • NLP Industry Survey Analysis: the landscape of natural language use cases in 2020 | NLP Summit 2020 0:20:23
  • Auto NLP: Pretrain, Tune & Deploy State-of-the-art Models Without Coding 0:19:57
  • Proof-of-Concept Delight | NLP Summit 2020 0:16:50
  • Distributed Natural Language Processing Apps for Financial Engineering | NLP Summit 2020 0:34:49
  • Bleeding Edge Applications of 2020 Transformers | NLP Summit 2020 0:33:34
  • How Freshworks Freddy AI leverages NLP for Ethics-First Customer Experiences | NLP Summit 2020 0:26:49
  • NLP for Recruitment Automation: Building a Chatbot from the Job Description | NLP Summit 2020 0:22:31
• Youtube: Explainability for Natural Language Processing
• Youtube: Gibberish Detector
• Youtube: NLP Lecture 7 Constituency Parsing
  • NLP Lecture 7 - Overview of Constituency Parsing Lecture 0:01:50
  • NLP Lecture 7 - Introduction to Constituency Parsing 0:10:29
  • NLP Lecture 7(a) - Context Free Grammar 0:17:03
  • NLP Lecture 7(b) - Constituency Parsing 0:13:28
  • NLP Lecture 7(c) - Statistical Constituency Parsing 0:09:38
  • NLP Lecture 7(d) - Dependency Parsing 0:17:15
• Youtube: LING 83 Teaching Video: Constituency Parsing
• Youtube: SpaCy for Digital Humanities with Python Tutorials
  • Introduction to SpaCy and Cleaning Data (SpaCy and Python Tutorials for DH - 01) 0:06:07
  • How to Install SpaCy and Models (Spacy and Python Tutorial for DH 02) 0:07:40
  • How to Separate Sentences in SpaCy (SpaCy and Python Tutorials for DH - 03) 0:08:33
  • Spacy and Named Entity Recognition NER (Spacy and Python Tutorial for DH 04) 0:08:32
  • Finding Parts of Speech (SpaCy and Python Tutorial for DH 05) 0:02:55
  • Extracting Nouns and Noun Chunks (SpaCy and Python Tutorial for DH 06) 0:05:46
  • Extracting Verbs and Verb Phrases (SpaCy and Python Tutorial for DH 07) 0:08:10
  • Lemmatization: Finding the Roots of Words (Spacy and Python Tutorial for DH 08) 0:04:52
  • Data Visualization with DisplaCy (Spacy and Python Tutorial for DH 09) 0:09:13
  • Customizing DisplaCy Render Data Visualization (Spacy and Python Tutorial for DH 10) 0:08:19
  • Finding Quotes in Sentences (SpaCy and Python Tutorial for DH 11) 0:08:45
  • Introduction to Named Entity Recognition (NER for DH 01) 0:16:43
  • Machine Learning NER with Python and spaCy (NER for DH 03 ) 0:13:36
  • How to Use spaCy's EntityRuler (Named Entity Recognition for DH 04 | Part 01) 0:36:50
  • How to Use spaCy to Create an NER training set (Named Entity Recognition for DH 04 | Part 02) 0:10:32
  • How to Train a spaCy NER model (Named Entity Recognition for DH 04 | Part 03) 0:15:40
  • Examining a spaCy Model in the Folder (Named Entity Recognition for DH 05) 0:15:06
  • What are Word Vectors (Named Entity Recognition for DH 06) 0:18:49
  • How to Generate Custom Word Vectors in Gensim (Named Entity Recognition for DH 07) 0:23:05
  • How to Load Custom Word Vectors into spaCy Models (Named Entity Recognition for DH 08) 0:10:46
  • Getting the Data for Custom Labels (Holocaust NER for DH 09.01) 0:11:00
  • How to Add a Custom NER Pipe in spaCy and a Custom Label (NER for DH 09.02 ) 0:07:49
  • How to Training Custom Entities into spaCy Models (Named Entity Recognition for DH 09 03) 0:15:29
  • How to Add and Place Pipes from other Models into a New Model (NER for DH 09 04) 0:12:24
  • How to Add Custom Functions to spaCy Pipeline (NER for DH 09.05) 0:15:20
  • Precision vs. Recall and Adding PERSON to Holocaust NER Pipeline (Named Entity Recognition DH 09.06) 0:26:02
  • Finalizing the Holocaust NER Pipeline (Named Entity Recognition for DH 09.07) 0:14:16
  • Classical Latin Named Entity Recognition (NER for DH 10.01) 0:55:30
  • How to Package spaCy Models (Even with Custom Factories) (NER for DH 11) 0:15:31
• Youtube: Billion-scale Approximate Nearest Neighbor Search
• Youtube: Data Science - Fuzzy Record Matching
• Youtube: Minimum Edit Distance Dynamic Programming
• Youtube: Cheuk Ting Ho - Fuzzy Matching Smart Way of Finding Similar Names Using Fuzzywuzzy
• Youtube: What's in a Name? Fast Fuzzy String Matching - Seth Verrinder & Kyle Putnam - Midwest.io 2015
• Youtube: Jiaqi Liu Fuzzy Search Algorithms How and When to Use Them PyCon 2017
• Youtube: 1 + 1 = 1 or Record Deduplication with Python | Flávio Juvenal @ PyBay2018
• Youtube: Mike Mull: The Art and Science of Data Matching
• Youtube: Record linkage: Join for real life by Rhydwyn Mcguire
• Youtube: Approximate nearest neighbors and vector models, introduction to Annoy
• Youtube: Librosa Audio and Music Signal Analysis in Python | SciPy 2015 | Brian McFee
• Video: Recent Advances in LM Pre-training
• Youtube: Deep Learning (for Audio) with Python
  • 1- Deep Learning (for Audio) with Python: Course Overview 0:08:02
  • 2- AI, machine learning and deep learning 0:31:15
  • 3- Implementing an artificial neuron from scratch 0:19:05
  • 4- Vector and matrix operations 0:25:51
  • 5- Computation in neural networks 0:23:19
  • 6- Implementing a neural network from scratch in Python 0:21:03
  • 7- Training a neural network: Backward propagation and gradient descent 0:21:41
  • 8- TRAINING A NEURAL NETWORK: Implementing backpropagation and gradient descent from scratch 1:03:00
  • 9- How to implement a (simple) neural network with TensorFlow 2 0:24:37
  • 10 - Understanding audio data for deep learning 0:32:55
  • 11- Preprocessing audio data for Deep Learning 0:25:05
  • 12- Music genre classification: Preparing the dataset 0:37:45
  • 13- Implementing a neural network for music genre classification 0:33:25
  • 14- SOLVING OVERFITTING in neural networks 0:26:29
  • 15- Convolutional Neural Networks Explained Easily 0:35:23
  • 16- How to Implement a CNN for Music Genre Classification 0:49:10
  • 17- Recurrent Neural Networks Explained Easily 0:28:35
  • 18- Long Short Term Memory (LSTM) Networks Explained Easily 0:28:08
  • 19- How to Implement an RNN-LSTM Network for Music Genre Classification 0:14:29
• Youtube: Fine-tuning a large language model without your own supercomputer
• Youtube: How to build a custom spell checker using python NLP

Be familiar with Recommendation Systems

• Google: Recommendation Systems
• Pluralsight: Understanding Algorithms for Recommendation Systems
• Youtube: Learning to Rank: From Theory to Production - Malvina Josephidou & Diego Ceccarelli, Bloomberg
• Youtube: Learning "Learning to Rank"
• Youtube: Learning to rank search results - Byron Voorbach & Jettro Coenradie [DevCon 2018]

Be able to implement computer vision models

• Article: The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)
• Article: How to extract Key-Value pairs from Documents using deep learning
• Article: Essential Pil (Pillow) Image Tutorial (for Machine Learning People)
• Article: What is Focal Loss and when should you use it?
• Article: Part 1: Deep Representations, a way towards neural style transfer
• Article: A gentle introduction to OCR
• Article: Breaking Linear Classifiers on ImageNet
• Article: Building an image search service from scratch
• Article: Squeeze and Excitation Networks Explained with PyTorch Implementation
• Article: Multimodal Neurons in Artificial Neural Networks
• Article: DenseNet Architecture Explained with PyTorch Implementation from TorchVision
• Article: EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Article: Group Normalization
• Article: A Short Introduction to Generative Adversarial Networks
• Article: Semi-supervised Learning with GANs
• Article: Densely Connected Convolutional Networks in Tensorflow
• Article: Convolutional neural networks
• Article: Common architectures in convolutional neural networks
• Article: An overview of semantic image segmentation
• Article: Evaluating image segmentation models
• Article: An overview of object detection: one-stage methods
• Article: A Brief History of CNNs in Image Segmentation: From R-CNN to Mask R-CNN
• Article: Object Detection for Dummies Part 1: Gradient Vector, HOG, and SS
• Article: Object Detection for Dummies Part 2: CNN, DPM and Overfeat
• Article: Object Detection for Dummies Part 3: R-CNN Family
• Article: Understanding coordinate systems and DICOM for deep learning medical image analysis
• Article: Understanding the receptive field of deep convolutional networks
• Article: Deep learning in medical imaging - 3D medical image segmentation with PyTorch
• Article: Intuitive Explanation of Skip Connections in Deep Learning
• Article: Human Pose Estimation
• Article: YOLO - You only look once (Single shot detectors)
• Article: Localization and Object Detection with Deep Learning
• Article: Semantic Segmentation in the era of Neural Networks
• Article: ECCV 2020: Some Highlights
• Article: NonCompositional
• Article: Looking Inside The Blackbox — How To Trick A Neural Network
• AWS: Semantic Segmentation Explained
• Book: Deep Learning for Computer Vision with Python
• Book: Practical Python and OpenCV
• Coursera: Convolutional Neural Networks
• Datacamp: Biomedical Image Analysis in Python
• Datacamp: Image Processing in Python
• Google: ML Practicum: Image Classification
• Stanford: CS231N Winter 2016
  • CS231n Winter 2016: Lecture 1: Introduction and Historical Context 1:19:08
  • CS231n Winter 2016: Lecture 2: Data-driven approach, kNN, Linear Classification 1 0:57:28
  • CS231n Winter 2016: Lecture 3: Linear Classification 2, Optimization 1:11:23
  • CS231n Winter 2016: Lecture 4: Backpropagation, Neural Networks 1 1:19:38
  • CS231n Winter 2016: Lecture 5: Neural Networks Part 2 1:18:37
  • CS231n Winter 2016: Lecture 6: Neural Networks Part 3 / Intro to ConvNets 1:09:35
  • CS231n Winter 2016: Lecture 7: Convolutional Neural Networks 1:19:01
  • CS231n Winter 2016: Lecture 8: Localization and Detection 1:04:57
  • CS231n Winter 2016: Lecture 9: Visualization, Deep Dream, Neural Style, Adversarial Examples 1:18:20
  • CS231n Winter 2016: Lecture 10: Recurrent Neural Networks, Image Captioning, LSTM 1:09:54
  • CS231n Winter 2016: Lecture 11: ConvNets in practice 1:15:03
  • CS231n Winter 2016: Lecture 12: Deep Learning libraries 1:21:06
  • CS231n Winter 2016: Lecture 14: Videos and Unsupervised Learning 1:17:36
  • CS231n Winter 2016: Lecture 13: Segmentation, soft attention, spatial transformers 1:10:59
  • CS231n Winter 2016: Lecture 15: Invited Talk by Jeff Dean 1:14:49
• Udacity: Introduction to Computer Vision
• Youtube: Deep Residual Learning for Image Recognition (Paper Explained)
• Youtube: Implementing ResNet from scratch
• Youtube: ConvNets Scaled Efficiently 0:13:19
• Youtube: Building an Image Captioner with Neural Networks 0:12:54
• Youtube: Evolution of Face Generation | Evolution of GANs 0:12:23
• Youtube: Autoencoders - EXPLAINED 0:10:53
• Youtube: Unpaired Image-Image Translation using CycleGANs 0:16:22
• Youtube: AI creates Image Classifiers…by DRAWING? 0:09:04
• Youtube: The Evolution of Convolution Neural Networks 0:24:02
• Youtube: Depthwise Separable Convolution - A FASTER CONVOLUTION! 0:12:43
• Youtube: Mask Region based Convolution Neural Networks - EXPLAINED! 0:09:34
• Youtube: Sound play with Convolution Neural Networks 0:11:57
• Youtube: Convolution Neural Networks - EXPLAINED 0:19:20
• Youtube: Generative Adversarial Networks - FUTURISTIC & FUN AI ! 0:14:20
• Youtube: How Convolution Works
• Youtube: DETR: End-to-End Object Detection with Transformers (Paper Explained)
• Youtube: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (Paper Explained)

Be able to model graphs and network data

• Datacamp: Network Analysis in Python (Part 1)
• Datacamp: Network Analysis in Python (Part 2)

Be able to implement models for timeseries and forecasting

• Datacamp: Machine Learning for Finance in Python
• Datacamp: Introduction to Time Series Analysis in Python
• Datacamp: Machine Learning for Time Series Data in Python
• Datacamp: Intro to Portfolio Risk Management in Python
• Datacamp: Financial Forecasting in Python
• Datacamp: Predicting CTR with Machine Learning in Python
• Datacamp: Intro to Financial Concepts using Python
• Datacamp: Fraud Detection in Python
• Datacamp: Forecasting Using ARIMA Models in Python
• Datacamp: Introduction to Portfolio Analysis in Python
• Datacamp: Credit Risk Modeling in Python
• Datacamp: Machine Learning for Marketing in Python
• Udacity: Machine Learning for Trading
• Udacity: Time Series Forecasting

Be familiar with Reinforcement Learning

• DeepLizard: Reinforcement Learning - Goal Oriented Intelligence
  • Reinforcement Learning Series Intro - Syllabus Overview 0:05:51
  • Markov Decision Processes (MDPs) - Structuring a Reinforcement Learning Problem 0:06:34
  • Expected Return - What Drives a Reinforcement Learning Agent in an MDP 0:06:47
  • Policies and Value Functions - Good Actions for a Reinforcement Learning Agent 0:06:52
  • What do Reinforcement Learning Algorithms Learn - Optimal Policies 0:06:21
  • Q-Learning Explained - A Reinforcement Learning Technique 0:08:37
  • Exploration vs. Exploitation - Learning the Optimal Reinforcement Learning Policy 0:10:06
  • OpenAI Gym and Python for Q-learning - Reinforcement Learning Code Project 0:07:52
  • Train Q-learning Agent with Python - Reinforcement Learning Code Project 0:08:59
  • Watch Q-learning Agent Play Game with Python - Reinforcement Learning Code Project 0:07:22
  • Deep Q-Learning - Combining Neural Networks and Reinforcement Learning 0:10:50
  • Replay Memory Explained - Experience for Deep Q-Network Training 0:06:21
  • Training a Deep Q-Network - Reinforcement Learning 0:09:07
  • Training a Deep Q-Network with Fixed Q-targets - Reinforcement Learning 0:07:35
  • Deep Q-Network Code Project Intro - Reinforcement Learning 0:06:26
  • Build Deep Q-Network - Reinforcement Learning Code Project 0:16:51
  • Deep Q-Network Image Processing and Environment Management - Reinforcement Learning Code Project 0:21:53
  • Deep Q-Network Training Code - Reinforcement Learning Code Project 0:19:46

Be able to optimize performance metric

• A recipe for training neural networks
• Article: Evaluating a machine learning model
• Article: Hyperparameter tuning for machine learning models
• Article: Hacker's Guide to Hyperparameter Tuning
• Article: Environment and Distribution Shift
• Coursera: Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
• Datacamp: Model Validation in Python
• Datacamp: Hyperparameter Tuning in Python
• Google: Testing and Debugging
• Troubleshooting Deep Neural Networks
• Youtube: How do GPUs speed up Neural Network training? 0:08:19
• Youtube: Why use GPU with Neural Networks? 0:09:43
• Youtube: Auto-Tuning Hyperparameters with Optuna and PyTorch

Be familiar with literature on model interpretability

• Article: How to leverage Explainable Machine Learning
• Article: TracIn — A Simple Method to Estimate Training Data Influence
• NeurIPS 2020: Tutorial on Explaining ML Predictions: State-of-the-art, Challenges, and Opportunities
• Youtube: Jay Alammar - Take A Look Inside Language Models With Ecco

Be able to optimize model size for production

• Article: A Survey of Methods for Model Compression in NLP
• Article: Why you should convert your NLP pipelines to ONNX
• Article: Neural Network Pruning
• Article: FasterAI
• Article: Is the future of Neural Networks Sparse? An Introduction (1/N)
• Article: Sparse Neural Networks (2/N): Understanding GPU Performance.
• Article: Block Sparse Matrices for Smaller and Faster Language Models
• Article: Plunging Into Model Pruning in Deep Learning
• Article: How to accelerate and compress neural networks with quantization

Be able to write unit tests

• Article: Effective testing for machine learning systems
• Article: Unit Testing for Data Scientists
• Article: Testing in Production, the safe way
• Article: How to cheat at unit tests with pytest and Black
• Article: 4 Lesser-Known Yet Awesome Tips for Pytest
• Datacamp: Unit Testing for Data Science in Python
• Pluralsight: Test-driven Development: The Big Picture
• Test Driven Development with Python
• Thoughtbot: Fundamentals of TDD
• Udacity: Software Analysis & Testing
• Udacity: Software Testing
• Udacity: Software Debugging
• Youtube: Beyond Accuracy: Behavioral Testing of NLP Models with CheckList | AISC

Be able to serve models via APIs

• Article: Deploy a Keras Deep Learning Project to Production with Flask
• Article: Deploy BERT for Sentiment Analysis as REST API using PyTorch, Transformers by Hugging Face and FastAPI
• Article: Microservice in Python using FastAPI
• Article: Selecting gunicorn worker types for different python web applications.
• Article: Better performance by optimizing Gunicorn config
• Article: Exponential Backoff And Jitter
• Django Best Practices
• Udacity: Authentication & Authorization: OAuth
• Udacity: HTTP & Web Servers
• Udacity: Designing RESTful APIs
• Udacity: Client-Server Communication
• Youtube: PyConBY 2020: Sebastian Ramirez - Serve ML models easily with FastAPI
• Youtube: FastAPI from the ground up
• Youtube: Python pydantic Introduction – Give your data classes super powers

Be able to build interactive UI for models

• Article: Build and Deploy a Dashboard with Streamlit
• Article: New layout options for Streamlit

Be able to deploy model to production

• Acloudguru: AWS Certified Machine Learning - Specialty
• Acloudguru: AWS Certified Developer - Associate
• Acloudguru: AWS Certification Preparation Guide
• AWS: Exam Readiness: AWS Certified Developer – Associate
• AWS: Thirty Serverless Architectures in 30 Minutes
• Article: Celery Execution Pools: What is it all about?
• Article: Getting machine learning to production
• Article: A Guide to Production Level Deep Learning
• Article: MLOps concepts for busy engineers: model serving
• Article: How to put machine learning models into production
• Article: Monitoring your Machine Learning Model
• Article: How to Deploy a Machine Learning Model
• Article: Smaller Docker images with Conda
• Article: Deploying conda environments in (Docker) containers - how to do it right
• Article: How to scale services using Docker Compose
• Article: Understand Linux Load Averages and Monitor Performance of Linux
• Article: Preventing model drift with continuous monitoring and deployment using Github Actions and Algorithmia Insights
• Article: Building a feature store
• Article: Combining rule engines and machine learning
• Article: Model artifacts: the war stories
• Article: Shadow mode deployments
• Article: When is a neural net too big for production?
• Article: Tensorflow in Docker
• Article: How to build scalable Machine Learning systems — Part 1/2
• Article: Architecting a Machine Learning Pipeline
• Article: Deploying Machine Learning Models: A Checklist
• Article: How to Serve Models
• Article: How to Monitor Models
• Article: Production Machine Learning Monitoring: Outliers, Drift, Explainers & Statistical Performance
• Article: Machine learning is going real-time
• Article: The correct way to evaluate online machine learning models
• Article: Machine Learning to Production
• Article: Online batching with Spell serving
• Article: A deep dive into AWS spot instance interruptions
• Article: Key Concepts for Deploying Machine Learning Models to Mobile
• Article: Configuring Gunicorn for Docker
• Article: How To Pass Environment Info During Docker Builds
• Article: Pass Docker Environment Variables During The Image Build
• Article: Distill: Why do we need Flask, Celery, and Redis? (with McDonalds in Between)
• Article: ML Infrastructure Tools for Data Preparation
• Article: ML Infrastructure Tools for Model Building
• Article: ML Infrastructure Tools for Production (Part 1)
• Article: ML Infrastructure Tools for Production
• Article: Using Statistical Distances for Machine Learning Observability
• Article: ML Infrastructure Tools — ML Observability
• Article: The Model’s Shipped; What Could Possibly go Wrong?
• Article: The Playbook to Monitor Your Model’s Performance in Production
• Article: Setting Default Docker Environment Variables During Image Build
• Article: Getting oriented in the RAPIDS distributed ML ecosystem, part 1: ETL
• Article: Getting oriented in the RAPIDS distributed ML ecosystem, part 2: training and scoring
• Article: Making model training scripts robust to spot interruptions
• Article: Getting started with large-scale ETL jobs using Dask and AWS EMR
• Article: Distributed model training using Horovod
• Article: MLOps concepts for busy engineers: model serving
• Article: Enough Docker to be Dangerous
• Article: How Docker Can Help You Become A More Effective Data Scientist
• Article: How to properly ship and deploy your machine learning model
• Article: Docker Explained Visually, For Non-Technical Folks
• Article: A Beginner-Friendly Introduction to Containers, VMs and Docker
• Article: The Playbook to Monitor Your Model’s Performance in Production
• Blog: Python Speed
  • Connection refused? Docker networking and how it impacts your image
  • Faster or slower: the basics of Docker build caching
  • Where’s your code? Debugging ImportError and ModuleNotFoundErrors in your Docker image
  • A tableau of crimes and misfortunes: the ever-useful docker history
  • Broken by default: why you should avoid most Dockerfile examples
  • A review of the official Dockerfile best practices: good, bad, and insecure
  • The best Docker base image for your Python application (February 2021)
  • A deep dive into the official Docker image for Python
  • Using Alpine can make Python Docker builds 50× slower
  • When to switch to Python 3.9
  • Building on solid ground: ensuring reproducible Docker builds for Python
  • Push and pull: when and why to update your dependencies
  • Installing system packages in Docker with minimal bloat
  • Less capabilities, more security: minimizing privilege escalation in Docker
  • Avoiding insecure images from Docker build caching
  • Build secrets in Docker and Compose, the secure way
  • Security scanners for Python and Docker: from code to dependencies
  • The high cost of slow Docker builds
  • Faster Docker builds with pipenv, poetry, or pip-tools
  • Elegantly activating a virtualenv in a Dockerfile
  • Poetry vs. Docker caching: Fight!
  • Speed up pip downloads in Docker with BuildKit’s new caching
  • Multi-stage builds #1: Smaller images for compiled code
  • Multi-stage builds #2: Python specifics—virtualenv, –user, and other methods
  • Multi-stage builds #3: Why your build is surprisingly slow, and how to speed it up
  • Configuring Gunicorn for Docker
  • Activating a Conda environment in your Dockerfile
  • Shrink your Conda Docker images with conda-pack
  • Reproducible and upgradable Conda environments: dependency management with conda-lock
  • What’s running in production? Making your Docker images identifiable
  • Decoupling database migrations from server startup: why and how
  • A Python prompt into a running process: debugging with Manhole
  • A thousand little details: developing software for ops
  • Your Docker build needs a smoke test
  • Where’s that log file? Debugging failed Docker builds
  • “Let’s use Kubernetes!” Now you have 8 problems
  • Docker BuildKit: faster builds, new features, and now it’s stable
  • Options for packaging your Python code: Wheels, Conda, Docker, and more
  • Docker vs. Singularity for data processing: UIDs and filesystem access
• Cortex Blog
  • Server-side batching: Scaling inference throughput in machine learning
  • How we served 1,000 models on GPUs for $0.47
  • Designing a machine learning platform for both data scientists and engineers
  • How to serve batch predictions with TensorFlow Serving
  • How to deploy Transformer models for language tasks
  • How we scale machine learning model deployment on Kubernetes
  • Why we built a serverless machine learning platform—instead of using AWS Lambda
  • Why we don’t deploy machine learning models with Flask
  • How to deploy machine learning models from a notebook to production
  • A/B testing machine learning models in production
  • How to deploy 1,000 models on one CPU with TensorFlow Serving
  • How to reduce the cost of machine learning inference
  • Improve NLP inference throughput 40x with ONNX and Hugging Face
  • How to deploy PyTorch Lightning models to production
• Doc: Environment variables in Compose
• Doc: Lecture 3: Data engineering
• Luigi Patruno: ML in Production
  • Video: You trained a machine learning model. Now what?
  • Article: Docker for Machine Learning – Part I
  • Article: Docker for Machine Learning – Part II
  • Article: Docker for Machine Learning – Part III
  • Article: Using Docker to Generate Machine Learning Predictions in Real Time
  • Article: Batch Inference vs Online Inference
  • Article: Storing Metadata from Machine Learning Experiments
  • Article: How Data Leakage Impacts Machine Learning Models
  • Article: An Introduction to Kubernetes for Data Scientists
  • Article: How to Use Kubernetes Pods for Machine Learning
  • Article: Kubernetes Jobs for Machine Learning
  • Article: Kubernetes CronJobs for Machine Learning
  • Article: Kubernetes Deployments for Machine Learning
  • Article: Kubernetes Services for Machine Learning
  • Article: The Ultimate Guide to Model Retraining
  • Article: Top ML Resources: Interview with Eric Colson
  • Article: Top ML Resources: Interview with Veronika Megler, PhD
  • Article: Top ML Resources: Interview with Erik Bernhardsson
  • Article: Top ML Resources: Interview with Rui Carmo
  • Article: Top ML Resources: Interview with Jeremy Jordan
  • Article: 5 Challenges to Running Machine Learning Systems in Production
  • Article: Enabling Machine-Learning-as-a-Service Through Privacy Preserving Machine Learning
  • Article: What Does it Mean to Deploy a Machine Learning Model? (Deployment Series: Guide 01)
  • Article: Software Interfaces for Machine Learning Deployment (Deployment Series: Guide 02)
  • Article: Batch Inference for Machine Learning Deployment (Deployment Series: Guide 03)
  • Article: The Challenges of Online Inference (Deployment Series: Guide 04)
  • Article: Online Inference for ML Deployment (Deployment Series: Guide 05)
  • Article: Model Registries for ML Deployment (Deployment Series: Guide 06)
  • Article: Test-Driven Machine Learning Development (Deployment Series: Guide 07)
  • Article: A/B Testing Machine Learning Models (Deployment Series: Guide 08)
  • Article: Lessons Learned from 15 Years of Monitoring Machine Learning in Production
  • Article: Why is it Important to Monitor Machine Learning Models?
  • Article: Maximizing Business Impact with Machine Learning
• Article: Proxy Metrics
• Article: Celery: an overview of the architecture and how it works
• Article: Unit Testing Celery Tasks
• Article: Testing Celery Chains
• Article: Task Routing in Celery
• Article: Dynamic Task Routing in Celery
• Article: Dockerize a Celery app with Django and RabbitMQ
• Article: How to call a Celery task from another app
• Article: Distributed Monte Carlo with Celery chords
• Article: An incredibly simple no-frills Celery setup
• Article: 3 Strategies to Customise Celery logging handlers
• Article: Celery task exceptions and automatic retries
• Article: Concurrency and Parallelism
• Article: Celery, docker and the missing startup banner
• Article: Monitoring a Dockerized Celery Cluster with Flower
• Article: Quick Guide: Custom Celery Task Logger
• Article: Celery on Docker: From the Ground up
• Article: Kubernetes for Python Developers: Part 1
• Article: Auto-reload Celery on code changes
• Book: Building Intelligent Systems: A Guide to Machine Learning Engineering
• Datacamp: Parallel Computing with Dask
• Datacamp: Data Engineering for Everyone
• Datacamp: Cloud Computing for Everyone
• Pluralsight: Docker and Containers: The Big Picture
• Pluralsight: Docker and Kubernetes: The Big Picture
• Pluralsight: AWS Developer: The Big Picture
• Pluralsight: AWS Networking Deep Dive: Virtual Private Cloud (VPC)
• Pluralsight: AWS VPC Operations
• Pluralsight: Building Applications Using Elastic Beanstalk
• Servers for Hackers Series
• The Hacker's Guide to Scaling Python
• Udacity: Intro to DevOps
• Udacity: Configuring Linux Web Servers
• Udacity: Scalable Microservices with Kubernetes
• Udemy: AWS Concepts
• Udemy: Serverless Concepts
• Udemy: AWS Certified Developer - Associate 2018
• Whitepaper: Architecting for the Cloud AWS Best Practices
• Whitepaper: AWS Well-Architected Framework
• Whitepaper: AWS Security Best Practices
• Whitepaper: Blue/Green Deployments on AWS
• Whitepaper: Microservices on AWS
• Whitepaper: Optimizing Enterprise Economics with Serverless Architectures
• Whitepaper: Practicing Continuous Integration and Continuous Delivery on AWS
• Whitepaper: Running Containerized Microservices on AWS
• Whitepaper: Serverless Architectures with AWS Lambda
• Youtube: Deploying a machine learning model to the cloud using AWS Lambda
• Youtube: Shawn Scully: Production and Beyond: Deploying and Managing Machine Learning Models
• Youtube: Human-Centric Machine Learning Infrastructure @Netflix
• Youtube: Instrumentation, Observability & Monitoring of Machine Learning Models
• Youtube: OpML '20 - How ML Breaks: A Decade of Outages for One Large ML Pipeline
• Youtube: Applied ML in Production
  • Objective · Applied ML in Production 0:02:38
  • Solution · Applied ML in Production 0:08:53
  • Evaluation · Applied ML in Production 0:04:02
  • Iteration · Applied ML in Production 0:04:35
  • Annotation · Applied ML in Production 0:14:34
  • Exploratory Data Analysis (EDA) · Applied ML in Production 0:09:02
• MLOps Community Talks
  • Our 1st MLOps Meetup - Luke Marsden 0:56:11
  • MLOps Community Meetup #3: Hierarchy of Machine Learning Needs with Phil Winder 0:58:25
  • MLOps Community March 25 2020 featuring Charles Radclyffe 1:03:42
  • MLOps #4: Shubhi Jain - Building an ML Platform @SurveyMonkey 0:55:42
  • MLOps Meetup #6: Mid-Scale Production Feature Engineering with Dr. Venkata Pingali 1:01:35
  • MLOps meetup #5 High Stakes ML with Flavio CLesio 0:55:27
  • MLOps meetup #7 Alex Spanos // TrueLayer 's MLOps Pipeline 0:56:17
  • #8 Optimizing your ML workflow with kubeflow 1.0 1:03:41
  • #10 MLOps the Elephant and the Blind Men with Saurav Chakravorty 0:55:02
  • #11 Machine Learning at scale in Mercado Libre with Carlos de la Torre 0:59:28
  • MLOps meetup #12 // Why data scientists should know data engineering with Dan Sullivan 0:58:28
  • #13 Maximizing job opportunities as a data scientists with Anthony Kelly 0:58:24
  • MLOps #14: Kubeflow vs MLflow with Byron Allen 0:54:57
  • MLOps #15 - Scaling Human in the Loop Machine Learning with Robert Munro 0:55:04
  • #16 Venture Capital In Machine Learning Startups with John Spindler 1:05:41
  • MLOps #17 // The challenges of ML Operations and how Hermione helps along the way 1:01:30
  • MLOps #18 // Nubank - Running a fintech on ML 0:53:19
  • MLOps #19 // DataOps and Data versioning in ML 1:01:55
  • MLOps #21 Build vs Buy an ML platform // Diego Oppenheimer - CEO Algorithmia 0:57:20
  • MLOps #22 Deep Dive on Paperspace Tooling // Misha Kutsovsky - Senior ML Architect at Paperspace 1:07:15
  • Feature Stores: An essential part of the ML stack to build great data / Kevin Stumpf - CTO at Tecton 1:05:46
  • MLOps #24 Monitoring the ML stack // Lina Weichbrodt 0:55:32
  • MLOps #25 - How to become a better data scientist: the definite guide // Alexey Grigorev 1:00:42
  • MLOps #26 Python and Dask: Scaling the DataFrame // Dan Gerlanc - Founder of Enplus Advisors 1:28:56
  • #27 How to Leverage ML Tooling Ecosystem Mariya Davydova Head of Product at Neu ro 0:55:57
  • MLOps #28 ML Observability // Aparna Dhinakaran - Chief Product Officer at Arize AI 0:55:04
  • MLOps #29 Continuous Evaluation & Model Experimentation // Danny Ma - Founder of Sydney Data Science 1:00:46
  • MLOps Coffee Sessions #6 Continuous Integration for ML // Featuring Elle O'Brien 1:01:46
  • MLOps #30 Scaling ML Capabilities in Large Organizations // Bertjan Broeksema & Axel Goblet 1:02:47
  • MLOps #31 Path to Production and Monetizing Machine Learning // Vin Vashishta - Data Scientist 0:56:35
  • MLOps #32 Creating Beautiful Ambient Music with Google Brain’s Music Transformer // Daniel Jeffries 0:55:52
  • MLOps #33 Building Say Less: An AI-Powered Summarization App // Yoav Zimmerman - Model Zoo 0:53:08
  • MLOps #34 Owned By Statistics: How Kubeflow & MLOps Can Help Secure ML Workloads // David Aronchick 0:56:18
  • MLOps #35: Streaming Machine Learning with Apache Kafka and Tiered Storage // Kai Waehner, Confluent 0:52:50
  • Bring Your On-Prem ML Use Cases to Production on Google Cloud using Kubeflow 0:20:28
  • Moving deep learning from research to prod using DeterminedAI and Kubeflow // David Hershey 0:56:06
  • MLOps Coffee Sessions #11: Analyzing “Continuous Delivery and Automation Pipelines in ML" // Part 3 1:06:28
  • MLOps Coffee Sessions #14 Conversation with the creators of Dask // Hugo Bowne and Matthew Rocklin 0:56:27
  • When You Say Data Scientist Do You Mean Data Engineer? Lessons Learned From Start Up Life 1:00:43
  • Scalable Python for Everyone, Everywhere // Matthew Rocklin // MLOps Meetup #37 0:57:07
  • Operationalize Open Source Models with SAS Open Model Manager // Ivan Nardini // MLOps Meetup #39 0:56:53
  • Data Engineering + ML + Software Engineering // Satish Chandra Gupta // MLOps Coffee Sessions #16 0:57:05
  • Hands-on serving models using KFserving // Theofilos Papapanagiotou // MLOps Meetup #40 0:57:40
  • Luigi in Production // MLOps Coffee Sessions #18 // Luigi Patruno ML in Production 0:47:23
  • Metaflow: Supercharging Our Data Scientist Productivity // Ravi Kiran Chirravuri // MLOps Meetup #41 1:00:30
  • UN Global Platform // Mark Craddock // Co-Founder & CTO, Global Certification // MLOps Meetup #42 0:58:48
  • The Current MLOps Landscape // Nathan Benaich & Timothy Chen // MLOps Meetup #43 0:58:31
  • A Conversation with Seattle Data Guy // Benjamin Rogojan // MLOps Coffee Sessions #21 0:47:11
• Youtube: Josh Wills: Visibility and Monitoring for Machine Learning Models
• Youtube: PyData Vancouver meetup: cortex.dev : Serving machine learning models in production
• Youtube: Why Your Web Server Should Log to Stdout (Especially with Docker)
• Youtube: What is Load Balancing?
• Youtube: What is Consistent Hashing and Where is it used?
• Youtube: What is a Message Queue and Where is it used?
• Youtube: Why do Databases fail? AntiPatterns to avoid!
• Youtube: What is a microservice architecture and it's advantages?
• Youtube: What is the Publisher Subscriber Model?
• Youtube: What's an Event Driven System?
• Youtube: 5 Tips for System Design Interviews
• Youtube: How to avoid a single point of failure in distributed systems
• Youtube: System Design: Tinder as a microservice architecture
• Youtube: System Design Basics: Horizontal vs. Vertical Scaling
• Youtube: What is Database Sharding?
• Youtube: A friendly introduction to System Design
• Youtube: Avoid cascading failures in a distributed system
• Youtube: Designing Instagram: System Design of News Feed
• Youtube: Whatsapp System Design: Chat Messaging Systems for Interviews
• Youtube: Introduction to NoSQL databases
• Youtube: Distributed Consensus and Data Replication strategies on the server
• Youtube: What is an API and how do you design it?
• Youtube: What is Distributed Caching? Explained with Redis!
• Youtube: Service discovery and heartbeats in micro-services
• Youtube: Relational database index vs. NoSQL index
• Youtube: How Netflix onboards new content: Video Processing at scale
• Youtube: How to start with distributed systems? Beginner's guide to scaling systems.
• Youtube: Capacity Estimation: How much data does YouTube store daily?
• Youtube: How databases scale writes: The power of the log
• Youtube: System design : Design Autocomplete or Typeahead Suggestions for Google search

Be able to perform load testing

• Youtube: Loading Testing with Python

Be able to perform A/B testing

• Article: Multi-Armed Bandit (MAB) – A/B Testing Sans Regret
• Article: When to Run Bandit Tests Instead of A/B/n Tests
• Datacamp: Customer Analytics & A/B Testing in Python
• Udacity: A/B Testing
• Udacity: A/B Testing for Business Analysts
• Youtube: Hypothesis testing with Applications in Data Science 0:10:33

Be proficient in Python

• Article: No Really, Python's Pathlib is Great
• Article: A deep dive on Python type hints
• Article: I wish I knew these things when I learned Python
• Article: Python Concurrency: The Tricky Bits
• Article: The Complete Python Development Guide
• Article: Hypermodern Python
• Article: Hypermodern Python Chapter 2: Testing
• Article: Hypermodern Python Chapter 3: Linting
• Article: Hypermodern Python Chapter 4: Typing
• Article: Hypermodern Python Chapter 5: Documentation
• Article: Hypermodern Python Chapter 6: CI/CD
• Article: Speeding Up Python with Concurrency, Parallelism, and asyncio
• Article: Speed Up Your Python Program With Concurrency
• Regex For Noobs (like me!) - An Illustrated Guide
• Book: A Byte of Python
• Book: Learn Python The Hard way
• Book: Python 201
• Book: Python Anti-Patterns
• Book: Real Python
• Book: The Python 3 Standard Library By Example
• Book: Writing Idiomatic Python 3
• Calmcode: logging
• Calmcode: tqdm
• Calmcode: virtualenv
• Calmcode: ray
• Codecademy: Learn Python
• Cognitiveclass.ai: Python for Data Science
• Doc: Python Lifecycle Training
• Datacamp: Python for R Users
• Datacamp: Python for Spreadsheet Users
• Datacamp: Importing Data in Python (Part 1)
• Datacamp: Intermediate Python for Data Science
• Datacamp: Python Data Science Toolbox (Part 1)
• Datacamp: Python Data Science Toolbox (Part 2)
• Datacamp: Intro to Python for Finance
• Datacamp: Writing Efficient Python Code
• Datacamp: Writing Functions in Python
• Datacamp: Working with Dates and Times in Python
• Datacamp: Object-Oriented Programming in Python
• edX: Introduction to Python for Data Science
• edX: Programming with Python for Data Science
• Google's Python Class
• Treehouse: Python Basics
• TheNewBoston: Python Programming Tutorials
• Udacity: Introduction to Python Programming
• Udacity: Programming Foundations with Python
• Youtube: Python 3 Programming Tutorial - Regular Expressions / Regex with re
• Youtube: Python Tutorial: re Module - How to Write and Match Regular Expressions (Regex)
• Youtube: Tutorial: Sebastian Witowski - Modern Python Developer's Toolkit
• Youtube: Python Concurrency and Multithreading
• Youtube: Aaron Richter- Parallel Processing in Python| PyData Global 2020
• Youtube: The Clean Architecture in Python

Be familiar with compiled languages

• Codecademy: Learn Java
• Udacity: C++ For Programmers
• Udacity: Java Programming Basics

Have a general understanding of other parts of the stack

• Book: Refactoring UI
• Codecademy: Learn HTML
• Codecademy: Learn SASS
• Codecademy: Make a website
• Codecademy: Learn ReactJS: Part I
• Codecademy: Learn ReactJS: Part II
• Codecademy: Learn JavaScript
• Codecademy: Jquery Track
• Codecademy: Learn Ruby
• Code School: Fundamentals of Design
• Code School: Blasting Off with Bootstrap
• (ES6) - Beau teaches JavaScript
• Pluralsight: UX Fundamentals
• Pluralsight: HTML, CSS, and JavaScript: The Big Picture
• Pluralsight: CSS Positioning
• Pluralsight: Introduction to CSS
• Pluralsight: CSS: Specificity, the Box Model, and Best Practices
• Pluralsight: CSS: Using Flexbox for Layout
• Pluralsight: Using The Chrome Developer Tools
• Thoughtbot: Design for Developers
• Treehouse: HTML
• Treehouse: Javascript Booleans
• Udacity: ES6 - JavaScript Improved
• Udacity: Intro to Javascript
• Udacity: Object Oriented JS 1
• Udacity: Object Oriented JS 2
• Udemy: Understanding Typescript

Be familiar with fundamental Computer Science concepts

• Article: Asymptotic Analysis Explained with Pokémon: A Deep Dive into Complexity Analysis
• Book: Grokking Algorithms
• Codecademy: Big O
• Crashcourse: Computer Science
• Khan Academy: Data Structures
• Udacity: Intro to Algorithms
• Udacity: Intro to Computer Science
• Udacity: Intro to Theoretical Computer Science
• Udacity: Programming Languages
• Udacity: Networking for Web Developers

Be able to apply proper software engineering process

• Pluralsight: Security Awareness: Basic Concepts and Terminology
• Pluralsight: Secure Software Development
• Pluralsight: Clean Architecture: Patterns, Practices, and Principles
• Thoughtbot: Software Development Process
• Thoughtbot: Refactoring
• Udacity: Design of Computer Programs
• Udacity: Product Design
• Udacity: Rapid Prototyping
• Udacity: Software Development Process

Be able to efficiently use a text editor

• Article: Work remotely with PyCharm, TensorFlow and SSH
• Article: Python remote debugging with PyCharm, CUDA, and Conda
• Article: How To Use Visual Studio Code for Remote Development via the Remote-SSH Plugin
• Article: Docker as Remote Interpreter for PyCharm Professional
• Mastering Pycharm
• Video: Pycharm Tips
• Youtube: Productive pytest with PyCharm
• Youtube: Getting Started with Python in Visual Studio Code
• Youtube: 42 PyCharm Tips and Tricks
• Youtube: Pycharm Quick Tips & Tricks
  • How to use Live Edit to edit HTML&CSS in PyCharm 0:00:49
  • How to let PyCharm install and generate the imports while you write a symbol 0:00:35
  • How to enforce One Import Per Line in PyCharm 0:00:39
  • How to optimize imports in PyCharm 0:01:15
  • How to use the navigation bar to move around your project tree in PyCharm 0:01:09
  • How to use Recent Files to open the tool window in PyCharm 0:00:43
  • How to navigate to symbol using the keyboard in PyCharm 0:00:41
  • How to navigate the cursor position back and forth in PyCharm 0:00:48
  • How to use the 'find action' shortcut in PyCharm 0:01:40
  • How to activate the navigation bar in PyCharm 0:01:36
  • How to navigate to file using the keyboard in PyCharm 0:00:58
  • How to open a file using the keyboard in PyCharm 0:01:03
  • How to use speed search to navigate files in PyCharm 0:01:07
  • How to activate the navigation bar and create a file in PyCharm 0:01:09
  • How to use the find in path dialog in PyCharm 0:00:50
  • How to use drag-n-drop to create a SQLite database in PyCharm 0:00:40
  • How to evaluate expressions during debugging in PyCharm 0:01:13
  • How to add conditions to your breakpoints in PyCharm 0:01:33
  • How to use refactor to rename a file and its references in PyCharm 0:01:06
  • How to quickly view parameter information in PyCharm 0:00:38
  • How to view arguments and documentation without interrupting your flow 0:01:08
  • How to make and extend selections in PyCharm 0:00:39
  • How to reformat your code in PyCharm 0:00:46
  • How to smart-add a new line in PyCharm 0:01:03
  • How to split the screen in PyCharm 0:01:35
  • How to get PyCharm adding fields in a constructor for you 0:00:47
  • How to use refactoring to rename symbols in PyCharm 0:01:13
  • How to avoid disasters by using the local history in PyCharm 0:01:21
  • How to run your project from the keyboard in PyCharm 0:01:21
  • How to disable tabs in PyCharm 0:00:55
  • How to speed up coverage in PyCharm 0:01:34
  • How to enable auto-run for your tests in PyCharm 0:01:45
  • How to run a single test in PyCharm 0:01:08
  • How to use the gutter to quickly spot missing tests in PyCharm 0:01:06
  • How to reword a commit message in PyCharm 0:00:49
  • How to do partial commits in PyCharm 0:01:01
  • How to put a project under version control in PyCharm 0:01:08
  • How to get started with a repository from GitHub in PyCharm 0:01:07
  • How to undo the last commit in PyCharm 0:00:52
  • How to quickly show your npm scripts in PyCharm 0:01:25
  • How to use 'surround with' to insert elements in PyCharm 0:01:11

Be able to communicate and collaborate well

• Google: Technical Writing
• Book: Emotional Intelligence
• Book: How to Win Friends & Influence People
• Book: Influence: The Psychology of Persuasion
• Book: Leaders Eat Last: Why Some Teams Pull Together and Others Don't
• Book: Multipliers: How the Best Leaders Make Everyone Smarter
• Book: Soft Skills: The software developer's life manual
• Book: The New One Minute Manager
• Calmcode: Remote Work
• Youtube: Building a psychologically safe workplace | Amy Edmondson | TEDxHGSE

Be familiar with the hiring pipeline

• Article: What You Need to Know Before Considering a PhD
• Article: Advice to aspiring data scientists: start a blog
• Article: Systems Design Interview Guide
• Article: A Guide to Cold Emailing
• Book: Machine Learning Systems Design
• Datacamp: Preparing for Statistics Interview Questions in Python
• Datacamp: Practicing Machine Learning Interview Questions in Python
• Datacamp: Kaggle Competition
• Udacity: Optimize your GitHub
• Udacity: Strengthen Your LinkedIn Network & Brand
• Udacity: Data Science Interview Prep
• Udacity: Full-Stack Interview Prep
• Udacity: Refresh Your Resume
• Udacity: Craft Your Cover Letter
• Udacity: Technical Interview
• Youtube: Stanford CS230: Deep Learning | Autumn 2018 | Lecture 8 - Career Advice / Reading Research Papers
• Youtube: The Importance of Writing in a Tech Career - Eugene Yan
• Youtube: How to prepare for Machine Learning interviews- Part 1 | Applied AI Course
• Youtube: How to prepare for Machine Learning interviews- Part 2 | Applied AI Course

Broaden Perspective

• Book: Atomic Habits
• Book: Deep Work
• Book: Outliers: The Story of Success
• Book: Platform: The Art and Science of Personal Branding
• Book: Rich Dad Poor Dad
• Book: The Power of Broke
• Book: The 10X Rule
• Book: The Millionaire Fastlane
• Book: The Subtle Art of Not Giving a F**k
• Calmcode: Pomodoro
• Youtube: Why specializing early doesn't always mean career success | David Epstein
• Youtube: Chamath Palihapitiya, Founder and CEO Social Capital, on Money as an Instrument of Change
• Youtube: How to Build a Personal Monopoly with Jack Butcher
• Youtube: How to Use Twitter
• Youtube: A. Jesse Jiryu Davis - Write an Excellent Programming Blog - PyCon 2016
• Youtube: The Great ML Stagnation (Mark Saroufim and Dr. Mathew Salvaris)
• Youtube: What Machine Learning Can Teach Us About Life: 7 Lessons - Talk Python Live Stream