Knowledge Base

Softmax Regression

• Generalization of logistic regression for multiple classification (predict cat, dog, lizard, mouse, ...)
• Coursera Video - Softmax

One-hot Vector

• https://towardsdatascience.com/ml-intro-5-one-hot-encoding-cyclic-representations-normalization-6f6e2f4ec001

ResNets - Residual Networks

• Vanishing and exploding gradients are problems with very deep neural nets
• Use what is called a Residual Block to skip connections connection an activation from one layer to another deeper layer
• https://towardsdatascience.com/an-overview-of-resnet-and-its-variants-5281e2f56035
• Medium Understanding ResNet

Natural Language Processing

Word Embedding

• Word Embedding is a way of representing words for NLP
  • Coursera Video - Featurized Representation
  • Introduction

word2vec

• Mikolov Orginal Paper
• Mikolov C Code implementation
• Skip-grams
  • Coursera Video - Skip-grams
• Negative Sampling
  • Coursera Video - Negative Sampling
• GloVe - Global Vectors for Word Representation
  • Coursera Video - GloVe
  • https://nlp.stanford.edu/projects/glove/

Debiasing word embeddings

• Coursera Video - Debiasing word embeddings
• Bias reduction algorithm - Boliukbasi et al., 2016

Applications using Word Embeddings

• Sentiment Classification
  • Coursera Video - Sentiment Classification

Sequence to Sequence Models

• Sutskever Paper - Sequence to Sequence Learning with Neural Networks
• Cho Paper - Learning Phrase Representations using RNN...