Course 5 - Sequence Models

Info: This course will teach you how to build models for natural language, audio, and other sequence data. Thanks to deep learning, sequence algorithms are working far better than just two years ago, and this is enabling numerous exciting applications in speech recognition, music synthesis, chatbots, machine translation, natural language understanding, and many others.

You will:

• Understand how to build and train Recurrent Neural Networks (RNNs), and commonly-used variants such as GRUs and LSTMs.
• Be able to apply sequence models to natural language problems, including text synthesis.
• Be able to apply sequence models to audio applications, including speech recognition and music synthesis.

This is the fifth and final course of the Deep Learning Specialization.

deeplearning.ai is also partnering with the NVIDIA Deep Learning Institute (DLI) in Course 5, Sequence Models, to provide a programming assignment on Machine Translation with deep learning. You will have the opportunity to build a deep learning project with cutting-edge, industry-relevant content.

This is the fifth course of the Deep Learning Specialization.

Week 1 - Recurrent Neural Networks

Learn about recurrent neural networks. This type of model has been proven to perform extremely well on temporal data. It has several variants including LSTMs, GRUs and Bidirectional RNNs, which you are going to learn about in this section.

• Video: Why sequence models

• Video: Notation

• Video: Recurrent Neural Network Model

• Video: Backpropagation through time

• Video: Different types of RNNs

• Video: Language model and sequence generation

• Video: Sampling novel sequences

• Video: Vanishing gradients with RNNs

• Video: Gated Recurrent Unit (GRU)

• Video: Long Short Term Memory (LSTM)

• Video: Bidirectional RNN

• Video: Deep RNNs

• Read: Building a recurrent neural network - step by step

• Read: Dinosaur Island - Character-Level Language Modeling

• Read: Jazz improvisation with LSTM

• Grading: Recurrent Neural Networks

• Grading: Building a recurrent neural network - step by step

• Grading: Dinosaur Island - Character-Level Language Modeling

• Grading: Jazz improvisation with LSTM

Week 2 - Natural Language Processing & Word Embeddings

Natural language processing with deep learning is an important combination. Using word vector representations and embedding layers you can train recurrent neural networks with outstanding performances in a wide variety of industries. Examples of applications are sentiment analysis, named entity recognition and machine translation.

• Video: Word Representation

• Video: Using word embeddings

• Video: Properties of word embeddings

• Video: Embedding matrix

• Video: Learning word embeddings

• Video: Word2Vec

• Video: Negative Sampling

• Video: GloVe word vectors

• Video: Sentiment Classification

• Video: Debiasing word embeddings

• Read: Operations on word vectors - Debiasing

• Read: Emojify

• Grading: Natural Language Processing & Word Embeddings

• Grading: Operations on word vectors - Debiasing

• Grading: Emojify

Week 3 - Sequence models & Attention mechanism

Sequence models can be augmented using an attention mechanism. This algorithm will help your model understand where it should focus its attention given a sequence of inputs. This week, you will also learn about speech recognition and how to deal with audio data.

• Video: Basic Models

• Video: Picking the most likely sentence

• Video: Beam Search

• Video: Refinements to Beam Search

• Video: Error analysis in beam search

• Video: Bleu Score (optional)

• Video: Attention Model Intuition

• Video: Attention Model

• Video: Speech recognition

• Video: Trigger Word Detection

• Video: Conclusion and thank you

• Read: Neural Machine Translation with Attention

• Read: Trigger word detection

• Grading: Sequence models & Attention mechanism

• Grading: Neural Machine Translation with Attention

• Grading: Trigger word detection