CS561 Deep Learning with NLP San Diego State University

December 2023

This repository is my complete implementation for assignments and projects in CS561: Deep Learning with Natural Language Processing.

Assignments

• Assignment 1: code | handout Exploring word vectors
• Assignment 2: code | handout - Training word2vec
• Assignment 3: code | handout - Neural Dependency Parser

Note, all instructions for CS561 SDSU are detailed in corresponding handout PDF's.

These assignments are heavily based on the Stanford CS224u course.

Final Project

Multitask BERT:

• handout
• starter code

---

Misc. Links:

Guides:

• Python Tutorial
• Introduction to Jupyter Notebooks
• Getting Started with NumPy
• Getting Started with Pytorch

Scientific Papers:

• Distributed Representations of Words and Phrases(Negative Sampling)
• Attention Is All You Need
• Layer Normalization
• Image Transformer
• Music Transformer: Generating music with long-term structure
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Contextual Word Representations: A Contextual Introduction

Readings:

• Understanding Backpropagation by Andrej Karpathy
• The Illustrated BERT, ELMo, and co.
• Martin & Jurafsky Chapter on Transfer Learning
• Transformer (Google AI blog post)
• The Illustrated Transformer