This repository contains slides for the course "20-00-0947: Deep Learning for Natural Language Processing" (Technical University of Darmstadt, Summer term 2023).
This course is jointly lectured by Ivan Habernal and Martin Tutek.
The slides are available as PDF as well as LaTeX source code (we've used Beamer because typesetting mathematics in PowerPoint or similar tools is painful). See the instructions below if you want to compile the slides yourselves.
The content is licensed under Creative Commons CC BY-SA 4.0 which means that you can re-use, adapt, modify, or publish it further, provided you keep the license and give proper credits.
Note: The following content is continuously updated as the summer term progresses. If you're interested in the full previous 2022 content, checkout the latest 2022 Git commit.
Subscribe the YouTube playlist to get updates on new lectures: https://youtube.com/playlist?list=PL6WLGVNe6ZcA4gUr5MaAKdrGxYzYAETK3
April 11, 2023
April 18, 2023
April 25, 2023
May 2, 2023
May 9, 2023
May 16, 2023
May 30, 2023
June 6, 2023
June 13, 2023
June 20, 2023
June 27, 2023
July 4, 2023
July 11, 2023
Thanks to Jan Kühnemund for generating the close caption for YouTube with Open Whisper. We track the subtitles here under subtitles, so if you spot an error there (there are many, such as "tanh" -> "10h"), just open a bug or PR.
- What are some essential pre-requisites?
- Math: Derivatives and partial derivatives. We cover them in Lecture 2. If you need more, I would recommend these sources:
- Jeremy Kun: A Programmer's Introduction to Mathematics. Absolutely amazing book. Pay-what-you-want for the PDF book. https://pimbook.org/
- Deisenroth, A. Aldo Faisal, and Cheng Soon Ong: Mathematics for Machine Learning. Excellent resource, freely available. Might be a bit dense. https://mml-book.github.io/
- Math: Derivatives and partial derivatives. We cover them in Lecture 2. If you need more, I would recommend these sources:
- Can I have the slide deck without "unfolding" the content over multiple pages?
- You can compile the slides with the
handoutparameter, see below the section Compiling handouts.
- You can compile the slides with the
- Where do I find the code for plotting the functions?
- Most of the plots are generated in Python/Jupyter (in Colab). The links are included as comments in the respective LaTeX sources for the slides.
If you run a linux distribution (e.g., Ubuntu 20.04 and newer), all packages are provided as part of texlive. Install the following packages
$ sudo apt-get install texlive-latex-recommended texlive-pictures texlive-latex-extra \
texlive-fonts-extra texlive-bibtex-extra texlive-humanities texlive-science \
texlive-luatex biber wget -y
Install Fira Sans fonts required by the beamer template locally
$ wget https://github.com/mozilla/Fira/archive/refs/tags/4.106.zip -O 4.106.zip \
&& unzip -o 4.106.zip && mkdir -p ~/.fonts/FiraSans && cp Fira-4.106/otf/Fira* \
~/.fonts/FiraSans/ && rm -rf Fira-4.106 && rm 4.106.zip && fc-cache -f -v && mktexlsr
Compile each lecture's slides using lualatex
$ lualatex dl4nlp2023-lecture*.tex && biber dl4nlp2023-lecture*.bcf && \
lualatex dl4nlp2023-lecture*.tex && lualatex dl4nlp2023-lecture*.tex
If you don't run a linux system or don't want to mess up your latex packages, I've tested compiling the slides in a Docker.
Install Docker ( https://docs.docker.com/engine/install/ )
Create a folder to which you clone this repository (for example, $ mkdir -p /tmp/slides)
Run Docker with Ubuntu 20.04 interactively; mount your slides directory under /mnt in this Docker container
$ docker run -it --rm --mount type=bind,source=/tmp/slides,target=/mnt \
ubuntu:20.04 /bin/bash
Once the container is running, update, install packages and fonts as above
# apt-get update && apt-get dist-upgrade -y && apt-get install texlive-latex-recommended \
texlive-pictures texlive-latex-extra texlive-fonts-extra texlive-bibtex-extra \
texlive-humanities texlive-science texlive-luatex biber wget -y
Fonts
# wget https://github.com/mozilla/Fira/archive/refs/tags/4.106.zip -O 4.106.zip \
&& unzip -o 4.106.zip && mkdir -p ~/.fonts/FiraSans && cp Fira-4.106/otf/Fira* \
~/.fonts/FiraSans/ && rm -rf Fira-4.106 && rm 4.106.zip && fc-cache -f -v && mktexlsr
And compile
# cd /mnt/dl4nlp/latex/lecture01
# lualatex dl4nlp2023-lecture*.tex && biber dl4nlp2023-lecture*.bcf && \
lualatex dl4nlp2023-lecture*.tex && lualatex dl4nlp2023-lecture*.tex
which generates the PDF in your local folder (e.g, /tmp/slides).
We're uploading the PDFs as presented in the lecture. You can compile the slides in a concise way using the handout settings. Just commment/uncomment the respective line at the beginning of the tex file of the lecture slides.