ChatGPT, Language Models, Stochastic Parrots, and The Infinite Monkey Theorem
Jupyter notebooks extracted from Colab. Presented for year 7 to 12 school girls in June 2023 as a part of GoGirlGoForIT event.
Abstract: I will explain the general principles that are used to create and train models such as ChatGPT (called "language models"). Then I will illustrate how they are used, discuss their limitations, and the reasons why they are called "stochastic parrots". At the end, we will develop and train our own "toy" language model together.
TL;DR The tutorial starts with character-level n-gram LMs and goes to word2vec and GPT2 covering some parts of distributional semantics (generating "Harry Potter" from scratch).