Week 9 2020 Fall

Homework:

• Build an interactive sketch with Text data, using word2Vec, sentiment, CharRNN, GPT-2 model
• Or Train your own GPT-2 model, document your training process, how did you collect the data, what is the result.
• Or Train your own CharRNN model, document your training process, how did you collect the data, what is the result.

Where to find data

• Free eBooks - Project Gutenberg
• All the news - Kaggle
• Wikipedia API, example query: https://en.wikipedia.org/w/api.php?format=json&action=query&prop=extracts&exintro&explaintext&redirects=1&titles=Stack%20Overflow
• Enron (600,000+ employee emails made public during fraud investigation)

Code Examples

• Sentiment
• Word2Vec
• CharRNN_Interactive
• CharRNN_Text
• CharRNN_Text_Stateful
• GPT-2 in p5 web editor, need to run Runway GPT-2 model at localhost:8000
• ml5.js word2vec, doc

Resources

• Training text generator with Runway
• Better Language Models and Their Implications: GPT-2 Blog
• Talk to Transformer: GPT-2 Demo
• Hugging face transformers
• CharRNN from ml5js
• Train CharRNN ml5js model

Questions?

• add your question here

Add your name / link to your assignment below

• Ji Park, BTS Lyric Generator
• KJ Ha, Conversation Generator
• Philip Cadoux, iMessage Generator
• Julie Lizardo, Toxic Tweets
• Jeeyoon Hyun, Dungeon Crawl Morgue Generator
• Zoe Gao, ex texter