Week 9 2021 Fall

Homework:

• Build an interactive sketch with Text data, using word2Vec, universal sentence encoder, sentiment, CharRNN, GPT-2 model
• Or train your own GPT-2 model, document your training process, how did you collect the data, what is the result.

Where to find data

• Free eBooks - Project Gutenberg
• All the news - Kaggle
• Wikipedia API, example query: https://en.wikipedia.org/w/api.php?format=json&action=query&prop=extracts&exintro&explaintext&redirects=1&titles=Stack%20Overflow
• Enron (600,000+ employee emails made public during fraud investigation)
• Scrapping Reddit data

Code Examples

• Sentiment
• Word2Vec
• CharRNN_Interactive
• CharRNN_Text
• CharRNN_Text_Stateful
• GPT-2 in p5 web editor, need to run Runway GPT-2 model at localhost:8000
• ml5.js word2vec, doc

Resources

• Training text generator with Runway
• Better Language Models and Their Implications: GPT-2 Blog
• Talk to Transformer: GPT-2 Demo
• Hugging face transformers
• CharRNN from ml5js
• Train CharRNN ml5js model

Questions?

• add your question here

Add your name / link to your assignment below

• Jenny https://jennywangyw.wixsite.com/nyuitp/post/dating-advice-text-generator
• Wendy https://brave-dinghy-437.notion.site/Machine-Poetry-8e897c6896b446baa9046d431aecd0b0
• Nart Ad Slogan poem model