Introduction

CS224N 스터디의 학습내용 정리와 issue공유를 위한 repo입니다.

• Stanford cs224n: Natural Language Processing with Deep Learning (Winter 2019)

  • youtube
  • syllabus, Winter 2019 // youtube version (기록 목차)
  • syllabus, Winter 2020

• 📚 References:

  • Dan Jurafsky and James H. Martin. Speech and Language Processing (3rd ed. draft)
  • Jacob Eisenstein. Natural Language Processing
  • Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing
  • Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning
  • PRML 한국어 번역/정리: http://norman3.github.io/prml/

Participant (alphabetical order)

이름	repo
유인혁	https://github.com/InhyeokYoo
장건희	https://github.com/ckrdkg
최슬기	github

기록 (20.10.13 - )

💡 Importatn Note:

• readme에 대한 관리는 제가 기본적으로 하겠지만, 각자의 작업물을 push하고 관리하는 건 각자 부탁드립니다.

• 매주 스터디에서 나온 질문/궁금증 등은 issue로 남기고 label을 달아주세요.

• 업데이트 하기 전에 fetch/pull 부탁드립니다.

• 양식의 통일 부탁드립니다 (markdown 추천)

  • format 추후 협의
  • naming convention
    • 강의 이름으로 된 파일을 올릴 것
    • 파일 이름 공백은 -로 대체하여 올릴 것
    • E.g. Introduction-and-Word-Vectors.md

• 수식 남기는 방법:

  • <img src="https://render.githubusercontent.com/render/math?math={w_1, ..., w_m}">에서 math=이후에 작성.
  • 결과:

• Note이쁘게 만드는 방법:

  | :exclamation:  This is very important   |
  |-----------------------------------------|

  • 결과

    ❗ This is very important

📑 강의 목차:

Lecture	2019	2020	일치여부	발표자	발표날짜	링크
1	Introduction and Word Vectors/Gensim word vectors example		O	유인혁	2020.10.13	Link
2	Word Vectors 2 and Word Senses		O	유인혁	2020.10.13	Link
3	Word Window Classification, Neural Networks, and Matrix Calculus	Word Window Classification, Neural Networks, and PyTorch	X	최슬기	2020.10.21	Link
4	Backpropagation and Computation Graphs	Matrix Calculus and Backpropagation	X	장건희	2020.10.21
5	Linguistic Structure: Dependency Parsing		O	유인혁	2020.11.01	Link
6	The probability of a sentence? Recurrent Neural Networks and Language Models		O	최슬기	2020.11.01	Link
7	Vanishing Gradients and Fancy RNNs		O	유인혁	2020.11.04	Link
8	Machine Translation, Seq2Seq and Attention		O	장건희	2020.11.04	Link
9	Practical Tips for Final Projects		O	장건희	2020.11.11	Link
10	Question Answering and the Default Final Project	Question Answering, the Default Final Project, and an introduction to Transformer architectures	X	최슬기	2020.11.11	Link
11	ConvNets for NLP		O	최슬기	2020.11.18	Link
12	Information from parts of words: Subword Models		O	유인혁	2020.11.18	Link
13	Modeling contexts of use: Contextual Representations and Pretraining	Contextual Word Representations: BERT (guest lecture by Jacob Devlin)	X	유인혁	2020.11.25	Link
14	Transformers and Self-Attention For Generative Models	Modeling contexts of use: Contextual Representations and Pretraining. ELMo and BERT	X	장건희	2020.11.25	Link
15	Natural Language Generation		O	장건희	2020.12.02	Link
16	Reference in Language and Coreference Resolution		O	최슬기	2020.12.02	Link
17	Multitask Learning: A general model for NLP? (guest lecture by Richard Socher)	Fairness and Inclusion in AI (guest lecture by Vinodkumar Prabhakaran)	X	최슬기	2020.12.09	Link
18	Constituency Parsing and Tree Recursive Neural Networks		O	유인혁	2020.12.09	Link
19	Safety, Bias, and Fairness (guest lecture by Margaret Mitchell)	Recent Advances in Low Resource Machine Translation (guest lecture by Marc'Aurelio Ranzato)	X	유인혁	2020.12.16	Link
20	Future of NLP + Deep Learning	Analysis and Interpretability of Neural NLP	X	장건희	2020.12.16	Link

다음 회차 스터디 목록

1. Paper
   1. NLP paper (BERT 이후의)
   2. paper (DL 전반적으로): distillation, annealing
2. 강의 (기초)
   1. ML: CS229 (PRML), 문일철 (KAIST)
   2. DL: Deep Learning Book, CS231, 수업이 있나?
3. 자연어 실전 (한국어 셋) -> 형태소 분석도 못함 -> 기초선행(2)??
   1. Competition (WMT, SQuAD)
   2. Deploy (Open source)