Curation of resources for LLM research.
| 🐱 GitHub | 🐦 X(Twitter) | 📝 Notion |
📢 If you have any suggestions, please don't hesitate to
- comment in the Notion page,
- comment under the X(Twitter) thread,
- post an issue in the GitHub repository,
- or E-mail Yuxuan Tong.
📥 If you want to subscribe to updates of this list, you can
- watch the GitHub repository / check the commit messages,
- or follow the X(Twitter) account / check the thread.
📊 There is also an interactable (i.e. sort / filter / search) version of the following table.
| Link | Abstract | Description | Language | Modality | Update Cycle | Type |
|---|---|---|---|---|---|---|
| 国立台湾大学: 李宏毅机器学习 - CS自学指南 | Basic theory and fundamental works of Deep Learning | Lectures from different years have different focuses, e.g. 2023 focuses on LLM. | EN(Text) ZH(Speech) | Speech Text Code | Year | Basic |
| Introduction - Hugging Face NLP Course | Basic NLP practice (based on HuggingFace ecosystem) | HuggingFace is so accessible that its success is a given (but this also comes with some hidden price for developers). | EN ZH … | Text Code | Dynamic | Basic |
| Yao Fu’s Blog | Fundamental research topics walkthrough | Such as emergent abilities, reasoning, long-context modeling. | EN | Text | Months | Fundamental |
| Transformer Math 101 | EleutherAI Blog | Transformer-related math estimation - Basic | Basic arithmetic about Transformer-based models. | EN | Text | None | Basic |
| 分析transformer模型的参数量、计算量、中间激活、KV cache - 知乎 | Transformer-related math estimation - Mediate | Detailed analysis of calculations in Transformer-based model. | ZH | Text | None | Basic |
| 紫气东来 - 知乎 | Specific engineering details | Such as inference and training frameworks. | ZH | Text | Weeks | Practical |
| WeChat official account 吃果冻不吐果冻皮 | Engineering detail summaries | Summarizing AI engineering techniques, such as inference, parallel computing, etc. | ZH | Text | Days | Practical |
| WeChat official account: 大猿搬砖简记 | Illustrated source code (e.g. vLLM, CUDA) and algorithms (e.g. FlashAttention) | ZH | Text | Weeks | Practical | |
| 游凯超 - 知乎 | Infrastructure-level engineering details | Such as CUDA, NCCL, torch.compile and other side infrastructures like Docker, etc. |
ZH | Text | Days | Practical |
| Alignment Guidebook - Notion | Introduction to LLM Alignment (SFT + RL) | EN | Text | Dynamic | Basic | |
| Spinning Up in Deep RL! — Spinning Up documentation | Basic Deep RL | EN | Text Code |
None | Basic | |
| 科学空间|Scientific Spaces | Blogs combining graceful theories and solid experiments | Blogs by Jianlin Su (苏剑林), the author of RoPE (de facto standard of positional encoding now), versed in math and ML theory while not unfamiliar with experiments and practice. | ZH | Text | Weeks | Fundamental |
| Research | OpenAI research blogs | “We keep re-discovering what OpenAI discovered five years ago.” | EN | Text | Months | Fundamental |
| Research \ Anthropic | Anthropic research blogs | EN | Text | Months | Fundamental | |
| Transformer Circuits Thread | Amazingly insightful and open Anthropic interpretability team research blogs | EN | Text | Month | Fundamental | |
| E.g. [2312.11805] Gemini: A Family of Highly Capable Multimodal Models | LLM technical reports | Such technical reports, while usually not very detailed, often do reveal some important details of SotA LLMs. | EN | Text | Months | Fundamental |
| Hazy Research | Blogs of pioneer visions | Blogs from Hazy Research led by Christopher Ré @ Stanford (one of the best NLP&AI research groups around the world). | EN | Text | Months | Fundamental |
| FAI-Seminar | High-quality talks (largely contributed by Yao class alumna) | ZH | Speech Text | Week | Trending | |
| Cool Papers - Immersive Paper Discovery | Daily arXiv paper & Kimi interaction | EN | Text | Day | Trending | |
| Daily Papers - Hugging Face | The most popular paper selection on Twitter. | EN | Text | Day | Trending | |
| WeChat official account SparksofAGI | Individual paper selection, some of which common popular paper collections might not notice | Selected by Jianbo Dai (戴建波)* (senior researcher at Huawei). | ZH | Text | Weeks | Trending |
| WeChat official account AINLP | Curations of other AI WeChat official accounts | ZH | Text | Day | Trending | |
| ”Big Four” in Chinese AI media: 机器之心、新智元、量子位、夕小瑶科技说 | Popular paper selection | ZH | Text | Day | Trending | |
| WeChat official account arXiv 每日学术速递 | arXiv paper from broader domains | ZH | Text | Day | Auxiliary | |
| WeChat official account: AI 前线 | Various AI news (not limited to research) | ZH | Text | Day | Auxiliary | |
| Video channel Zhao Song (YouTube / BiliBili) | Various practical academic-relevant affairs (e.g. paper submission, job choices) | A little “abstract” though … | ZH | Speech Text | Weeks | Auxiliary |