| Topics | Description |
|---|---|
| Prompt Engineering | The president of NVIDIA, Jensen Huang said during an interview with the UAE's Minister of AI, Omar al-Olama at the World Government Summit in Dubai. Everybody in the world is now a programmer. This is the miracle of AI. For the very first time, the technology divide has been completely closed. To engage with AI is a lot easier now than at any time in the history of computing. |
| LLM Parameters and Memory Estimation | Understanding the memory requirements of models is vital for optimizing deployment and usage. It impacts the choice of hardware and the overall cost. Discover a practical estimate for the VRAM needs of a system, based on a general understanding, along with viable solutions available on the internet. By framing the information in terms of scaling laws, it becomes clear how each factor contributes to the overall performance and what trade-offs must be considered when scaling up LLMs. |
| LLM Deployment - llama.cpp | The main goal of llama.cpp is to enable LLM inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud. |
| Common LLM-Related Files | Understanding these files is critical for anyone working with LLMs, especially if you're doing local inference, fine-tuning, deployment, or managing LLMOps pipelines. |
| LLM Training and Inference | The primary objective of this paper is to provide a comprehensive overview of LLMs training and inference techniques to equip researchers with the knowledge required for developing, deploying, and applying LLMs. |
| LLMOps Developer Guide | As teams deploy large language models to production, the same challenges around performance and task measurement still exist. Hence, LLMOps is essential to scale large language models and deploy them to production effectively. |
| LLM Productionization | Understand LLMOps, architectural patterns, how to evaluate, fine tune & deploy HuggingFace generative AI models locally or on cloud. |
| Synthetic Data for Finetuning | Synthetic data is generally defined as artificially annotated information generated by computer algorithms or simulations. |
| LLM Multi-GPU Training | Explained by Hsiu-Hsuan Wang, a graduate student in the Department of Electrical Engineering and Computer Science at National Taiwan University, published on Professor Hung-yi Lee's Youtube channel. |
| MCP - Model Context Protocol | "LLMs by themselves are incapable of doing anything meaningful... The only thing an LLM in its current state is good at is predicting the next text." - Ras Mic "Think of every tool that I have to connect to make my LLM valuable as a different language... MCP, you can consider it to be a layer between your LLM and the services and the tools." - Ras Mic |
這個庫整體的取材邏輯清晰: 從硬體限制到軟體開發, 從理論基礎到生產部署, 構成一條完整的 LLM 工程師學習路徑. 每篇都有其不可替代的位置.
定位: 入口篇, 但不淺薄. 從 10 級技法到情緒提示的學術論文(arxiv), 有層次地涵蓋了從入門到進階. 特別是 Emotional Prompting 那段, 引用了 Big-Bench 115% 提升的實驗數據, 這讓筆記有實證依據, 不只是技巧清單. 許多人以為 Prompt Engineering 只是"問法的技巧", 這篇用數據和框架說明它是有理論基礎的工程行為.
定位: 最務實的一篇, 幾乎是必備速查表. 從推論到訓練的 VRAM 估算, 量化方式對照, GPU 型號清單, NVLink 限制(RTX 4090 不支援 NVLink 這點很多人踩坑), 一直到兩張 4090 能不能合算 48GB 的常見誤解, 都逐一澄清. 這篇資訊密度最高, 對任何想在本地跑模型的人來說等同避坑指南.
定位: 本地部署的核心實踐篇. llama.cpp 是目前本地推論的實際標準工具, 這篇的存在將"理論上可以本地跑" 落地成"實際上怎麼跑". 與上一篇 Memory Estimation 形成直接的承接關係, 先知道需要多少 VRAM, 再知道用什麼工具跑.
定位: 最容易被忽視, 卻極其必要的一篇.
初學者遇到 .gguf, .safetensors, adapter_config.json 時往往不知道是什麼. 這篇把 LLM 生態中所有重要檔案格式一次說清楚, 包括 LoRA adapter, PEFT config, Prompt Template 等, 這些是動手做微調或部署時必然遇到的, 但幾乎沒有人系統整理過. 這篇的稀缺性最高.
定位: 理論基石篇, 來源最學術. 直接引用 arxiv 論文, 完整涵蓋 Transformer 架構細節, Pre-training 任務設計, 各種架構差異(Encoder-only, Decoder-only, Prefix Decoder). 這是其他筆記的理論依據, 讀懂了這篇, 再看量化, LoRA, Multi-GPU, 才會知道"在動什麼".
定位: 從工程師視角看生產化的核心. 這篇不談部署細節, 而是談"如何在生產環境中管理 LLM 應用的全生命週期", 包含 Prompt 管理, Agent 架構, 可觀測性(Observability). 那句"an agent does not come as a one-size-fits-all solution" 是很誠實的業界判斷, 防止讀者對 Agent 過度神化.
定位: 覆蓋面最廣的整合篇. 目錄本身就是一張完整的 LLM 工程 roadmap: 從 HuggingFace 開始, 覆蓋本地部署, AWS SageMaker, 容器化, CI/CD, 邊緣設備, 甚至 Flash Attention 的原理. 與 LLMOps Guide 的差別在於: 這篇更偏技術選型清單, 那篇更偏操作哲學. 兩者互補, 不重複.
定位: 時代性最強的一篇. 這篇切入點很獨特, 它從法律與倫理壓力(Google/ByteDance 的真實案例) 出發解釋為什麼合成資料是必要選項, 而不只是"一種方法". 再加上 Alpaca, WizardLM, Orca 等里程碑模型的演進脈絡, 讓讀者理解合成資料從實驗性做法到成為 Meta Llama 3.1 主要策略的完整歷程. 這篇有歷史縱深, 不只是技術介紹.
定位: 最接地氣的硬體訓練篇. 來源是台大李宏毅頻道的研究生講解, 用廚房比喻說明 QLoRA 和 FSDP, 並且誠實地標注了圖表中的計算錯誤("diagram incorrectly lists 64GB"). 這種對原始資料的批判性態度是少見的, 說明筆記者在學習時是思考過的, 而不是純粹抄錄.
定位: 最具前瞻性的一篇. MCP 出現在這個庫裡, 代表筆記持續跟上 2024-2025 年的技術演進. 從 API 基礎概念出發, 到 LLM+Tools → LLM+MCP 的演化路徑, 論述邏輯清晰. 這篇的存在讓整個庫從"如何跑模型" 延伸到"如何讓模型連接世界", 補完了最後一塊拼圖.
總結: 這 10 篇沒有重複, 每篇都在覆蓋不同的工程層次: 理論基礎, 硬體現實, 本地部署, 生產化操作, 資料策略, 工具整合. 刪去任何一篇, 整條學習路徑都會出現缺口.
The LLM course is divided into three parts.
- LLM Fundamentals covers essential knowledge about mathematics, Python, and neural networks.
- The LLM Scientist focuses on building the best possible LLMs using the latest techniques.
- The LLM Engineer focuses on creating LLM-based applications and deploying them.
This project is licensed under the MIT License - see the LICENSE file for details.
Made with care by Allen Sun