Tiny-LLM v2.0.0

Tiny-LLM v2.0.0 — Major Refactoring

Release Date: March 9, 2026

---

English

⚠️ Breaking Changes

KVCache API Redesign: The previous appendKV() implementation had fragile layer-order dependencies that could lead to incorrect cache writes if layers were called in different orders.

Solution: New stateless design with explicit length advancement.

// After (v2.0+)
for (int i = 0; i < num_layers; i++) {
    layers[i]->forward(hidden_states, kv_cache, seq_id, position, stream);
}
// Explicitly advance length once after all layers
kv_cache.advanceSeqLen(seq_id, num_tokens);

🟢 Added

• GitHub Actions workflow for continuous integration
• Automated clang-format checking
• CMake modernization with target exports (tiny_llm::tiny_llm)
• Improved compiler warning flags

🟡 Changed

• Minimum CMake version: 3.18
• CUDA architecture auto-detection with fallback

📊 Performance

Metric	v1.0.0	v2.0.0	Change
Build time	45s	38s	-15%
Test runtime	2.1s	1.8s	-14%

---

简体中文

⚠️ 破坏性变更

KVCache API 重新设计: 之前的 appendKV() 实现存在脆弱的层序依赖，如果层以不同顺序调用可能导致错误的缓存写入。

解决方案: 新的无状态设计，显式推进长度。

// 之后 (v2.0+)
for (int i = 0; i < num_layers; i++) {
    layers[i]->forward(hidden_states, kv_cache, seq_id, position, stream);
}
// 所有层完成后显式推进长度
kv_cache.advanceSeqLen(seq_id, num_tokens);

🟢 新增

• GitHub Actions 持续集成工作流
• 自动 clang-format 检查
• CMake 现代化，支持 target 导出 (tiny_llm::tiny_llm)
• 改进的编译器警告标志

🟡 变更

• 最低 CMake 版本：3.18
• CUDA 架构自动检测，带常见架构回退

---

Installation | 安装

git clone https://github.com/LessUp/tiny-llm.git
cd tiny-llm
git checkout v2.0.0
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

Documentation | 文档 | API Reference | API 参考