Tiny-LLM v2.0.1

Tiny-LLM v2.0.1 — Bug Fixes

Release Date: April 16, 2026

---

English

🔵 Fixed

Critical: Scale Dimension Calculation Error

Severity: Critical
Impact: Test utility only
File: tests/test_integration.cu

The createRandomWeight function had an incorrect scale tensor dimension calculation:

// ❌ INCORRECT (rows and cols swapped)
int num_groups = (cols + group_size - 1) / group_size;
w.scales = randomDeviceFP16(rows * num_groups, ...);

// ✅ CORRECT  
int num_groups = (rows + group_size - 1) / group_size;
w.scales = randomDeviceFP16(num_groups * cols, ...);

Why this matters: W8A16 matmul uses [rows/group_size, cols] to index scales, requiring ceil(rows/g) * cols elements.

Code Cleanup: Removed 12 lines of unused q_reg array loading code in kernels/attention.cu.

✅ Verification

$ ctest --output-on-failure
100% tests passed, 0 tests failed

---

简体中文

🔵 修复

严重: 尺度维度计算错误

严重程度: 严重
影响范围: 仅测试工具
文件: tests/test_integration.cu

createRandomWeight 函数中存在尺度张量维度计算错误：

// ❌ 错误 (rows 和 cols 互换)
int num_groups = (cols + group_size - 1) / group_size;
w.scales = randomDeviceFP16(rows * num_groups, ...);

// ✅ 正确  
int num_groups = (rows + group_size - 1) / group_size;
w.scales = randomDeviceFP16(num_groups * cols, ...);

重要性: W8A16 矩阵乘使用 [rows/group_size, cols] 索引尺度，需要 ceil(rows/g) * cols 个元素。

代码清理: 移除了 kernels/attention.cu 中 12 行未使用的 q_reg 数组加载代码。

✅ 验证

$ ctest --output-on-failure
100% tests passed, 0 tests failed

---

Installation | 安装

git clone https://github.com/LessUp/tiny-llm.git
cd tiny-llm
git checkout v2.0.1
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

Documentation | 文档 | API Reference | API 参考