Skip to content

colibri v1.2.0

Latest

Choose a tag to compare

@github-actions github-actions released this 28 Jul 20:18
2d62381

Pure-C MoE engine that streams experts from disk β€” run a 744B model on a ~25 GB box. Highlights since v1.1.1:

Hardware & correctness

  • Fix OOM on integrated / unified-memory GPUs β€” GB10 / DGX Spark (#653)
  • Recognize AMD/ROCm in doctor and resource_plan β€” a HIP build is no longer reported CPU-only (#662, #663)
  • Chat stop-set fix (#633 / #381); async packed-int4 parity (#632); stable KV slot per conversation (#634, #639)

Performance

  • AVX2 matmul_e8 β€” fmt=6 (E8/IQ3) was 92% of decode on the scalar kernel (#654)
  • Native SIMD fmt=6 encoder, 15Γ— over numpy, byte-identical (#655)
  • AVX-512 matmul_i3 for fmt=5 int3-g64 (#661)

Web & docs

  • Web dashboard with the live Expert Atlas (#641); new landing page (#645)
  • README / quickstart point to the gs64 model container (#642)

Verify your download against SHA256SUMS.txt.