🚀 FastFlowLM v0.9.29: Major Prefill Speedup

⚠️ A bug affecting the gpt-oss:20b model was fixed in v0.9.30. Please use v0.9.30 or later.

---

📦 What’s New

⚡ Massive Prefill Speed Upgrade

We introduced a new attention engine that dramatically accelerates prefill, with larger gains at longer context lengths (especially 16K+).

• Up to 2.6× faster prefill
• More speedup at longer prompts
• No model re-download required (seamless upgrade)

📈 Prefill Speed with 32K prompt (tok/s)

Model	Before → After	Speedup
lfm2:1.2b	1059 → 1916	1.8×
lfm2:2.6b	654 → 1053	1.6×
lfm2-transcript:2.6b	654 → 1053	1.6×
lfm2.5-it:1.2b	1059 → 1916	1.8×
lfm2.5-tk:1.2b	1059 → 1916	1.8×
llama3.2:1b	577 → 1157	2.0×
llama3.2:3b	214 → 500	2.3×
llama3.1:8b	167 → 281	1.7×
deepseek-r1:8b	167 → 281	1.7×
Phi4-mini-it:4b	173 → 447	2.6×

🔜 Prefill speed upgrades for other models are on the way too — stay tuned!

📊 Detailed Benchmarks

• 🔗 LFM2 Results
• 🔗 LLaMA3 Results
• 🔗 Phi-4 Results

🩹 “No Pain” Update 😎

Just update and run — long-context prefill is now much faster. 🚀

🛠️ Tool Call Bug Fix (Non‑Stream Case)

We’ve squashed a pesky bug affecting tool calls when not using streaming.
Now it behaves exactly like you thought it should. 😌

---

Thanks for being awesome!
Happy modeling 🤖💙

— The FastFlowLM Team