Skip to content

🦃 FastFlowLM v0.9.21 — Thanksgiving LLaMA Turbocharge Release

Choose a tag to compare

View all tags
@FastFlowLM FastFlowLM released this 24 Nov 13:05
· 1096 commits to main since this release
v0.9.21
fe1aa00

Happy Thanksgiving!

Today we’re dropping one of our biggest speed upgrades ever for LLaMA and DeepSeek models (our first batch of models) — just in time for the holiday break. Fire up your Ryzen™ AI NPU and enjoy some seriously boosted performance. 🔥

🔄 1. Quantization Upgrade

  • All models migrated from AWQ to Q4_1
  • Better LLM accuracy and quality.

⚡ 2. Massive Decoding Speedup

  • llama3.2:1b: ~50% faster decoding, reaching 66 tps
  • llama3.2:3b: ~40% faster decoding, reaching 28 tps
  • llama3.1:8b: ~40% faster decoding, reaching 13 tps
  • deepseek-r1:8b: ~40% faster decoding, reaching 13 tps

🚀 3. Prefill Phase Optimized

  • Slight improvements to prefill speed of all above, especially impactful for large context initializations.

🎙️ 4. Standalone Whisper ASR Server

You can now serve Whisper (OpenAI’s ASR model) as a standalone model for speech transcription — or pair it with GPU LLMs in a hybrid pipeline.

Use either:

flm serve -a 1

or

flm serve --asr 1

This release wraps up a bundle of performance gifts for LLaMA models on FastFlowLM.

Thank you for being part of the FastFlowLM journey — and happy Thanksgiving! 🦃🔥

Assets 3
Loading