Skip to content

🚀 FastFlowLM v0.9.20 — Massive Decoding Speed Boosts for GPT-OSS & Gemma3

Choose a tag to compare

View all tags
@FastFlowLM FastFlowLM released this 20 Nov 16:04
· 1062 commits to main since this release
v0.9.20
88615a7
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

FastFlowLM v0.9.20 introduces substantial performance improvements across multiple model families, with special focus on decoding efficiency.

âš¡ Performance Improvements

🔸 1. GPT-OSS Models

  • Decoding speed of gpt-oss:20b and gpt-oss-safeguard:20b are reaching ~19 tokens/sec and are over 60% faster at 1K context length.

🔸 2. Gemma3 Models

  • gemma3:4b reaching ~19 tokens/sec and enjoys over ~20% decoding speed boost at 1K context length.
  • gemma3:1b (reaching ~43 tokens/sec)
  • gemma3:270m (reaching ~79 tokens/sec; Note that this model is experimental)

This release is focused on raw speed—making FastFlowLM even more efficient for both high-capacity and portable deployments.

Assets 3
Loading