Skip to content

🚀 FastFlowLM v0.9.14: New ASR Capabilities & API Integration

Choose a tag to compare

View all tags
@FastFlowLM FastFlowLM released this 17 Oct 20:53
· 1019 commits to main since this release
v0.9.14
fbefef7
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

✨ What’s New

🎙️ 1. New Model: whisper-large-v3-turbo (by OpenAI)

The first Automatic Speech Recognition (ASR) model on FLM:

  • Runs fully offline on AMD Ryzen™ AI NPU
  • Multilingual audio recognition
  • Supports MP3, WAV, OGG and M4A formats
  • Lightweight footprint — only 900MB memory

Try it out:

  1. Start flm in CLI mode with ASR enabled:
# CLI
flm run gemma3:4b --asr 1 # Load the ASR model (whisper-v3:turbo) in the background, with concurrent LLM loading (gemma3:4b).
  1. Type (replace filename.mp3 with your audio file path):
/input "path\to\audio_sample.mp3" summarize it

🌐 2. OpenAI-Compatible ASR API: v1/audio/transcriptions API

FastFlowLM now supports the OpenAI v1/audio/transcriptions endpoint — making it easy to integrate ASR into any OpenAI-compatible client or UI.

How to use:

  1. Start your FLM server with ASR enabled:
# serve
flm serve gemma3:4b --asr 1 # # Load the ASR model (whisper-v3:turbo) in the background, with concurrent LLM loading (gemma3:4b).
  1. Send audio to:
    POST /v1/audio/transcriptions
    via any OpenAI client or Open WebUI.

Examples: OpenAI client

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:52625/v1",  # FastFlowLM's local API endpoint
    api_key="flm",  # Dummy key (FastFlowLM doesn’t require authentication)
)

with open("audio.mp3", "rb") as f:
    resp = client.audio.transcriptions.create(
        model="whisper-v3",
        file=f,
    )
    print(resp.text)

Example: Open WebUI
1. Follow Open WebUI setup guide.
2. In the bottom-left corner, click User icon, then select Settings.
3. In the bottom panel, open Admin Settings.
4. In the left sidebar, navigate to Audio.
5. Set Speech-to-Text Engine to OpenAI.
6. Enter:
-- API Base URL: http://host.docker.internal:52625/v1
-- API KEY: flm (any value works)
-- STT Model: whisper-large-v3-turbo (type in the model name; can be different)
7. Save the setting.
8. You're ready to upload audio files! (Choose an LLM to load and use concurrently)

🌟 Summary

FastFlowLM v0.9.14 introduces offline ASR with whisper-large-v3-turbo and support for OpenAI’s v1/audio/transcriptions API, making speech-to-text integration seamless across clients and WebUI. Just start the server with --asr 1 and you're ready to transcribe.

Assets 3
Loading