🚀 FastFlowLM v0.1.0 — MVP Release: Llama3 on AMD NPU

FastFlowLM v0.1.0 — MVP Release

We’re excited to announce the v0.1.0 MVP release of FastFlowLM, a lightweight and efficient LLM inference engine optimized exclusively for AMD NPUs.

✅ Supported Models

• Llama3.1:8B
• Llama3.2:1B
• Llama3.2:3B

💾 Installation

💡 Platform: Windows only

1. Download the installer and run:
   👉 flm-setup.exe
2. Open PowerShell to run FastFlowLM.

🧠 Usage Modes

FastFlowLM supports two invocation modes:

• Interactive Mode
  Launch a terminal-based chat interface (CLI):

  flm run llama3.2:1B

• Server Mode
  Start a REST API server on port 11434:

  flm serve llama3.2:1B

  In server mode, the model tag (e.g., llama3.2:1B) specifies the initial model to be loaded.
  If a request is made for a different model, FastFlowLM will automatically reload the appropriate model.

⚙️ Notes

• This MVP focuses on functionality and performance for AMD NPU platforms.
• Only one model can be loaded into memory at a time.
• For best performance, it is recommended to set the NPU power mode to performance or turbo:
  AMD XRT SMI Documentation

  xrt-smi configure --pmode performance