Skip to content

Arc Llama 0.9.0

Choose a tag to compare

View all tags
@offbyonebit offbyonebit released this 07 Oct 13:12
· 24 commits to main since this release
v0.9.0
48bc7e9
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Arc Llama 0.9.0 is the stable release of a more complete local inference workflow for Intel Arc GPUs.

Highlights

  • Guided setup and launch: arc-llama run accepts a registered model, local GGUF, or Hugging Face GGUF source, checks the launch plan, and starts the API. Use --setup-only to inspect the plan first.
  • Better model-fit decisions: GGUF metadata informs context and VRAM estimates, and known oversized launch plans are rejected before startup.
  • Local performance tuning: benchmark and tune model recipes for the installed GPU and llama.cpp build; speculative-decoding candidates can be measured and rolled back when they do not meet the configured improvement threshold.
  • More ways to connect: OpenAI-compatible and Ollama-compatible routes, upstream model support, a browser UI, and an optional terminal UI.
  • More reliable model management: safer model ownership checks and switching, request draining, clearer startup diagnostics, chat-history validation, and atomic history writes.
  • Safer Linux model loading: avoid starting a model when host RAM is below the configured safety reserve.

Validation

The release branch passes Linux and Windows CI on Python 3.10, 3.12, and 3.14, including bare-wheel installs and the browser UI regression suite. Physical GPU validation for this release used an Arc Pro B60 on Linux with Vulkan and SYCL. This does not certify every GPU model, driver, or operating-system/backend combination.

Install

python -m pip install --upgrade arc-llama
Assets 4
Loading