Skip to content

ExLlamaSharp 1.0.0

Choose a tag to compare

View all tags
@Kortexio Kortexio released this 17 Aug 14:09
· 9 commits to main since this release
v1.0.0
46b4f32

First public Windows release of ExLlamaSharp — a local LLM server with an OpenAI-compatible API, Blazor admin UI, and EXL3 inference on NVIDIA GPUs.

Install

Download ExLlamaSharp-Setup-win-x64.exe, run it (UAC), then open http://127.0.0.1:14563

  • Admin UI: admin / changeme (change this before production)
  • API key: sk-exllamasharp-dev

One-liner:

irm https://raw.githubusercontent.com/Kortexio/ExLlamaSharp/main/packaging/install-web.ps1 | iex

Included

  • Windows service + system tray
  • Chat playground with streaming completions
  • Hugging Face EXL3 search and download (Jobs progress)
  • API keys, usage audit, setup wizard, diagnostics
  • ZIP fallback for scripted installs

See the README for the full feature list and what is still stub/partial.

Assets 4
Loading