Skip to content

v0.1.0-rc.2 - rename + multi-turn chat + npu-convert + winget

Pre-release
Pre-release

Choose a tag to compare

View all tags
@bpbonker bpbonker released this 30 Apr 20:45
· 19 commits to main since this release
v0.1.0-rc.2
4306557

npurun v0.1.0-rc.2

Heads up — the project was renamed from hexrun to npurun.
The name "hexrun" collided with several existing GitHub projects, so
the repo moved from bpbonker/hexrun to bpbonker/npurun. The
binary, the env vars (NPURUN_MODELS_DIR), and the manifest filename
(npurun.json) all use the new name. There are no other behavioural
changes from the rename itself.

NPU-first local LLM runtime for Snapdragon X Elite (Windows on ARM).
Ollama, llama.cpp, LM Studio, and text-generation-webui all currently
run CPU-only on these laptops — the 45 TOPS Hexagon NPU sits idle.
npurun fixes that. Native Rust on top of Qualcomm's Genie SDK, an
Ollama-class CLI, and an OpenAI/Ollama-compatible HTTP server.
Verified on hardware.

What's new since rc.1

  • Multi-turn chat works. The HTTP server's /v1/chat/completions
    and /api/chat now thread the full messages array through the
    engine instead of only the latest user message. The engine sends
    the full transcript to Genie with SentenceCode::Complete on the
    first call (priming the KV cache) and SentenceCode::Rewind on
    every subsequent call. Genie matches the prefix and only
    re-prefills the new tokens. Verified on Phi 3.5 Mini end-to-end:
    turn-2 questions about facts established in turn 1 are answered
    correctly across both blocking + streaming + OpenAI + Ollama paths.
  • hex-convert → npu-convert Python sidecar (Phase 5 starter).
    Three subcommands: manifest writes a npurun.json for any local
    Genie bundle (sha256 sealing, arch + quant sniffing, chat-template
    defaults — runs on Windows ARM64), inspect pretty-prints +
    verifies, export orchestrates a fresh qai-hub-models cloud
    compile and chains into manifest. 8 pytest cases pass.
  • winget manifest at manifests/b/bpbonker/npurun/0.1.0-rc.2/.
    Zip-installer type with a portable nested binary; users can
    winget install --manifest <dir> directly off a clone. Validated
    by winget validate.
  • Tag-triggered release workflow at
    .github/workflows/release.yml, gated for self-hosted ARM64
    runner enrolment.
  • CI matrix expanded: python-test (pytest on npu-convert)
    and winget-validate jobs joining the existing fmt/clippy/build/
    test/python-lint set.
  • Dev-cert helper for self-signed MSIX testing
    (scripts/dev-cert.ps1) — generates a code-signing cert in
    CurrentUser\My, imports into LocalMachine\TrustedPeople so a
    signed dev MSIX installs without developer mode.
  • Release runbook at docs/release.md — copy-paste flow for
    cutting a release, plus three production code-signing options
    with pricing.

Verified performance (Snapdragon X Elite X1E80100, QAIRT 2.45)

Model Quant Steady-state TTFT Energy
Phi 3.5 Mini (NPU) w4a16 ~11.7 tok/s ~194 ms ~1.27 J/token at 6.9 W delta
Qwen 2.5 7B (NPU) w8a16 ~1.9 tok/s ~660 ms not yet measured

Phi 3.5 Mini is chat-usable and roughly 2–3× more energy-efficient
per token than llama.cpp on the same laptop's CPU. Qwen 2.5 7B runs
but is slower than CPU paths on this generation of silicon. See
docs/benchmarks.md
for raw timings and methodology.

Quick start (with this release)

  1. Install QAIRT SDK 2.44 or 2.45 from the
    Qualcomm developer portal
    (free account; product name "Qualcomm AI Engine Direct"). Set
    QNN_SDK_ROOT to the install path. The SDK is not redistributable,
    so no release artifact bundles Genie.dll / QnnHtp.dll.
  2. Download npurun-0.1.0-rc.2-aarch64-windows.zip, extract anywhere.
  3. Verify QAIRT: pwsh -File scripts\setup-qnn.ps1
  4. Pull and run: 
    $env:NPURUN_MODELS_DIR = "$env:LOCALAPPDATA\npurun\models"
.\npurun.exe pull phi-3.5-mini
.\npurun.exe run phi-3.5-mini "Tell me a one-line joke about Snapdragon laptops."
.\npurun.exe serve --model phi-3.5-mini   # http://localhost:11435

Full runbook in INSTALL.md inside the zip.

Artifacts

File Size sha256
npurun-0.1.0-rc.2-aarch64-windows.zip 2.58 MB cf25a36901a5c002e28f72285e01974aed66354a43d3a5f3ade0f78e779120c4
npurun-0.1.0-rc.2-aarch64-windows.msix 2.58 MB a9ad260f30f8a25a8114d3b66d414595c9ee6ba2105d0c44331f678296aabafe

The .zip is the recommended path. The .msix is unsigned in
this release candidate — to install it you must enable Windows
developer mode and run Add-AppxPackage -Path npurun-...msix. A
code-signed MSIX + winget catalog submission is on the Phase 6
roadmap (item 6.6).

What's still open before final v0.1.0

  • Production code-signing cert (Azure Trusted Signing or equivalent).
  • Public microsoft/winget-pkgs catalog submission (gated on signed
    installer).
  • Self-hosted ARM64 CI runner with QAIRT enrolled (so cloud CI can
    build full release artifacts on tag push).
  • Remote registry beyond the hardcoded npurun pull list.
  • Docs site (mdBook → github.io).

See docs/roadmap.md
for the wave plan.

Project links

License and legal

npurun is open-source software released under your choice of the
MIT License (LICENSE-MIT)
or the Apache License, Version 2.0 (LICENSE-APACHE).

No warranty. This software is provided "as is", without warranty
of any kind, express or implied, including but not limited to the
warranties of merchantability, fitness for a particular purpose,
non-infringement, and accuracy of any output.

No liability. In no event shall the author(s) or contributors be
liable for any claim, damages, or other liability — whether in an
action of contract, tort, or otherwise — arising from, out of, or in
connection with the software or the use or other dealings in the
software, including but not limited to data loss, hardware damage,
business interruption, or any consequential, incidental, or punitive
damages. By downloading, installing, or running npurun you accept
these terms and assume all risk.

Independent project — no vendor endorsement. npurun is an
independent open-source project. It is not affiliated with, endorsed
by, sponsored by, or supported by Qualcomm Technologies, Inc.,
Microsoft Corporation, or any model vendor. "Snapdragon", "Hexagon",
"QAIRT", and "Genie" are trademarks of their respective owners and
are used here descriptively to identify the hardware and SDKs that
npurun targets. The QAIRT SDK is the property of Qualcomm and is
governed by its own end-user license; your use of it is a matter
between you and Qualcomm.

Third-party model bundles. Pre-built model bundles downloaded
via npurun pull are hosted by third parties (currently Qualcomm's
HuggingFace organization) and are subject to those parties' license
and acceptable-use terms. npurun fetches and executes these bundles
on your behalf — it does not host or redistribute them. You are
responsible for complying with each model's license, and for any
output generated by those models.

No audit. This release has not been audited for safety, security,
or correctness. Do not deploy it in environments where any of those
properties are load-bearing without your own evaluation.

Use at your own risk.

Assets 6
Loading