Skip to content

NInfer Windows 0.1.0 (portable, CUDA 13.1)

Choose a tag to compare

View all tags
@natpate natpate released this 17 Aug 06:49
· 52 commits to master since this release
v0.1.0
a46e4b1

First portable Windows release of ninfer-windows — a fork of Neroued/ninfer with native Windows 11 x64 support.

This zip is self-contained: executables, all runtime DLLs (FFmpeg, libcurl, zlib, VC++ runtime; the CUDA runtime is statically linked), launcher scripts, a models\ folder, and SHA256SUMS for verification.

Requirements

  • Windows 11 x64
  • NVIDIA GeForce RTX 5090 with a driver supporting CUDA 13.1

Quick start

  1. Extract the zip anywhere.
  2. Download a model into models\ (links in the README at the top of the archive, or in the repo README).
  3. Run the matching launcher, e.g. qwen3_8_27b.bat. This starts ninfer-serve on http://127.0.0.1:8080 with the API at /v1 and the llama.cpp WebUI at the root URL. The first run downloads the WebUI automatically, so it needs an internet connection.

The launchers default to a 150,000-token context to leave VRAM headroom for the Windows desktop; 200,000 is safe when VRAM is completely free at startup.

Changelog

  • Native Windows 11 x64 build and run (MSVC + vcpkg; static CUDA runtime).
  • Stock llama.cpp WebUI: ninfer-serve serves the unmodified UI in-process (--webui auto-downloads, --webui-dir DIR serves a local copy), and the server accepts the stock UI's API dialect.
  • Portable release packaging with launcher scripts.
Assets 3
Loading