NInfer RTX 3090 — Linux sm_86 build, engine 0.6.1

Prebuilt Linux sm_86 binaries for the NInfer RTX 3090 engine — the build upstream doesn't ship.

Upstream's RELEASE_NOTES_0.6.1.md states: "The project does not publish a prebuilt Linux archive." Their last release carrying a Linux tarball was v0.3.1-rtx3090; v0.4.0, v0.5.0 and v0.6.0 are Windows-only. This archive is built from 0.6.1-rtx3090.

Install

curl -fLO https://github.com/coder903/ninfer/releases/latest/download/ninfer-3090-kit-v0.6.1-sm86.tar.gz
curl -fLO https://github.com/coder903/ninfer/releases/latest/download/SHA256SUMS.txt
sha256sum -c SHA256SUMS.txt
tar xzf ninfer-3090-kit-v0.6.1-sm86.tar.gz -C /root

Gives you /root/kit/bin/{ninfer,ninfer-serve} — no CUDA toolchain, no 883-second build.

Contents

Path
kit/bin/ninfer	CLI (197 MB)
kit/bin/ninfer-serve	OpenAI/Anthropic-compatible server (199 MB)
kit/scripts/ninfer-argv.txt	Tuned server argv — 64K context, int8 KV, spec MTP w/ 3 draft tokens, concurrency 4
kit/scripts/{provision,dl_model,ninfer-supervise}.sh	Provisioning helpers
kit/BUILDINFO.txt	Build provenance

Checksums

SHA-256  6c21947f870adc2084049334eeed5a8694db20c8339c5d2a6ec53e1249d998f3
SHA-1    5c0c002b5df93ac7aaa1443a18660788d0a77541

Verified on

Run end to end against a real model, not merely compiled — upstream notes their own Linux acceptance run had no model artifact available:

Host	Vast.ai RTX 3090 (24 GB), Ubuntu 24.04
Model	neroued/Qwen3.8-27B-NInfer, 18.2 GB
Resident	21,013 MiB of 24,576 MiB
Serving	GET /v1/models → 200; POST /v1/chat/completions round-trip 0.69 s

Attribution

These binaries are not original work of this project. Stock, unmodified build of Don-Chad/ninfer-3090 release/v0.6.0-rtx3090 rev 403fc56d71576aa1feddb771cfed3264e7378b20, itself downstream of Neroued/ninfer. Licensed Apache-2.0 — full text included as LICENSE-Apache-2.0.txt below and in third_party/ninfer-3090/. No source changes were made; only the build target (sm_86) and environment (nvidia/cuda:13.1.2-devel-ubuntu24.04) are ours.

The ninfer orchestration script in this repo is separate work under MIT.

RTX 5090 — prebuilt Linux sm_120a binaries (upstream feaf4dd0)

Prebuilt Linux binaries of the NInfer inference engine for the
RTX 5090 (sm_120a).

Upstream publishes no binaries for any platform — no releases, no tags — and states in its README:

There is no install target or packaged binary distribution; NInfer is run from its source build tree.

Its only packaging path is a Dockerfile you build yourself on a machine that already has a 5090
and a CUDA 13.1 toolchain. This archive is the output of that build, so renting a 5090 for an hour
does not mean spending the first part of it compiling.

Contents

File
ninfer-5090-kit-feaf4dd0983f-sm120a.tar.gz	kit/bin/{ninfer,ninfer-serve}, BUILDINFO.txt, LICENSE-Apache-2.0.txt
SHA256SUMS.txt	checksum for the tarball
ATTRIBUTION.md	provenance and Apache-2.0 §4 statement
LICENSE-Apache-2.0.txt	upstream's licence, in full

Build

Source	Neroued/ninfer master
Revision	feaf4dd0983fdaeb2ba4c06eec6da350e644fb3a
Architecture	sm_120a
Image	nvidia/cuda:13.1.2-devel-ubuntu24.04
Host	Vast.ai RTX 5090, Ubuntu 24.04, driver 610.43.03
Build	302 s at -j128, 2026-08-20 16:39:34 UTC

No source changes were made. Upstream's CMakeLists.txt rejects any CMAKE_CUDA_ARCHITECTURES
other than 120a and any CUDA older than 13.1, so neither was a choice made here.

Use it

T=rtx5090-linux-sm120a-feaf4dd0
curl -fLO https://github.com/coder903/ninfer/releases/download/$T/ninfer-5090-kit-feaf4dd0983f-sm120a.tar.gz
curl -fLO https://github.com/coder903/ninfer/releases/download/$T/SHA256SUMS.txt
sha256sum -c SHA256SUMS.txt
tar xzf ninfer-5090-kit-feaf4dd0983f-sm120a.tar.gz -C /root

The binaries need the runtime libraries the build had: libavcodec, libavformat, libavutil,
libswscale, libcurl.

Verified

Run end to end, not merely compiled. Serving Qwen3.6-27B NVFP4 (17.07 GiB) with a 32K context,
int8 KV cache, MTP-3 speculative decoding and concurrency 8: GET /v1/models returned 200 and a
POST /v1/chat/completions returned a correct 91-token answer in 0.93 s.

Licence

The binaries are Apache-2.0 and are not original work of this project — see ATTRIBUTION.md.
The ninfer orchestration script in this repository is separate work under the MIT Licence.