Prebuilt Linux binaries of the NInfer inference engine for the
RTX 5090 (sm_120a).
Upstream publishes no binaries for any platform — no releases, no tags — and states in its README:
There is no install target or packaged binary distribution; NInfer is run from its source build tree.
Its only packaging path is a Dockerfile you build yourself on a machine that already has a 5090
and a CUDA 13.1 toolchain. This archive is the output of that build, so renting a 5090 for an hour
does not mean spending the first part of it compiling.
Contents
| File | |
|---|---|
ninfer-5090-kit-feaf4dd0983f-sm120a.tar.gz |
kit/bin/{ninfer,ninfer-serve}, BUILDINFO.txt, LICENSE-Apache-2.0.txt |
SHA256SUMS.txt |
checksum for the tarball |
ATTRIBUTION.md |
provenance and Apache-2.0 §4 statement |
LICENSE-Apache-2.0.txt |
upstream's licence, in full |
Build
| Source | Neroued/ninfer master |
| Revision | feaf4dd0983fdaeb2ba4c06eec6da350e644fb3a |
| Architecture | sm_120a |
| Image | nvidia/cuda:13.1.2-devel-ubuntu24.04 |
| Host | Vast.ai RTX 5090, Ubuntu 24.04, driver 610.43.03 |
| Build | 302 s at -j128, 2026-08-20 16:39:34 UTC |
No source changes were made. Upstream's CMakeLists.txt rejects any CMAKE_CUDA_ARCHITECTURES
other than 120a and any CUDA older than 13.1, so neither was a choice made here.
Use it
T=rtx5090-linux-sm120a-feaf4dd0
curl -fLO https://github.com/coder903/ninfer/releases/download/$T/ninfer-5090-kit-feaf4dd0983f-sm120a.tar.gz
curl -fLO https://github.com/coder903/ninfer/releases/download/$T/SHA256SUMS.txt
sha256sum -c SHA256SUMS.txt
tar xzf ninfer-5090-kit-feaf4dd0983f-sm120a.tar.gz -C /rootThe binaries need the runtime libraries the build had: libavcodec, libavformat, libavutil,
libswscale, libcurl.
Verified
Run end to end, not merely compiled. Serving Qwen3.6-27B NVFP4 (17.07 GiB) with a 32K context,
int8 KV cache, MTP-3 speculative decoding and concurrency 8: GET /v1/models returned 200 and a
POST /v1/chat/completions returned a correct 91-token answer in 0.93 s.
Licence
The binaries are Apache-2.0 and are not original work of this project — see ATTRIBUTION.md.
The ninfer orchestration script in this repository is separate work under the MIT Licence.