tensorless v0.1.0

The first tensorless release: run a model tuned for the NVIDIA Tesla P100, on the engine it was tuned with, in one command.

tensorless run iamMrDuncan/Qwen3.8-27B-PN3-GGUF

Launch targets

Model	What it is	Decode on one P100, published image
iamMrDuncan/Qwen3.8-27B-PN3-GGUF	Qwen3.8-27B in PN, our P100-native 3-bit weight format	30.03 tok/s
iamMrDuncan/Qwen3.8-27B-UD-IQ3_XXS-Q3Floor-GGUF	Unsloth's UD-IQ3_XXS with a 3-bit floor (standard llama.cpp types)	26.18 tok/s

Measured with tensorless bench (median of 3 runs of 256 greedy tokens after a 429-token prompt, q4_0 KV cache, 65,536-token context allocated) on a Tesla P100-PCIE-16GB, against the research receipts of 30.06 and 26.26 tok/s. Receipts: docs/receipts/engine-v0.1.0.

What's in it

• Launcher (tensorless v0.1.0), a static binary for Linux x86_64 and aarch64 and for macOS arm64:
  • run, pull, serve, stop;
  • bench, which measures decode the way the receipts were measured;
  • models, info, doctor, capabilities.
• Engine (engine-v0.1.0): a llama.cpp fork tuned for sm_60, with the PN types and the kernels from an automated decode hill climb (Qwen3.8-27B-PN3: 7.74 → 30 tok/s). Its source is in engine/, with PROVENANCE.md.
• Image: ghcr.io/iammrduncan/tensorless-engine:0.1.0-sm60 = sha256:35409ff97c2f518e468790c28bd19206ae777a945fcffe5775585e28fd9939a7.
  • It is the same digest that was measured as 0.1.0-rc1, promoted without a rebuild.
  • It uses CUDA 12.6, and the host needs only the NVIDIA driver (R525–R580).

Install

V=0.1.0
curl -fLO https://github.com/iammrduncan/tensorless/releases/download/v$V/tensorless-$V-linux-x86_64.tar.gz
curl -fLO https://github.com/iammrduncan/tensorless/releases/download/v$V/tensorless-$V-linux-x86_64.tar.gz.sha256
sha256sum -c tensorless-$V-linux-x86_64.tar.gz.sha256 && tar -xzf tensorless-$V-linux-x86_64.tar.gz
cd tensorless-$V-linux-x86_64 && ./tensorless doctor

You need podman (with CDI: sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml) or docker with the NVIDIA container toolkit, and a free P100: the model and its 65,536-token context take most of the 16 GB. See docs/launcher.md.

Known limits

• Long context. Decode at 32K and 65K has not been measured on this engine yet. At those lengths only the hill climb's starting point was measured for Qwen3.8-27B-PN3, and attention will take a larger share of each token.
• PN on the CPU. The CPU backend's PN dot product decodes the wrong codes (engine/PROVENANCE.md). Every recipe runs fully on the GPU; don't run PN models on the CPU or with partial offload.
• No speculative decoding yet for these recipes.
• One card family: only the Tesla P100 (sm_60) is supported.

Credits

The engine builds on llama.cpp, llama-cpp-turboquant and llamAmpere (MIT). The Q3Floor model builds on Unsloth's quantization of Qwen3.8-27B by the Qwen team. tensorless is Apache-2.0; see NOTICE and THIRD_PARTY_NOTICES.md.

tensorless v0.1.0-rc1

Full Changelog: https://github.com/iammrduncan/tensorless/commits/v0.1.0-rc1