Miner Llama v3.0

Miner Llama v3.0: llama.cpp for P106 mining rigs and other Pascal GPUs

A llama.cpp build for NVIDIA GPUs of compute capability 6.1: the P106 and P104 mining cards, the GTX 10 series and the Tesla P4/P40. Built from commit 1551f5fe12424d7006748c1628b0df9550f20171 of this fork (branch pascal-p106-v3-b12076-ladder5, on upstream llama.cpp b12076), for Linux on x86-64 without AVX, so it runs on mining rigs' Celeron hosts. This release is tuned and measured on three models on nine P106-090 cards; other compute-6.1 cards run the same code but were not measured. Measurements, settings and the scripts behind every number: llms-on-crypto-mining-rigs.

Speed

Qwen3.8-Flash-Next, a 125B-parameter model, at 20.40 tok/s for a coding agent on nine 6 GB mining cards and a 4 GB host. On 2026-11-03 it ran at 6.4 tok/s on stock llama.cpp and 7.8 on v2.0's code. Flint and Ornith-1.5-122B run on the same build at their v2-era speed or better.

A coding agent (pi) fixing twelve small bugs, Python and JavaScript, real test runs, the context growing every turn, on nine P106-090 cards (one PCIe 1.0 lane each, a 2-core Celeron with 4 GB of RAM). Generation in tokens per second over every request:

Model, file	Stock llama.cpp b12076	Miner Llama v2.0	v3.0, the model's preset
Qwen3.8-Flash-Next, UkisAI's Swift-1.5 GSQ-RCO Q2_0, 65,536-token context	8.20, 9 of 9 (3 not reached)	7.8, 9 of 9 (no draft)	20.40, 12 of 12
Flint, Unsloth Q8_0, 262,144-token context	14.4, 10 of 12 (65,536-token context)	45.9 and 44.6 (two runs), 12 of 12	45.9, 12 of 12
Flint, Flint Pascal Q8 GGUF (p065, 40.4 GB), 262,144-token context	measured after the release	47.1, 12 of 12	47.4, 12 of 12
Flint, Unsloth UD-Q5_K_XL, 262,144-token context	13.7, 11 of 12 (65,536-token context)	44.2, 12 of 12	46.4, 12 of 12
Flint, Flint Pascal Q5 GGUF (29.4 GB), 262,144-token context	measured after the release	53.8, 12 of 12	49.4, 12 of 12
Ornith-1.5-122B, Ornith-1.5-122B Pascal N2P GGUF (valid1, 48.2 GB), 49,152-token context	6.0, 11 of 12 (UD-Q2_K_XL: stock cannot load the N2P/N3P types)	9.7 (UD-Q2_K_XL)	16.3, 12 of 12

• Flash-Next: the preset on this build generates 20.40 tok/s, 12 of 12, with prompts at 69.4 tok/s and the first token of a turn at 6.89 s (median). Stock ran the same server line with the MTP draft and without the fork's switches. My Pascal type mix of the file, Swift Qwen3.8-Flash-Next Pascal Q2 GGUF (pmix-q2-fast, not published), ran the preset at 20.75 tok/s, 2.45 against 2.47 minutes a task at equal work: within the spread of two runs, so the preset uses UkisAI's file as published. Data. The keep-alive alone: 20.75 against 19.19 tok/s for the same line without it, −8.4 % time a task at equal work (data). v2.0's 7.8 and an older stock commit's 6.4 (7 of 8 tasks tried): data.
• Flint: 45.9 tok/s on this build (1f7215948) with v2's command line, against v2.0's 45.9 and 44.6 in two runs; first token per turn 2.67 s (v2.0: 2.6). Flint Pascal Q8 GGUF, my 40.4 GB file of Q8_0's quality, on 1551f5fe1: 47.4 tok/s, first token 2.66 s. Data; v2.0's numbers: results.md. Stock b12076 runs at 65,536 tokens of context (at 262,144 its host-side attention mask does not fit in 4 GB), tuned to -b 2048 -ub 512 with flash attention: 14.4 tok/s, 10 of 12, first token 4.13 s (median). Flint's 5-bit lane on 1551f5fe1 with v2's line for those files (-b 512 -ub 128 --pipe-depth 4, layers by count): Flint Pascal Q5 GGUF 49.4 tok/s, 12 of 12, against v2.0's 53.8. The gap is one long session (complex-numbers, 31 turns, 14,024 tokens at 38.0 tok/s as its context grew); the other eleven tasks generate at 53.64 against 54.04 tok/s. Unsloth's UD-Q5_K_XL: 46.4 tok/s, 12 of 12, against v2.0's 44.2 (one run each), first token 2.68 s. Stock on UD-Q5_K_XL, the same tuned line: 13.7 tok/s, 11 of 12.
• Ornith-1.5-122B: 16.3 tok/s, 12 of 12, first token per turn 8.39 s, with the keep-alive (GGML_CUDA_KEEPALIVE=decode), which its preset sets. Without it 14.9 tok/s on this build (1f7215948), against 14.5 for the same file on 2026-10-28, first token 9.41 s against 9.57. Data; the earlier run and v2.0's 9.7 (stock 6.2, both UD-Q2_K_XL): data.

Flint's Q8_0 runs and Ornith's runs on its N2P file used 1f7215948; this release is 1551f5fe1, the same code plus one row-guard fix (below) that cannot change these models' tokens. On the rigs both commits give the same greedy tokens for Flint (8 of 8 requests) and Flash-Next (both prompts); Ornith's is measured after the release; on the P106 the release build's KL divergence against Ornith's Q8_0 is within noise of the low-bit build's (d = −0.00068 ± 0.00069), and its preset runs 16.42 tok/s, 12 of 12, on the release build (a second rig).

Speed-up against stock llama.cpp b12076, the upstream release this engine is built on (20a51f24a, no fork commit), each engine on the same file:

• Flash-Next: 20.40 against 8.20 tok/s (2.48×), 12 of 12 against 9 of 9 reached; on the nine tasks stock reached, a task's work takes 1.90 against 3.74 minutes.
• Flint: 45.9 against 14.4 tok/s (3.18×), 12 of 12 against 10 of 12, 1.2 against 2.9 minutes a task (median). On Unsloth's UD-Q5_K_XL: 46.4 against 13.7 tok/s (3.38×), 12 of 12 against 11 of 12, 1.0 against 3.0 minutes a task.
• Ornith-1.5-122B: on Unsloth's UD-Q2_K_XL, 12.6 against 6.0 tok/s (2.10×), 12 of 12 against 11 of 12, 1.8 against 3.1 minutes a task (median). Stock cannot load Ornith-1.5-122B Pascal N2P GGUF, on which the preset runs at 16.3.

What is new since v2

Every feature behind a switch. Every behaviour the fork adds to upstream b12076 can be turned off on its own at run time, in one binary, and every experiment stays off unless set. With every switch at its default the build gives the same logits, bit for bit, as the build the Flash-Next runs before it used, on every architecture llama.cpp's test-llama-archs builds (data). The list with defaults, what each does and which preset sets it: docs/switches.md.

One preset per model. Each model has a settings.env with the switches and the command line it was measured with; one command starts it (below).

Pascal kernels for Flash-Next's shapes (on by default on compute 6.1):

• A Pascal kernel for BF16 matrices at 1–8 tokens, and for 2-bit Q2_0 experts.
• k-column kernels for dense products at 1–8 tokens, and a multi-token expert kernel that reads each distinct expert once for 2–8 tokens.
• Together with upstream b12076, a Flash-Next token on nine cards went from 136.0 ms to 74.9 ms, and a four-token verify from 305.4 to 162.8 ms (data).
• Pascal kernels for my own 2- and 3-bit types N2P and N3P (Ornith-1.5-122B Pascal N2P GGUF).

The hop cast (LLAMA_HOP_CAST=q8_0 GGML_CUDA_HOP_STAGED=1, off by default; Flash-Next's preset sets it): the model's state crosses each of the eight card boundaries in Q8_0 through a pinned host ring instead of F32. Without a draft, 14.98 → 17.06 tok/s on Flash-Next, with no measurable change in KL divergence against the F32 hops (data). Flint and Ornith keep F32 hops until their own quality check.

The MTP draft with a shortlist head (Flash-Next): the published Gyro draft guesses three tokens ahead, and LLAMA_DRAFT_SHORTLIST=32768,8192,2 runs its output head over a shortlist of tokens with an exact fallback. The draft took Flash-Next from 7.8 to 12.2 tok/s on v3's base (data); the shortlist adds 10 % on the newer kernels, 14.6 → 16.0 tok/s (data). With the hop cast and the newest kernels on top: 18.8 tok/s (data).

The keep-alive (GGML_CUDA_KEEPALIVE=decode, off by default; Flash-Next's and Ornith's presets set it): on a layer split only one card works at a time, and the driver drops the waiting cards from their boost clock. One block of one thread per card, on the card's lowest-priority stream, polls a word in host memory while the split decodes, so every card stays at 1,873–1,911 MHz instead of 1,354. Plain decoding takes 51.06 and 51.08 ms a token against 62.75 and 59.62 without it (two loads each); on the agent check 19.19 → 20.75 tok/s and prompts +12.5 %, at 16.5 and 16.7 J a token against 15.1 and 15.8 ([data](https:...

Miner Llama v2.0

Miner Llama v2.0: llama.cpp for P106 mining rigs and other Pascal GPUs

A llama.cpp build for NVIDIA GPUs of compute capability 6.1: the P106 and P104 mining cards, the GTX 10 series and the Tesla P4/P40. Built from commit 37e717ff65f641a2b365eebeef33fc1e1292e707 of this fork, for Linux on x86-64 without AVX, so it runs on mining rigs' Celeron hosts. This release is tuned and measured on Flint, on nine P106-090 cards; other compute-6.1 cards run the same code but were not measured. Measurements, settings and the scripts behind every number: llms-on-crypto-mining-rigs.

Speed

Flint in near-lossless 8-bit on nine mining cards at the full 262,144-token context, about three times as fast as stock llama.cpp for a coding agent. Unsloth's Q8_0 (42.3 GB) picks the original model's next token 93.3 % of the time (KL divergence 0.052 against the 16-bit weights), against 87.7 % (0.120) for the common 4-bit file UD-Q4_K_XL; it runs 9–17 % slower than the 4-bit files. The fastest file in this test is the project's own Flint-Pascal-Q5 (29.4 GB, 88.8 %): closer to the original than UD-Q4_K_XL, a little further than Unsloth's UD-Q5_K_XL (90.7 %), which ran at Q8_0's speed here (44.2 tok/s).

A coding agent (pi) fixing twelve small bugs, Python and JavaScript, real test runs, the context growing every turn, on nine P106-090 cards (one PCIe 1.0 lane each, a 2-core Celeron with 4 GB of RAM). Generation in tokens per second over every request, and minutes per task (median):

	Stock llama.cpp	v2.0
Unsloth Q8_0, -ts 6,6,5,5,6,5,5,6,6	14.4 tok/s, 2.9 min (65,536-token context)	44.6–45.9 tok/s, 1.1–1.4 min (262,144)
Flint-Pascal-Q5	15.6 tok/s, 2.1 min	53.8 tok/s, 1.2 min (262,144; all twelve tasks in 17 minutes)
Unsloth UD-Q4_K_XL	14.0 tok/s, 3.1 min	50.7 tok/s, 1.1 min (262,144)

Stock llama.cpp runs at 65,536 tokens because at 262,144 its attention mask, built on the host, does not fit 4 GB of RAM. Pipelined speculation alone is worth +52 % in this test (29.3 → 44.6 tok/s on Q8_0). Results match stock llama.cpp within rounding (the kernels change only the order of additions), and pipelined speculation gives exactly the tokens of plain decoding.

What is new since v1

One conversation, faster per token:

• One persistent kernel per MoE layer at one token, instead of 13 launches: +5.6–8.8 % per token on Flint. It covers Flint's layer shapes; other models take the normal path.
• 32-bit activation sums in the Pascal kernels, so llama.cpp's MoE down-projection rescale is dropped for one-token steps automatically on compute 6.1 (+9.6 % on Flint UD-Q4_K_XL). LLAMA_NO_MOE_DOWN_RESCALE=0 keeps the rescale.
• A Pascal decode-attention kernel for attention of Flint's shape (head size 128, 8 query heads per KV head), in the build but off by default (GGML_CUDA_FATTN_PASCAL=1 turns it on): it slowed generation in the release test.
• Pascal one-token kernels for Q4_K and Q6_K as well, next to Q4_0, Q4_1, Q5_0, Q8_0, Q5_K and IQ4_XS: the bulk of the common Q4_K_M downloads of other models (GGML_CUDA_MMVQ_PASCAL_SKIP=q4_K,q6_K switches either off).
• Pascal kernels for steps of 2–4 tokens, and the shared expert fused with its activation (+0.7 %).
• Less host work per token on compute 6.1: the attention block of one token in two launches. The embedding row looked up on the first card and one packed upload per card and token are in the build but off by default (LLAMA_INPUT_ON_GPU=1, LLAMA_INPUT_UPLOAD=1): they slowed generation in the release test.

Pipelined speculation in llama-server: on a layer split only one card works at a time, so guessed tokens enter the idle cards while the later cards finish the current token, and each guess is checked by sampling the real token: the output is the same as without it.

• --pipe-depth N guesses in flight (4 on nine cards), --pipe-guess none|copy|cont, --pipe-copy-min N.
• --pipe-floor N keeps at least N guesses in flight, with a filler token when the guesser has none, so idle cards don't fall to their base clock.
• For requests sampled on the GPU (-bs) without repetition penalties; tool requests run pipelined until the tool call.

Agents:

• Tool requests sample on the GPU: one token is checked against the grammar on the host, and the full row of scores crosses the PCIe link only when the grammar rejects it. Before, every tool request sampled on the CPU and decoded 12–16 % slower than chat. LLAMA_GRAMMAR_HOST_SAMPLING=1 restores sampling on the host.
• Prompt caching for agents: --prefix-cache DIR saves the state of the shared system and tool prompt to a file the first time it is read and restores it into any conversation that starts with it, across server restarts; one checkpoint per request holds the whole sliding-window ring; --checkpoint-gen also takes checkpoints while generating, at </think> and <tool_call>; and a request no longer re-reserves the scheduler before its first token. Off unless set: in the agent test it cost 11 s of fixed time per task and gained nothing.

Prefill, long context and several conversations:

• MoE tiles sized by tokens per expert (upstream PR ggml-org#31047): prefill +22–29 %.
• The attention mask built on each card (LLAMA_GPU_KQ_MASK=N: for micro-batches of N tokens or more), in the build but off by default: built for every micro-batch it slowed long prompts 3.6× for +2 % in generation.
• A Pascal prefill-attention kernel (head size 128 or 256, 8 query heads per KV head) that skips fully masked key tiles and reads a Q8_0 KV cache directly, without an F16 copy: prefill +10 % at 8,192 tokens and +3 % at 32,768 on Flint-Pascal-Q5; on by default on compute 6.1 (GGML_CUDA_FATTN_TILE_PASCAL=0 turns it off).
• Several conversations through one pipeline (the llama-pipespec tool's streams mode): conversations staggered through the cards, and a new request's prefill chunks on CUDA streams below the generating tokens' priority, so they no longer stall the conversations that are generating.

Coming later (v2.1): a trained guesser, the staircase drafter (--pipe-draft FILE), and a store of Flint's earlier answers (--pipe-store FILE). The code is in this build; the files come with a later release (how the drafter works).

Install

Requirements: Linux with glibc 2.38 or newer (tested on Ubuntu 24.04) and NVIDIA's proprietary driver 580 (nvidia-driver-580, not -open). No CUDA toolkit: the second asset carries the CUDA runtime libraries.

sha256sum -c SHA256SUMS
tar xzf llama-p106-v2.0-bin-ubuntu-cuda12.9-sm61.tar.gz && tar xzf cudart-llama-p106-v2.0-bin-ubuntu-cuda12.9.tar.gz
llama-p106-v2.0/llama-server --list-devices

Both archives unpack into llama-p106-v2.0/. scripts/p106_check.sh in the repository checks the driver, the cards, host memory, the model's read speed and the shipped command lines. Flint Q8_0 on nine cards, chat with the full context:

llama-p106-v2.0/llama-server -m flint-Q8_0.gguf --alias flint -ngl 99 -fa on -c 262144 -np 1 -b 512 -ub 128 -ts 6,6,5,5,6,5,5,6,6 -bs --ctx-checkpoints 4 --cache-ram 0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 --pipe-depth 4 --host 127.0.0.1 --port 8080

-ts 6,6,5,5,6,5,5,6,6 splits the layers by memory so that Q8_0 and the full context fit; for a 4-bit file (UD-Q4_K_XL, or this project's Flint-Pascal-Q5 on six cards) drop it.

The repository's Flint page explains every flag, the six-card and agent variants, and the speed and quality behind them.

Assets

File	Size	SHA-256
llama-p106-v2.0-bin-ubuntu-cuda12.9-sm61.tar.gz	254 MB	f213259e52f11a2893be46e69dc1683bc8b98296f07cfb476ff4c19a681baec1
cudart-llama-p106-v2.0-bin-ubuntu-cuda12.9.tar.gz	554 MB	65b46821ab94c5a46ec762f4fc8896de857ada120770d5a2499695ed3277a3bf

BUILDINFO.txt in the first archive records the commit, the build image, and that the binaries contain no AVX instructions and need glibc 2.38 at most. A third archive carries test-backend-ops, llama.cpp's operator tests, for anyone who wants to check the kernels on their own cards.

Licences

The binaries are llama.cpp and ggml with this fork's changes, all under the MIT licence (LICENSE, THIRD_PARTY_NOTICES.md). The CUDA runtime libraries in the second archive are NVIDIA's, redistributed unmodified under the NVIDIA CUDA Toolkit End User License Agreement (NVIDIA_CUDA_RUNTIME_NOTICE.md). Model files are not part of this release and keep their own licences.