DeroStorm 1.5.8 - GPU +11.2%, three key bytes not four
Two changes to the GPU descriptor sort. 142,400 H/s on an RTX 5080 against 128,000, measured interleaved against the shipped 1.5.6 binary on the same machine on the same day.
| 1.5.8 | 1.5.6 | ||
|---|---|---|---|
RTX 5080, --bench --gpu=all |
142,400 H/s | 128,000 | +11.2% |
CPU + GPU, --bench sum |
175,400 H/s | 160,900 | +9.0% |
The real mining path was not re-measured; it needs a node and submits shares. Read the --bench sum as an upper bound on it. CPU mining is untouched.
Hashes are unchanged and still bit-identical to the CPU. gpu/hash_parallel_test.exe matches all 512 reference vectors, gpu/desc_test.exe is correct on all 512 suffix arrays, go test ./cmd/derostorm/ is green, and the miner still re-verifies the device against the CPU before it will submit anything.
One pass, asked twice (+3.7%)
A GPU hash sorts ~20,000 descriptors and then has to place their positions. Step 5a does the placing: it walks the sorted descriptors in windows of 256, stages each window in shared memory, and writes the output coalesced.
Step 5 then walked all 20,000 again, reading three global words for each one — the descriptor, its predecessor and its successor — to ask a single question: does anything else share this key? About 98.7% of the time the answer is no, and step 5a had already read all three words.
So the question is asked there instead, out of shared memory, and the ~250 groups that answer yes are pushed to a compact list. Step 5 iterates that list. The list grows down from the top of the merge's scratch array while the merge's own allocator grows up from the bottom, so it costs no memory and the overflow guard that was already there covers both.
| phase | before | after |
|---|---|---|
| expand to sa | 368.2M cycles | 392.2M |
| find groups | 326.8M cycles | 137.8M |
| the pair | 695.0M | 530.0M |
This is the same mistake 1.5.0 fixed once already and the 1.5.5 scatter rewrite re-introduced: two loops over all nd descriptors, both opening by asking the same question of the same words. It is worth checking for after any change that splits a pass in two.
The sort was ordering a byte it did not need (+6.4%)
The column walk builds a four-byte key and groups the suffixes that share all four, so the radix sort ordered 32 bits in five passes.
Three bytes is enough, and it wins twice: the sort orders 24 bits in four passes, and it has fewer descriptors to order, because a coarser key merges neighbouring groups into one. What it costs is collisions — and the merge that resolves them was 1.6% of the kernel against the radix sort's 20.7%, so there was room for it to grow into.
DESC_KEY_BYTES |
desc_test SA/s |
|---|---|
| 4 (was shipped) | 115,946 / 116,215 |
| 3 | 117,205 / 125,742 |
| 2 | 105,906 / 100,657 |
Two bytes is a clear loss, so three is a peak and not just a direction. DESC_CHUNKS, BR_BITS and DESC_MERGE_WIDE were all re-swept against the new shape over six interleaved rounds each and none of them moved.
This is the trade the CPU sort settled long ago — DSA_KEY_BYTES is 3 there, and a fourth byte measured a clear loss for exactly this reason — and the GPU had never been asked the same question.
What the hardware counters say, now that they can be read
ERR_NVGPUCTRPERM was cleared by hand this session (NVIDIA Control Panel → Desktop → Developer settings → Manage GPU Performance Counters → all users; no reboot needed, it applies to newly launched processes).
Nothing in suffix_kernel is saturated — DRAM 34%, SM 31% — so it is a latency kernel. Its scheduler finds an eligible warp on 23.8% of cycles, the L1 hit rate is 55.6%, only 17.25 of 32 threads in a warp are live, and 57% of every global sector fetched is excessive. The 27.8 cycles between two issues divide almost evenly between waiting at a barrier for sibling warps (35%) and waiting on an L1TEX load (33%).
By source line, the wasted sectors are 43% the text gather in descLoadBE64, 20% the arena write in the column walk, 10% the radix scatter. The largest single stall site is the first instruction after the barrier that ends the column walk — the walk's imbalance wearing a load's clothing.
Seven changes built against that data, and seven killed
| hand the runs out longest-first, to even the walk | -9% |
| arena laid out column-major, so a warp's runs write adjacent spans | +0.5%, noise |
| warp-uniform chunks, so all 32 lanes walk one column | -8% |
| L2 persistence pinned on the texts | -8% |
col_same as a 64-bit register mask instead of a 64-byte local array |
-1.4% |
five uint2 loads for a whole 32-byte suffix comparison step |
-1.8% |
| a two-load eight-byte text load | -4.6% |
| seed the column walk by its key, full compare only on ties | -4.2% |
Every one was exact and correct on all 512 vectors. They divide into two lessons.
This kernel will not trade text locality for anything — not for coalescing, not for balance, not for cache residency. The first four are that, and the losses are 8-9% where the gains are noise.
And it is register-saturated at exactly 64. 64 registers is what fits four blocks on an SM, which is the occupancy the whole kernel is tuned around, and ptxas will not go to 65 — it spills instead. The last four all added a live value, all spilled, and all paid more for the spill than they saved. An optimisation here has to be register-neutral or register-negative before anything else about it matters.
A note on sweeps
cmd.exe splits a batch argument on = as well as on spaces, so a build script taking its defines as %2 receives -DDESC_CHUNKS=8 as -DDESC_CHUNKS and defines it as 1. That produced a confident and wrong "DESC_CHUNKS 2 and 8 are 10% worse" — one build measured twice — and it nearly killed the three-byte key by making its own control five times slower than it is. Defines go through an environment variable now. Distrust any sweep where two settings land on the same number.