DeroStorm 1.5.6 - Stage 1 under the sort

Stage 1 now runs underneath the suffix sort instead of waiting for it. 127,930 H/s on an RTX 5080 against 119,463, and 159.0 – 159.5 KH/s with the CPU mining beside it.

	1.5.6	1.5.5
RTX 5080, --bench --gpu=all	127,930 H/s	119,463	+7.1%
CPU + GPU, real mining path	159.0 – 159.5 KH/s	153.9 – 154.7	+3.2%

Hashes are unchanged and still bit-identical to the CPU. gpu/hash_parallel_test.exe matches all 512 reference vectors, the pipelined path is checked against the one-batch path by go test -run GPU, and the miner still re-verifies the device against the CPU before it will submit anything.

---

The card was doing one thing at a time

A GPU hash is three kernels: stage 1 builds a ~71 KB text, the suffix sort sorts it, and a SHA-256 check turns the result into a difficulty test. They ran strictly one after another, because all three shared a single set of texts and suffix arrays and none of them could start until the last had finished with them.

Measured on a 5080 at the default 32,768-nonce batch:

kernel	ms a batch	share
suffix_kernel	207.5	77.8%
stage1_kernel	39.0	14.7%
sha_check_kernel	20.0	7.5%

So 22% of the card's time was spent on two kernels while the thing that actually does the work stood still.

They do not have to take turns

The three are short of completely different things. Stage 1 is capped at about 193 threads an SM — it needs 516 bytes of shared memory per thread for the state machine and the RC4 permutation, and no block size changes that, which is 9.4% occupancy and the whole reason it is slow. The suffix sort runs four 256-thread blocks an SM on 11 KB each and waits on memory latency. The SHA check waits on bandwidth.

gpu/overlap.cu is a new probe that puts two of them on two streams over separate storage and times the pair against each alone:

stage 1 alone      13.01 ms       suffix alone   29.88 ms
both, two streams  32.48 ms   ->  80% of stage 1 hidden

sha alone           9.95 ms       suffix + sha   31.91 ms
                              ->  80% of the SHA check hidden

Both look free. Only one of them is, and finding out which is most of this release.

Two banks, a stream each

The inter-kernel storage is now two banks. A chunk takes a bank and owns it from stage 1 through to its SHA check, and each bank has its own stream, so the ordering a chunk needs against the previous chunk on its own bank is the stream's own and costs no events at all.

One thing does need an event. The suffix sort's scratch pool is indexed by blockIdx and shared by every block in the grid, so two suffix kernels must never overlap however independent their data is; each bank waits for the other's sorted event before launching its own.

The banks together hold exactly what one bank held, because the chunk is divided by the same number they multiply it. The overlap is paid for in chunk size, not in VRAM.

The half that does not pay

Letting the SHA check run under the next chunk's sort as well makes the miner slower than never having started:

	H/s
one bank	123,265
two banks, SHA overlapping too	116,116
two banks, SHA kept out of the way	127,532

The SHA check reads back the suffix arrays the sort has just written, so it fights the sort for the one thing the sort is short of. Profiled, its own elapsed time nearly trebles — 20.0 ms a batch becomes 55.5 — and the sort gains nothing for it. Stage 1, which waits on shared memory rather than DRAM, costs the sort 6.5% of its own elapsed time and hands back four times that.

So the sort's turn is released after the SHA check rather than before it. DSG_OVERLAP_SHA is the knob, and 0 is the shipped answer.

Per batch, before and after:

one bank      suffix 207.5   stage1 39.0   sha 20.0   = 266.5 ms, all serial
two banks     suffix 221     stage1 55.1   sha 25.0   =   246 ms, stage 1 hidden

DSG_BANKS=1 rebuilds the old one-thing-at-a-time shape if you want to see it.

---

Measured and not kept

Three things were built or swept in this round and are not in the release. They are written up in gpu/derostorm_gpu.cu and the README so nobody repeats them.

Aligned loads in the stage-1 hashes. ld32le in crypto.cuh builds a 32-bit word out of four byte loads and ld64le out of eight — exactly the mistake the suffix kernel's descKey32 had, and every load xxhash64, siphash24 and fnv1a64 make walks the state buffer from a four-byte-aligned base. Fixing it is exactly neutral: 25.6 ms against 25.8, inside the noise. Stage 1 is not short of load slots, it is short of threads.

Ablating stage 1 to find its cost centres. This does not work at all, and it is worth knowing why. Every part of the loop feeds lhash, and lhash picks the next operation — so removing any part sends the state machine down a different path through a different number of iterations. Removing all three probabilistic hashes "saves" 69%; removing them one at a time saves 2% each. Both numbers are real measurements of something that is not the question.

S1_BLOCK on its own. 32, 64 and 128 are flat, and 96 and above will not launch — stage 1 never asked for the >48 KB dynamic shared-memory attribute. It would not help if it did: 516 bytes a thread pins an SM at ~193 threads whatever the block size. Under the overlap, 64 stays best (127,532) with 16 close behind (127,397) and 32 worst (125,623).

What is left

suffix_kernel is still 78% of the card, and its column walk still has the 1.98× load imbalance measured at 1.5.5 — the slowest thread takes twice the average, so a perfectly balanced walk would cost half of what this one does, which is ~16% of the kernel. Nothing here touched it, and the two ways of collecting it that were tried at 1.5.5 both failed. The SHA check is now the largest exposed cost at 25 ms a batch, running at about 465 GB/s of the card's ~960.

The CPU row of the README's headline table reads 31,150 H/s. Nothing on that path changed in this release; that is this machine on this day, and it is left as measured.

Checksums (SHA-256)

1233a0bb93e95354fdbbd5f9af5517bade21092e46dada3c24b2b6498ccc2881  derostorm-1.5.6-windows-amd64.zip
fee003daccd5cf3cc52b256cd05a477193482dccb8cc11fead7bc38f5dd8a4f3  derostorm-1.5.6-linux-amd64.zip
0b02e470f6ef9599e3664a728761965040042ca031caaad0823419d9c4bc4fa4  derostorm-1.5.6-linux-arm64.zip
11bfe70e4083ab96219a5534e5a8533342cadbd82b9c4ea9a8a42257fb779993  derostorm-1.5.6-macos-amd64.zip
7696351db085a61deb11c6d851cb285b98ef80179341889b951175f80de5d98d  derostorm-1.5.6-macos-arm64.zip