SuperSLM 1.7.0

SuperSLM 1.7.0

Released 2026-09-23.

SuperSLM 1.7.0 can take the token finish's logits off the calling thread in two independent ways:
a caller-supplied host parallel-for hook on both backends, and an opt-in device-resident head on
the GPU. Neither changes a token. The release also lets a host choose the directory GPU shaders
load from, keeps a GPU context usable after an out-of-memory failure during an upload, and makes
sslm_inspect and tok_verify build again.

What changed

Shader directory. GpuContextConfig gains shader_dir, an absolute UTF-8 directory holding
the compiled .cso set, read only during sslm_gpu_context_create. NULL, which every
zero-initialized config already holds, keeps loading from a shaders directory beside the host
executable. The directory is process-wide: the first create that names one, or the first shader
load through the default location, fixes it. A value that cannot name a shader set is refused with
SSLM_GPU_SHADER_DIR_INVALID, and a directory that differs from the one the process already uses
with SSLM_GPU_SHADER_DIR_CONFLICT, both before any device is created.

The token finish. The finish is the final norm, the logits over the whole vocabulary, then the
argmax, schema mask, damped-greedy selection and dead-end rule. The logits are the expensive part.
Every logit is an exact integer sum, and the narrowing, mask, argmax and dead-end rule run
unchanged on the host, on the identical row, so both options below produce the same tokens as
1.6.0.

• A host parallel-for hook, both backends. superslm/parallel_for.h defines
  sslm_parallel_for, a caller-supplied run that the finish hands its logits row blocks to.
  Install it with sslm_workspace_set_parallel_for (CPU) or
  sslm_gpu_context_set_host_parallel_for (GPU). SuperSLM never creates a thread. A run that
  omits, repeats or invents a task index fails the call without producing a token and leaves the
  sequence ready to retry. docs/parallel_for_reference.hpp is a reference run over
  std::thread for hosts with no job system.
• A device-resident head, GPU backend, opt-in per model. Mapping a model with
  SSLM_GPU_RESIDENCY_HEAD_ON_DEVICE uploads its output head table to the device, and the finish
  computes the exact int64 logits row there and narrows it on the host. The new VRAM per mapped
  model is the head table plus two rows, and is measured below. With the flag clear a model maps
  exactly as before and uses no new VRAM.

With no hook installed the finish runs serially on the calling thread, as in 1.6.0, and its tokens
are identical to 1.6.0. Its cost is not identical: see No-hook cost.

One host copy of a tied head. A tied model's head is its embedding table, which the GPU model
handle already holds, so the handle no longer takes a second copy of it. An untied model mapped
without the device head still keeps its lm_head copy.

Out-of-memory recovery. A GPU allocation failure during a model map, an adapter map, a
sequence create or a sequence restore no longer leaves the context's command list recording, so
the next call on the context succeeds. The statuses those calls return are unchanged. Those four
calls, and sslm_gpu_context_create, now set their output handle to null on every refusal.

Tools and build. tools/build_inspect.bat links again, and CMake builds sslm_inspect and
tok_verify as targets linking the superslm library. build.bat completes again: its ABI
verb-count gate now expects 38, and it compiles four test suites that previously had no build
recipe.

Compatibility and required action

GpuContextConfig grows from 4 to 16 bytes on x64 and is passed by value. The change is
source-compatible and binary-incompatible: code compiled against an earlier gpu_1p0.h must be
recompiled against this one before it links with 1.7.0.

GpuResidencyConfig's ignored int reserved becomes uint32_t flags, with the same size, offset
and alignment, so a zero-initializing caller is unaffected. Undefined flag bits are refused with
SSLM_GPU_RESIDENCY_FLAGS_INVALID.

Five statuses are appended to SslmGpuStatus (19 to 23), so no existing ordinal moves. The CPU C
ABI adds one verb, sslm_workspace_set_parallel_for, and no status.

The shipped shader set gains logits_site.cso. Deploy the whole 1.7.0 set: a model mapped with
the device head refuses with SSLM_GPU_SHADER_BINARY_STALE on a stale binary and
SSLM_DEVICE_LOST on a missing one.

GPU sequence saves are unchanged (SLM5), and no model-artifact reconversion is required.

A host gets the speedups below only by installing a hook or mapping with the device head. Without
either, see No-hook cost.

Measured performance

The cell. Measured 2026-09-23 on the reference development machine: AMD Ryzen 9 3950X (16
cores, 32 threads) with 64 GB of DDR4 at 2133 MT/s (four 16 GB modules), Windows 10 Pro, and an
NVIDIA GeForce RTX 2080 SUPER with driver 32.0.15.6094. CPU load was 3% at the start of the run and
0% at the end. 1.6.0 and 1.7.0 are both CMake Release builds with the GPU backend (MSVC 14.33),
from the v1.6.0 tag and the 1.7.0 product source. The models are Qwen2.5-0.5B-Instruct (24
layers) and Qwen2.5-1.5B-Instruct (28 layers) as .sslm artifacts, with no adapter. Every run
decodes greedily from the same 11-token prompt, at the full layer budget per step.

Tokens were identical to 1.6.0 in every cell below, at every thread setting and with the
device head on or off.

These are measurements of these models, this prompt and this machine, not guarantees. No other
CPU or GPU was measured for these figures.

Device-resident head (GPU)

Flag set, no hook. tok/s is decode steps per second after the first token, each step being the
token's embed, layers, wait and finish. Host finish is the median time of the finish call.

Model and run	1.6.0 tok/s	1.7.0 device head tok/s	Host finish median, 1.6.0 → 1.7.0
Qwen2.5-0.5B, 100 tokens	42.67	73.21 (1.72×)	11.118 → 1.379 ms
Qwen2.5-1.5B, 100 tokens	27.85	55.49 (1.99×)	18.047 → 1.661 ms
Qwen2.5-0.5B, schema potion_shop_order, 29 tokens to the schema's dead end	43.94	75.54 (1.72×)	11.163 → 1.456 ms

VRAM cost. Measured as the process's VRAM use across sslm_gpu_model_map:

Model	Flag clear (same as 1.6.0)	Flag set	Device head cost
Qwen2.5-0.5B	366,936,064 B	504,369,152 B	137,433,088 B (131.07 MiB)
Qwen2.5-1.5B	1,352,531,968 B	1,587,159,040 B	234,627,072 B (223.76 MiB)

Nothing further is allocated per sequence or per token.

Host parallel-for hook

64 tokens, device head off. T is the number of threads the finish's logits run on: the calling
thread plus T − 1 workers of the reference run in docs/parallel_for_reference.hpp, installed
with max_tasks = T. T = 0 is no hook installed. On both backends tok/s is decode steps per second after
the first token. On the GPU, finish is the host finish median; on the CPU it is the median of 30
(0.5B) or 10 (1.5B) finish-only calls.

Backend and model	Measure	1.6.0	T = 0	T = 2	T = 4	T = 8	T = 16
GPU, 0.5B	tok/s	42.61	42.29	55.23	59.64	59.12	57.95
	finish ms	11.056	11.356	6.598	5.438	5.500	5.688
GPU, 1.5B	tok/s	27.85	28.12	36.52	39.37	37.90	38.06
	finish ms	18.045	18.450	10.510	8.900	9.138	9.269
CPU, 0.5B	tok/s	21.78	21.36	23.54	24.16	24.06	23.89
	finish ms	10.072	10.237	5.969	5.006	4.998	5.224
CPU, 1.5B	tok/s	7.68	7.56	7.98	8.09	8.09	8.04
	finish ms	16.789	17.134	9.637	8.477	8.881	8.793

Thread guidance. T = 4 gave the best or equal-best tok/s in every row on this 16-core CPU; 8
and 16 were flat or slightly worse. The finish is a small share of a CPU token, which is mostly
layer time, so on the CPU halving the finish moves tok/s by about 11% at 0.5B and 5% at 1.5B.
Measure T on the target machine.

Grade. Each cell in these two tables is one process per build, with medians taken over the
steps inside that process. At T = 4, and with the device head, the finish times' whole min–max
range lies below 1.6.0's in every row. The CPU 1.5B tok/s gain is smaller than the process-to-process
spread of CPU 1.5B step times in the whole-token runs below, so it is not a resolved figure.

No-hook cost

With no hook installed and the device head off, 1.7.0's tokens are identical to 1.6.0, but it is
measurably slower in some cells. Each figure below is ten processes per build, run alternately
(1.6.0, 1.7.0, 1.6.0, …), with the 95% Welch interval of the difference in means. A difference is
resolved when its interval excludes zero; otherwise the resolution achieved is given. The cause has
not been identified yet.

The finish-alone figures come from the same run as the tables above. The whole-token figures come
from a second run later the same day, on the same machine, builds, models and prompt, with CPU load
2% at its start and 46% at its end.

The finish alone (Qwen2.5-0.5B; CPU median of 30 finish-only calls per process, GPU host
finish median over 32 tokens per process):

Measure	1.6.0	1.7.0	Difference, 95% interval	Reading
CPU finish	10.0596 ms	10.2698 ms	+0.2102 ms (+2.09%), [0.1748, 0.2456]	resolved
GPU host finish	11.0903 ms	11.4455 ms	+0.3552 ms (+3.20%), [0.3182, 0.3922]	resolved

Whole tokens (CPU: median step time; GPU: median token time):

Backend, model, tokens	Measure	1.6.0	1.7.0	Difference, 95% interval	Reading
CPU, 0.5B, 32	step	45.850 ms	46.786 ms	+0.936 ms (+2.04%), [0.871, 1.001]	resolved
	tok/s	21.764	21.334	−0.430 (−1.98%), [−0.495, −0.365]	resolved
CPU, 1.5B, 24	step	135.022 ms	137.443 ms	+2.421 ms (+1.79%), [−5.209, 10.051]	not resolved, ±7.63 ms
	tok/s	7.376	7.194	−0.182 (−2.47%), [−0.541, 0.177]	not resolved, ±0.359
GPU, 0.5B, 64	token	23.746 ms	23.695 ms	−0.052 ms (−0.22%), [−0.497, 0.394]	not resolved, ±0.445 ms
	tok/s	42.054	42.155	+0.101 (+0.24%), [−0.702, 0.904]	not resolved, ±0.803
GPU, 1.5B, 64	token	36.706 ms	37.013 ms	+0.307 ms (+0.84%), [0.064, 0.550]	resolved
	tok/s	27.177	26.920	−0.257 (−0.95%), [−0.476, −0.038]	resolved

So a host that installs no hook and maps without the device head measured about 2% slower per
token on the CPU at 0.5B and about 1% slower on the GPU at 1.5B. At CPU 1.5B and GPU 0.5B no
difference was resolved at the resolution shown.