v0.5.7: speculative decoding

A new side topic of stage 4: 4b, speculative decoding.

A cheap draft guesses the next tokens and the model checks them all in one pass, keeping those it agrees with: with the rule of Leviathan et al. and Chen et al., the output is distributed exactly as the model's own. Measured one reply at a time on this laptop's GPU: drafted by the 0.5B model, Qwen2.5-7B writes 38.2 tokens a second against 16.9 alone, 2.26 times as fast; drafted by the 1.5B model, 1.76 times. Qwen2.5-1.5B gains only 1.18 times from the 0.5B draft, a third of its size, and the story models of 3a are slower with any draft: a round of draft and check has to cost less than it yields. Prompt lookup, which drafts from the text itself and costs almost nothing, makes the 1.5B and 7B models about 1.45 times as fast on GSM8K, where a solution copies the numbers of its problem.

In 16-bit floats on the GPU, greedy replies with a draft part from the model's own at near ties, at nearly the same token whatever the draft; in 32-bit floats on the GPU, and on the CPU, every reply measured is the model's own.

Also: speculative.generate now ends a reply before its stop token, as inference.generate does; before, a reply with a draft looked one token longer than the model's own. scripts/bench_speculative.py takes --dtype, and scripts/analyze_speculative.py summarizes the measurements.

The films of every chapter are attached, 25 of them.