Repository navigation
Releases
v0.37.2
Crucible v0.37.2 uses the v0.37.0 engine, built with ROCm 7.2.4. This release fixes the installer and image input.
v0.37.1 was not released. Its setup and update changes are in this release. Its ROCm 10.1 engine is not.
Images
- Transparent PNG and WebP images now get a white background before resize (#14).
- Previously, transparent pixels kept their stored color, usually black.
- Images without transparency keep the same pixels and
image_digest. - Thanks to @dblaber for the report and the exact repro.
Build and ROCm
make,./crucible setupand./crucible updatestop with a ROCm 10 compiler, AMD clang 24 or later.- The ROCm 10 compiler gives about 42% less throughput for a measured 32K prefill. Use ROCm 7.2.4 for a source build.
- To continue with ROCm 10, use
ALLOW_ROCM10=1with make, or--allow-rocm10with setup and update. - The v0.37.0 updater does not know
--allow-rocm10. To update its source build with ROCm 10, useALLOW_ROCM10=1 ./crucible update. - The release engine runs on ROCm 7.2.4 and ROCm 10.1. Its asset is
crucible-engine-v0.37.2-gfx1151-rocm7.2-linux-x86_64.gz.
Known limits
- Peak memory for a plain ROCm 10.1 build under the updater's 4 GiB limit is not measured.
- If that build runs out of memory, use ROCm 7.2.4.
- On a slow disk, a state write can exceed the 8-second handover limit. The server then drops that state.
- The next turn restores the longest saved prefix and computes the rest again. Its tokens can differ from a fast-disk run.
See CHANGELOG.md for the full list.
v0.37.0
Release v0.37.0 is the engine of train 37. It follows v0.34.0.
It adds an opt-in long context of up to 524,288 tokens (YaRN), and the installer now finds the ROCm runtime and can
install a private copy of it. The default configuration (4 sessions of 262,144 tokens) does not use YaRN.
Long context (opt-in)
-
--ctxabove 262,144 turns on YaRN (factor 2, up to 524,288 positions). It is a start option of the same engine.
At or below 262,144 the server sets nothing, and the engine starts as before. -
A request that needs 262,144 positions or fewer runs with the plain rotary positions, also in a long configuration.
A longer request runs with YaRN. A request above 524,288 tokens gets HTTP 400context_length. -
The named configurations (each needs about the same GPU memory for the session state as the default, about 28 GB):
Name Server flags Sessions default --streams 4 --ctx 2621444 of 262,144 long --streams 3 --ctx 524288 --short-streams 2 --short-ctx 2621441 of 524,288 and 2 of 262,144 two long --streams 2 --ctx 5242882 of 524,288 ./crucible setup --layout long(ortwo-long) writes the flags.--layout defaultgoes back. -
--yarn-reply-reserve N(default 32,768) is the reply room that the server counts when it decides the mode of a
request: plain when the prompt plusmin(max_tokens, N)fits in 262,144 positions, else YaRN. -
A conversation that grows past 262,144 tokens prefills once from its first token, in YaRN mode. Later turns use
its YaRN states. -
Limits of this first version: a YaRN start refuses
--state-dir(no disk tier), and the host conversation cache does
not hold a 524,288-token state. A long conversation resumes only from its session slot. -
The engine line limit and the default request body follow
--ctx(4 MiB at 262,144, 8 MiB at 524,288;--vision
keeps 256 MiB). A prompt that fills the context reaches the engine. You do not need to raise--max-body-mib.
Setup and service
./crucible setupdoes not lose a restart. When a setup run stops before the restart (a stopped download, for example),
the next run restarts the server with the new settings../crucible doctorand./crucible setupaccept a service that is still loading the model at boot. They no longer
report "GPU memory is held by another program".- After
./crucible service install,./crucible setupno longer offers the service again, and it names the service logs. ./crucible setuprestarts a server that./crucible startmade when a setting changed.- The "Host header is not allowed" message names the command for both cases.
ROCm
- Current ROCm. Crucible recommends the current ROCm (10.1 tested) and supports ROCm 7.2.4. Setup passes both with no message and
asks for no older version. The guide's first step says "install the current ROCm (10.1 tested)". - The release engine was built with ROCm 7.2.4. It was checked on the ROCm 10.1 runtime and on ROCm 7.2.4 with the same
greedy tokens. The speed numbers of this release come from ROCm 7.2.4. - Private runtime. When a machine has no HIP runtime,
./crucible setupoffers a private copy of the ROCm 10.1 runtime (90 MB from
AMD's package index, with a sha256 check of each file). The default answer is yes. Script use:--private-runtime yes|no.
It needs no root and changes no system package../crucible uninstallremoves it. A system ROCm is always used first. - Kernel parameters. When the GPU memory check fails, setup finds the boot method (GRUB, systemd-boot or a unified kernel
image) and prints the exact change. It applies the change withsudoonly after you typeyes, makes a backup first, and never
restarts the machine.--yesnever applies it.
Other changes
- Per-request abliteration: with
CRUCIBLE_ABLATE_MODE=requestone engine serves stock and ablated requests (request
fieldablate).--model-name NAMEadds a second public model name; its-abliteratedform selects the ablated mode. - The server accepts
reasoningas another name ofreasoning_content, and treats an empty or null value as absent. - The SSD conversation cache keeps its states over an engine update when the new engine has the same state epoch.
Speed
- The default configuration has no new speed column: the README speed tables show v0.34.0. The GPU code of the default
path is the same as in the gated build of this release; we did not measure its speed against v0.34.0 for this release. - The README section "Long context" has rows measured on the release engine of v0.37.0 (4.05 pack, public files,
configurationlong), with the power profile, the number of runs and the date.
Known limits
- With more active agent sessions than slots (for example 6 or 8 sessions on 4 slots), a session can lose its saved
state; its next turn then computes the whole prompt again, which takes seconds to minutes at 100,000 tokens or more.
The output does not change. A fix is in test.
Install
curl -fsSL https://raw.githubusercontent.com/jaedync/crucible-strix-halo/main/install.sh | bashFiles
crucible-engine-v0.37.0-gfx1151-rocm7.2-linux-x86_64.gz: the engine, built with ROCm 7.2.4 for gfx1151.crucible-engine-v0.37.0-gfx1151-rocm7.2-linux-x86_64.needs: the release manifest (commit, sha256, libraries).SHA256SUMS: the sha256 of the two files above.
v0.34.0
Release v0.34.0 is the engine of train 34. It follows v0.33.1.
It has a security hardening of the server, the installer and the update, and it makes the measured speed configuration the
default of the service. The security changes are listed first. The speed paragraph names the numbers.
Security
- API key. A key is required for an address that is not loopback, except in the Docker image. Give the key as a file with
--api-key-file PATH. Without it, a start on such an address exits with code 2 before the model loads. Every route but
GET /healththen needs the key. - Docker. The image allows
--host 0.0.0.0(and::) with no key, so thedocker runline of v0.33 keeps working.
The start prints oneWARNINGline that says a published port on all addresses is an open server. Publish the port
only to127.0.0.1(-p 127.0.0.1:8790:8790), or use a key. A key stays optional in the container: mount a key file
and pass--api-key-fileto reach the port from other machines. See docs/DOCKER.md.
The switch also needs the file/.dockerenvor/run/.containerenv, so it does nothing on a host.
Do not use--network hostwith no key: the container then opens the network of the host. - Docker and the Host list. The image has no Host list by default, so with no key a web page can reach a published port
through DNS rebinding. The service outside Docker (./crucible setupand./crucible start) has a Host list by default.
To turn the list on in the image, set-e CRUCIBLE_ALLOWED_HOSTS=localhost,127.0.0.1,[::1], or use a key
(--api-key-file). The Dockerfile sets no default. - Uninstall removes less.
install.sh uninstall --remove-modelsand./crucible uninstall --remove-modelsremove the
pack folders that setup makes. They no longer remove the whole models folder. They remove the state folder only when
setup made it (setup writes a record in.crucible/state-dir.json). A folder that holds other files, such as
~/.cache, is refused as a state folder at setup. A pack that you keep stays, also when the models folder is the
install folder itself or is reached through a link. The command stops with exit 1 when the server does not stop. The
uninstall of an old install (made before./crucible uninstallexisted) now takes the lock of the update and stops
when an update was cut off.
--remove-modelsalso looks inside the state folder: it removes the folder only when it holds nothing but what the
engine writes there (write-ledgerand the identity folders). Otherwise it keeps the folder and says why. - Update keeps your files. The update stops with exit 1 and a list when a file of yours (tracked by git or not) is in a
folder that the update replaces, for exampledocs/ortools/. Before, the update deleted such a file. Files at
the top level are carried over as before. Tool caches such as__pycache__are not counted. - Update changes the Python packages after your yes. pip no longer changes
.venvbefore the question. The update
records the exact old versions first (pip freeze --all, so pip, setuptools and wheel too). A rollback puts them back
and removes the packages that the new requirements added. install.shstops when it runs as root (exit 2). It downloads over https only: a redirect tohttp://is refused.
It stops (exit 4) when the files thatgit clonefetched are not the commit of the release tag, for example when a
branch has the name of the tag. It asks for sudo once for all missing packages, not once for the tools and once for the
libraries. With--source-buildit still downloads the release list and the manifest (not the engine) and refuses a
tag whose commit is not the commit in the manifest.install.shfinds apython3that cannot make a venv (stock Ubuntu has nopython3-venv). On Debian and Ubuntu it
addspython3-venvto the one package question. Before, setup stopped at the venv step.- The timer template
crucible-update.serviceruns./crucible update --yes. It ranmake updatebefore. - Connection limits. A request head must arrive within 10 seconds. A server with a key holds at most 16 connections
for each address that have not sent the key (after a grace time of 2 seconds), and closes the oldest connection of
the busiest address when all 256 slots are full. When every address holds one connection, only a connection older than
the grace time can be closed, and a new connection is refused with HTTP 503 if none is. An IPv6/64counts as one
address. A server with no key has the head limit only. - Host list for the service.
./crucible setupand./crucible startstart the server with
--allowed-host localhost,127.0.0.1,[::1]and the bind address. This stops DNS rebinding. Add the name of a reverse
proxy with./crucible setup --allowed-host NAME. The remote access guide in the repository (docs/REMOTE_ACCESS.md) has the steps.
When the service runs and a setting changed (the list, the key or the address),setuprestarts the service, so the
running server uses the new setting.setup --api-keyon a configured host now starts the server in the same run.
A server that you start withpython -m serveris not changed. - Request-ID header. The server refuses
Authorization,Proxy-Authorization,x-api-key,CookieandSet-Cookie
as the name of the client request-ID header. - The
cruciblecommand checks a whole value withfullmatch(for example a tag). A trailing newline is
not accepted any more.
Speed
- Measured speed. The README speed tables have a v0.34.0 column. It is the ladder run of Nov 7, 2026 on a
build of this release (it differs from the release engine in the version text only), with the public files, the
balanced power profile and the six words below. The GPU clock was 2,303 MHz on the 4.05 pack and 2,292 MHz on the 3.05
pack. Against v0.33.0, decode with 4 sessions reads +3.9% on the 4.05 pack and +9.0% on the 3.05 pack, decode at 32K
reads +3.8% (4.05 prose) and +3.8% (3.05 code), and the cold prefill rates are within 1.1% on both packs
(4.05 at 32K: -1.1%; the two runs of that row differ by 1.1% themselves). Serial decode with no speculation is
within 0.5%. The charts and the by-length text of the README still show v0.33.0, because no chart run was made for
this release. - The service has the measured configuration.
./crucible startand thedocker runline of docs/DOCKER.md set the words of
the README speed table:CRUCIBLE_PLE_HOST=1,CRUCIBLE_COMPACT_STATE=1,CRUCIBLE_PLE_FETCH=1,CRUCIBLE_DRAFT_HEAD=q2,
CRUCIBLE_DRAFT_CHAIN=hc8,leadandCRUCIBLE_PF_LAUNCH_ROWS=16384+128, with 4 streams of 262,144 tokens,
--conversation-cache-mib 9700and--host-reserve-mib 11264. The 4.05 pack no longer pins 8,192 launch rows. The draft
words change drafts only (the verify pass checks every draft). In the engine they stay off unless set.
A word that you set in the environment of./crucible startstays as you set it; the six words are defaults. python -m serverkeeps--conversation-cache-mibat 0 (off). It now prints one NOTICE line at start that names the
flag and the value of the service, because a server with no cache prefills a whole conversation again on every turn.
Build
- The engine
build-inforecords the compile settings (GPU_ARCH,HIPFLAGS,CPPFLAGS,LDFLAGS,LDLIBS,
ROCM_PATH), the hipcc variablesHIPCC_COMPILE_FLAGS_APPEND,HIPCC_LINK_FLAGS_APPEND,HIP_CLANG_PATHand
HIP_DEVICE_LIB_PATH, and the version of
hipcc.make release-enginerefuses an engine that was built with other settings than the defaults, and an
engine whose compiler version is unknown. - The fault test (
tools/service_faulttest.py) step N9 runs its server undertimeout, in the session of the driver,
so the server ends at its life limit even when the driver is killed.
Known issues
- While the service starts (systemd state
activating, for example during the model load after a boot),./crucible doctor
and./crucible setupreport the check "GPU memory is free" (gpu-free) as failed, because the starting server already
holds GPU memory. Wait until./crucible service statusshowsactive (running), then run the command again. - When
./crucible setupchanges a setting while the service runs, and setup stops before its end (for example during a
download), a second run of./crucible setupdoes not restart the service. The server then keeps the old setting. Restart it
with./crucible service restart.
Install
curl -fsSL https://raw.githubusercontent.com/jaedync/crucible-strix-halo/main/install.sh | bashThe installer downloads the engine of this release, checks every sha256, and runs ./crucible setup. The files of this release are:
crucible-engine-v0.34.0-gfx1151-rocm7.2-linux-x86_64.gzcrucible-engine-v0.34.0-gfx1151-rocm7.2-linux-x86_64.needsSHA256SUMS
Full list of changes: CHANGELOG.md. The previous release is v0.33.1.
v0.33.1
Release v0.33.1 is a patch release of v0.33.0. It has two fixes and one server option that is on by default.
The fixes are the end-of-turn guard (#6, PR #9) and the top-N log-probabilities of a scoring request that the cache
answers in full (#7). Thanks @m-max for both reports and for the pull request.
Fixes
- End-of-turn guard (#6, PR #9, thanks @m-max). A model can write an end-of-turn token (
<|im_end|>or
<|endoftext|>) inside its reasoning or inside a tool call, for example when it quotes a chat template. The reply
then ended at once, with finish reasonstop, no content and no tool call. The server now keeps that token as plain
text and the model goes on. The limit is 16 times for each reply in the reasoning and 4 times for each reply in a
tool call. After the limit the token ends the reply as before, and an end-of-turn token in the answer itself always
ends the reply. The guard acts on chat requests only, not on/v1/completions, and not when the kept text would pass
the thinking cap or leave no room for the reply. Each kept token starts a new engine job, so on a busy server a kept
token can add a short delay (not measured). The option is--end-of-turn-guard on|off(default on);offgives the old behavior. The count of kept tokens is in
usage.completion_tokens_details.end_of_turn_kept, and each kept token writes onecrucible_end_of_turn_keptline
to stderr with no model text. Two fixes to the pull request are in this release. A tool call whose argument holds a
call tag, such as a path or a quoted template, no longer looks open after it closes. Astopstring of the client
that the kept text matches now ends the reply. - Scoring log-probabilities on a cache hit (#7, thanks @m-max). A
/v1/completionsrequest withlogprobsor
top_logprobs(max_tokens1, temperature 0) whose whole prompt the cache answered returned the top-N list of a
later decode step, not of the end of the prompt. The token was right, the list and the value of the chosen token
were wrong. The engine now reads the saved row of the prompt end. Every release since v0.26.0 has this fault.
A request that the cache does not answer in full, a chat request and a sampled request were not affected.
On an older release, sendcache_controlbypasswith the scoring request. This release fixes the top-N list only.
The token difference after a session replay stays a documented limit (docs/SESSION_REPLAY.md). - The
Dockerfiledefault release tag and the build lines indocs/DOCKER.mdare v0.33.1.
Speed
- This release has no new speed numbers. The numbers in the README tables and charts are the v0.33.0
numbers. The one engine change reads a saved row on the path of a temperature 0 request that asks for
log-probabilities, and no other request runs it. The guard runs in the server and acts only on an end-of-turn token.
This release changes no prefill or decode arithmetic. We did not run a new speed measurement for it.
Checks
- A token gate of a test build with the scoring fix, against the engine of daily use, gave byte-identical greedy tokens on
both packs (4.05 and 3.05). The same test build gave the same top-N list for a cache hit and for abypassrequest
on two synthetic prompts. The engine source of that test build is the same as in this release, except the version
string. The numbers are results of that test binary, not of the build of this release. make test-cpupasses on the export tree andmake test-faston the private tree. The export scan with gitleaks
reports no finding.
Full list of changes: CHANGELOG.md. The previous release is v0.33.0.
v0.33.0
Release v0.33.0 is the engine of train 33. It follows v0.32.0.
It has one speed option, one speed change for a second chat, and three fixes. The option is a read hint for the n-gram
(PLE) table when the table is in host memory. The change for a second chat is the late copy of the shared header point.
The fixes are the start-time memory message, a reply with no tokens in the vision check, and the fixed-state size on the
start line. The two draft switches and the smaller arena of v0.32.0 are in this release too.
Speed
CRUCIBLE_PLE_FETCH=1(opt-in, default 0; needsCRUCIBLE_PLE_HOST=1) adds a read hint to the host gather of the n-gram
table. The bytes are the same. In the runs by context length (Nov 3, 2026, same tool and prompts, public files) the rate
against v0.32.0 changes by +1.5% to +2.6% for prefill and by +1.7% to +2.9% for decode on the 3.05 pack, at every length
from 32K to 250K. On the 4.05 pack prefill changes by -1.8% to +1.9% and decode by -3.2% to +0.4%; prefill reads 1.2%
higher at 32K and 1.8% lower at 192K, and decode reads equal at 32K and 3.2% lower at 250K. The two runs are on the same
day but not in one session, so we do not claim a gain or a loss on that pack.- The late copy of the shared header point lets a second cold chat use the header point of its first block, also when
the server runs several slots. It acts only withCRUCIBLE_EXACT_RESUME=1. A night run of Nov 1 to 2, 2026 measured a
test build of Nov 1, 2026 with this change against the train 32 engine, both withCRUCIBLE_EXACT_RESUME=1. All of
its checks passed: the token gates on both packs, the exact-extend load, short and long groups on both packs, and 4
pairs of the burst and sequential times to first token, pooled at 0.927 of the old build (7.3% lower). These are
numbers of that test build, not of the build of this release. The runs by context length did not set
CRUCIBLE_EXACT_RESUME, so the tables and the charts do not include this change. - The v0.33.0 column of the README tables and the charts is the release ladder and the runs by context
length, measured Nov 3, 2026 on the build of this release, balanced power profile (GPU clock 2,289 to 2,295 MHz for
the ladder), with the public files and no tuned draft head file.
The v0.32.0 column of the same tables is the ladder run of Nov 3, 2026. It has no 4-session value on the 4.05 pack
(one session of each run stopped early) and one run on the 3.05 pack. The runs useCRUCIBLE_DRAFT_HEAD=q2,
CRUCIBLE_DRAFT_CHAIN=hc8,leadandCRUCIBLE_PF_LAUNCH_ROWS=16384+128, as v0.32.0 did. - The engine of this release adds the mem-error text, the fixed-state field and the HF vision loader after the chart
binary was built. These changes act at start or at load and change no prefill or decode arithmetic.
Exactness and checks
- The output tokens of a request do not change with the PLE read hint: it reads the same table bytes. The late copy of
the header point keeps the exact state (296 rows restored in both packs of the night run) and the token gates and
exact-extend groups passed on both packs. - The engine of this release is the build that our own server ran in daily use on Nov 4, 2026, plus the start-time
and load changes named under Speed. We publish after that day. make test-cpupasses on the export tree andmake test-faston the private tree. The export scan with gitleaks reports no finding.
Fixes
- The start-time memory refusal names the free host and device memory, the size of the state that does not fit, and a
setting that the same check accepts. The example is a 1-stream 256K request in Docker (8.5 GiB free, 9 GiB reserve):
the message now says that all of the state is already allocated, and that a lower--streamsor--ctxcannot help.
The accept or refuse decision does not change. - The vision token check fails only a reply with no tokens. A reply that ends with a natural stop after a few tokens no
longer fails the check. - The compact-state start line prints the fixed part of one slot's state as
fixed_state_bytes=<n>, after
logical_state_bytes. The field carries sizes only, never request content. A dashboard can draw the KV band and the
recurrent band separately. - A
Dockerfilebuilds the engine for gfx1151 from a pinned release tag and installs the HTTP server; the image
holds no model.docs/DOCKER.mdlists the tested versions and the run line. tests/draft_tally_test.cppcompiles with g++ 13 and glibc 2.39 (Ubuntu 24.04). The compiler read the comma in the
compared type as a second macro argument. A second pair of parentheses keeps the argument whole, and the test checks the
same thing. This is a test-only fix, with no change to the engine. The public CPU tests job failed on this line
(thanks @m-max).- The cache rule in
docs/ARCHITECTURE.mdnames its exception: a turn that resumes from the
answer of an earlier turn (session replay) keeps the rows that the decode path wrote for that answer, so a later
restore of that session is not bitwise equal to a cold prefill. A hit on a snapshot or a block entry
still gives the same bits as a miss. Docs only (thanks @m-max).
Measurements
The speed numbers above are on one machine (Strix Halo, 128 GB) with the balanced power profile and the same tool as
v0.32.0. The tables for both packs, with the run counts and the GPU clock, are in docs/RESULTS.md.
Known issue
- A scoring request (a completion with
logprobsortop_logprobs,max_tokens1, temperature 0) can return wrong
log-probabilities when the server has just served the same prompt from its cache. The returned token is correct.
Requests without logprobs are not affected. All releases since v0.26.0 have this bug. Until v0.33.1, send scoring
requests withcache_controlset to "bypass". Details: #7.
v0.32.0
Release v0.32.0 is the engine of train 32. It follows v0.31.0.
It has three speed changes and one loader fix. The speed changes are a low-bit draft head for the MTP drafts, a draft chain that overlaps the first draft pass with the server work, and a smaller prefill stage arena that lets a prefill launch of 16,384 rows fit on the 4.05 pack. The loader fix is for a checkpoint with a plain HF vision tower (see Fixes).
The two draft changes are opt-in switches (default off). The arena change is on by default.
Speed
- Decode is faster.
CRUCIBLE_DRAFT_HEAD=q2makes the MTP draft head read a 2-bit copy of its rows for coarse logits. The 8 best blocks of 128 tokens are
then recomputed from the EXL3 tiles. The draft changes only; the verify pass checks every draft, so the output does not change.
CRUCIBLE_DRAFT_CHAIN=hc8,leadchanges the draft calls:hc8reads the hyper-connection weights of the MTP layer as int8, andleadstarts the first
MTP pass of the next round while the server handles the tokens of the step.CRUCIBLE_DRAFT_HEADandCRUCIBLE_DRAFT_CHAINare unset by default. - The prefill stage arena is smaller. Row buffers of the attention and MoE views share bytes where their lifetimes do not overlap. At the launch of
16,384 rows (CRUCIBLE_PF_LAUNCH_ROWS, the default,16384+128) the arena is 2,174 MiB instead of 4,656 MiB (2,482 MiB less). The arithmetic does not change.
The launch of 16,384 rows now fits on the 4.05 pack with the long-context layout. - Gate numbers (4.05 pack, ABAB, 4 pairs, order 2 AB and 2 BA, order-corrected; the v0.32.0 engine with
CRUCIBLE_DRAFT_HEAD=q2,CRUCIBLE_DRAFT_CHAIN=hc8,lead
andCRUCIBLE_PF_LAUNCH_ROWS=16384+128against the v0.31.0 engine with its own launch rows of 8192+128; both engines with the same tuned draft head file, which is not part of the release; the cold prefill gain includes the launch-row change):
cold 32K prefill +4.21% (se 0.14%, faster in 4 of 4 pairs), decode of 4 sessions at 32K +4.45%, decode of 1 stream with code text +2.07%, serial decode +0.55% (se 0.07%, a small change),
energy per arm -3.2%. D1 (mixed text, 6 pairs): +4.09%, equal token IDs in 30 of 30 requests of every pair. - Measured Nov 3, 2026 on the build of this release, balanced power profile, in the runs by context length (32K to 250K), same tool and prompts as v0.31.0, public files, no tuned draft
head file: cold prefill against v0.31.0 changes by +1.0% to +3.4% on the 4.05 pack (v0.31.0 measured Nov 1, v0.32.0 on Nov 3) and by -1.9% to -1.0% on the 3.05 pack (v0.31.0 measured Nov 2, v0.32.0 on Nov 3);
decode changes by +2.8% to +4.7% on the 4.05 pack and by +0.9% to +2.6% on the 3.05 pack. The v0.32.0 runs setCRUCIBLE_DRAFT_HEAD=q2,CRUCIBLE_DRAFT_CHAIN=hc8,leadand launch rows 16,384, and also
CRUCIBLE_PLE_HOST=1andCRUCIBLE_COMPACT_STATE=1. The v0.31.0 runs set neither draft switch, and on the 4.05 pack they used 8,192 launch rows.
On the 3.05 pack both builds used 16,384 launch rows; prefill reads 1.0% to 1.9% lower at every length with a higher GPU clock. The runs are on different nights (Nov 2 and Nov 3) and there is no same-night
comparison, so we do not know yet if this is a loss. With the defaults (noCRUCIBLE_DRAFT_HEADand noCRUCIBLE_DRAFT_CHAIN) we expect decode as in v0.31.0; we did not measure the defaults.
The vision loader fix of this release came after these runs; it changes only the vision load path, not prefill or decode.
The tables and the run counts are indocs/RESULTS.md(section Decode by context length). - The release ladder row of v0.32.0 is not in the tables: the ladder run of Nov 3 did not complete, and the row comes with v0.33.0.
Exactness and checks
- The output tokens of a request do not change with speculation, the draft head, the draft chain or the arena: the verify pass checks every draft, and the arena change does not change the arithmetic.
- Token gates on both packs (4.05 and 3.05) with the three switches on: the output tokens equal the v0.31.0 engine on the fixed gate set (CHAIN PASS).
- The exact-extend load gave the same token IDs on both packs: 6 of 6 cases each.
- A memory soak of 3 runs passed (lowest free memory 13.1 GiB, 124 turns and 123 cache hits per run).
tests/stage_plan_test.cppreproduces the old arena sizes and checks that the live buffers of each pass stay disjoint. The draft head and the draft chain have GPU and host tests
(tests/draft_quant_test.hip,tests/draft_chain_switch_test.cpp).
Abliterated mode on both packs (opt-in)
docs/ABLITERATED.mdnow covers the 4.05 pack.tools/ablation_pack.py repackagemakes the 4.05 copy of the public andrevp sidecar from the downloaded file
and refuses to write it unless the copy has the sha256 that the engine pins. Crucible hosts no sidecar file. The mode is off by default.- The sidecar was tuned on the 3.05 pack. On the 4.05 pack we measured (26 to 29 Oct 2026): 0 of 25 harmful prompts refused (stock 25 of 25); KL from the 16-bit model
0.0248 against 0.0206 (896 positions); decode 2.7% slower; cold prefill 1.3% slower at 32K. 2 of 2,337 turns had a tool call that came back as text
(0 of 10,721 on stock builds). The page lists the details and the responsibility note.
Fixes
- History messages may carry
reasoningnext to or instead ofreasoning_content(LiteLLM and Hermes send both). An empty or null value counts as absent. Only two different non-empty texts, or a value that is not a string, give a 400. Request errors now name the field and the message index (messages[N]) (#4, thanks @dblaber). - A checkpoint whose vision tower is in plain HF format (BF16, no EXL3 tower) now runs vision when
vision_k6.safetensorsfrom the stock 4.05 pack is in the model folder. Without that file,--visionstops at start with a message that names the file and thehf downloadcommand. The engine runs the stock EXL3 tower fromvision_k6.safetensorsand does not use the HF tower of the checkpoint; images are exact only when the HF tower holds the stock weights. The loader also accepts BF16 for the patch embedding and the vision biases and converts them to F16 at load (#3, thanks @OleksKon). repetition_penaltyat1or null is now accepted and has no effect, likepresence_penaltyandfrequency_penalty
at0. It was refused as an unknown field. Any other value returns HTTP 400.
Measurements
Every speed number above except the last two bullets of Speed is on the 4.05 pack, measured with an A/B script on one machine (Strix Halo, 128 GB), alternating the old and
new build in balanced order, with the balanced power profile. The tables for both packs, with the run counts and the GPU clock,
are in docs/RESULTS.md.
v0.31.0
Release v0.31.0 is the engine of train 31. It follows v0.30.0.
It has one engine change: the sparse attention step of prefill (the QSA union flash) uses a new kernel. The output is exact.
The decode path does not change. The server, the cache and the options do not change.
Speed
- Prefill is faster (measured at 32K and 130K on the 4.05 pack, see below). The new kernel
QsaUnionStageKernelreplaces
QsaUnionHalfQKernelin the QSA union flash (12 attention layers). It does the same arithmetic in the same order, so
the bytes are equal. It splits the work between two groups of waves: loader waves bring K and V into LDS with buffer
loads, one step ahead, and the other waves run the QK chain, the softmax and the P V step. It is a build-time constant
(kQsaUnionStageinengine/layers/kernels/prefill_qsa_union.h), not aCRUCIBLE_*switch. Set it tofalseand
rebuild to get the old kernel.- First gate (an engine build of Oct 30 with the new kernel against the same engine without it; 4.05 pack; ABAB, 4 pairs,
order-corrected): cold 32K prefill +2.62% and cold 130K prefill +3.86%. - Against the v0.30.0 engine (4.05 pack; ABAB, 4 pairs, order 2 AB and 2 BA, order-corrected): cold 32K prefill
+2.91% (medians of the arms: 1,729 and 1,766 tok/s), faster in 4 of 4 pairs; the tool rated this as within noise, because the old arm spread 3.4%.
- First gate (an engine build of Oct 30 with the new kernel against the same engine without it; 4.05 pack; ABAB, 4 pairs,
- Measured Nov 1 to 2, 2026 on the build of this release, balanced power profile, in the runs by context length (32K to 250K), same tool and prompts as v0.30.0:
cold prefill against v0.30.0 changes by +1.6% to +4.5% on the 4.05 pack (both measured Nov 1) and by +2.6% to +4.7% on the 3.05 pack (v0.30.0 on Nov 1,
v0.31.0 on Nov 2, a repeat; public files, no tuned draft head file); decode changes by -0.2% to +2.1% and -0.3% to +0.4%. The tables and the reason for the
repeat are indocs/RESULTS.md(section Decode by context length).
Exactness and checks
- Token gates on both packs (4.05 and 3.05): the output tokens are equal to the old kernel on the fixed gate set.
- The bench
bench/qsa_attn.hipcompares the new kernel with a verbatim copy of the old one (bench/kernels/qsa_attn_trunk.h)
on 9 shapes, and 3 deliberate errors in the new kernel must show a difference. All checks passed. - The exact-extend load (a restore of a long state against a cold prompt) gave the same token IDs on both packs: 6 of 6 cases.
- Decode: D1 (6 pairs, 4.05 pack, against the v0.30.0 engine, both engines with the same tuned draft head file, which is not part of the release) is within noise (corrected ratio 0.9987), and the token IDs are equal in all
30 of 30 requests of every pair. A memory soak of 3 runs passed.
Measurements
Every speed number above except the last bullet of Speed is on the 4.05 pack, measured with an A/B script on one machine (Strix Halo, 128 GB), alternating the old and
new build in balanced order, with the balanced power profile. The tables for both packs, with the run counts and the GPU clock,
are in docs/RESULTS.md.
v0.30.0
Release v0.30.0 is the engine of train 30. It follows v0.26.1.
The v0.26.1 engine is the v0.26.0 engine (v0.26.1 changed docs and one tool). So this list covers all engine changes since v0.26.0.
It makes prefill and decode faster, adds a gateway request id to the server log, and stops at a bad draft head file.
Every change below keeps the output exact, except where the text says that it is opt-in.
Speed
Against v0.26.0 on the same machine with the same tool and prompts (v0.26.0 measured Oct 28 to 29, v0.30.0 on Oct 31; balanced power profile,
prefill launches of 8,192 rows): cold prefill is 5% to 6% faster on the 4.05 pack and 8% to 11% faster on the 3.05 pack, from 32K to 250K
(8% to 9% against v0.26.0 measured the same night). Decode is 6% to 8% faster on the 4.05 pack from 64K to 250K (same reply text, same tokens
per verify pass). On the 3.05 pack, decode is about equal (-2% to +1% against v0.26.0 measured the same night). The 3.05
decode values come from builds that used a tuned draft head file that is not part of the release. In the ladder, decode with 4 sessions on the
4.05 pack reads 4% lower (102.4 against 106.9 tok/s); we have not found a cause. The tables are in docs/RESULTS.md in the repository.
- Long prompts prefill in launches of 16,384 rows (was 8,192). The MoE layers have tile kernels for launches of 8,192
rows or more. Method: one lease on Oct 25, 4.05 pack, same prompts, engine build of that date with the tile kernels, three arms in
ABC order (not ABAB). Cold 32K prefill is 3.1% faster (1,746.7 tok/s) and cold 130K is 3.0% faster, with equal token IDs.
The launch needs more device memory: the stage arena is 4,656 MiB instead of 2,346 MiB (3.3 GB more in total).
To keep the old size, setCRUCIBLE_PF_LAUNCH_ROWS=8192+128. - The dense prefill GEMMs have a new half-precision kernel. The kernel is 4.4% to 4.8% faster per call, and its output
bytes are equal to the old kernel. Method: ABAB, 4 pairs, 4.05 pack. Cold 32K prefill is 1.3% faster
(1,709.95 to 1,732.60 tok/s) and cold 130K is 1.2% faster, in 4 of 4 pairs. Energy per prompt is 1.2% lower. - Decode with several sessions is faster. A new MoE kernel (
CRUCIBLE_MOE_B128, default now 12) serves the verify
passes of 2 to 16 rows with a capped grid. Method: ABAB, 6 pairs, 4.05 pack, no draft head file, 4 sessions at 32K, measured on an earlier build
(train 28, switch value 3; the default is now 12, which we did not measure the same way). Decode total was 8.3% faster, in 6 of 6 pairs. Serial decode and cold prefill do not change (within 0.1%).
The start check stops the server with a clear message if the grid is under one block.
To go back to the old kernel, setCRUCIBLE_MOE_B128=0. A bad value stops the start with exit code 2. - The server now starts the weight lookahead for batched passes. Method: ABAB, 6 pairs, 4.05 pack. Serial decode
through the server is 2.9% faster, in 6 of 6 pairs. - Serial decode on the 4.05 pack is 1.7% faster with a new exact kernel for the hyper-connection weights
(ABAB, 5 pairs, equal token IDs). - Other exact prefill changes: a deeper load pipeline in the MoE gate and up kernel, row tiles for MoE prefill from
1,024 rows, a fused GDN scan, and a carried GDN scale. The tail rows of a prompt join the next launch.
Each one is bitwise equal to the old path. We do not claim a separate gain for them.
Conversation cache
- A cache restore of a long state uses far less host memory. The host copy of a spill to the disk tier fell from
3.1 to 5.2 GB to about 250 MB, written in pieces of 8 MiB. The state is restored exactly: a restore of 18,285 tokens
gave the same token IDs as a cold prompt. - A fix makes the write ledger of the disk tier exact: the day count no longer adds a file twice when two block writers
charge at the same time. CRUCIBLE_EXACT_RESUME(opt-in, default 0) makes the resume of a longer prompt equal to a cold prompt. It costs at
most 20 extra prefill rows, one small launch and one save per prompt. Seedocs/SESSION_REPLAY.md.
On the default path, a resume point 1 to 8 rows below the end of a bulk launch is now refused, so more resumes
equal a cold prompt.CRUCIBLE_LAZY_SHADOW(default off) is a check for testing: it keeps the old eager copy and compares each piece.
Server
- New option
--client-request-id-header NAME(alsoCRUCIBLE_CLIENT_REQUEST_ID_HEADER). The default header is
X-Client-Request-Id. A valid value (1 to 64 letters, digits and_ . : -) goes into everycrucible_prefill,
crucible_decode,crucible_turn,crucible_errorandcrucible_cancelline asclient_request_id. A gateway can
pair a request with its lines when requests overlap. The value never goes into a reply. - If
CRUCIBLE_MTP_SIDECARnames a file that is missing, or one that replaces fewer than 17 tensors, the engine now
stops with exit code 2 and the linemtp sidecar error. Before, the engine ran on, and the exact verify step hid the fault: only the draft acceptance fell. - The engine writes an
active config:line at start. It lists theCRUCIBLE_*switches that the environment sets,
with their values. A value with a path shows as(path), and the draft head file shows as its size and a hash prefix.
Build
make split-factsandSPLIT_CHECK=1(default) keep the parallel build exact: a part that sees other compile facts
than the whole module stops the build. Runmake -j 8 split-factswhen the build saysargument facts drift.
SPLIT_CHECK=0skips the check. Seedocs/BUILD.md.CRUCIBLE_GDN_VERIFY(default off) is a GDN verify kernel that is bitwise equal.CRUCIBLE_HC_WMMA=2(default off)
reads int8 hyper-connection weights. It is a quality change, so it stays off.
Measurements
Every number above is a speed change on the 4.05 pack, measured with an A/B script on one machine (Strix Halo, 128 GB),
alternating the old and new build in balanced order. The tables for both packs, with the run counts and the GPU clock,
are in docs/RESULTS.md. The speed table of the README compares v0.30.0 with v0.26.0 and v0.22.0.
v0.26.1
- README: new section "16-bit n-gram table (opt-in, experimental)" with one command,
python tools/ngram16/fetch.py, that downloads only the 33 source files (104.3 GB), builds the BF16 table and checks its size and sha256. Opt-in withCRUCIBLE_PLE_HOST=1 CRUCIBLE_PLE_BF16=<file>. docs/OPENAI_CLIENTS.md: request fields and refusals for OpenAI clients (pull request 1).
v0.26.0: train 26 engine, fast re-send, update notice
Release v0.26.0 is the engine of train 26. It follows v0.22.4.
It adds new packs, a faster build, a conversation cache that explains its misses, and a way to update.
New packs
- The loader reads the
h6_ng8EXL3 packs from turboderp, with self-calibrated weights. The sizes are 2.18, 2.68,
3.18, 3.68 and 4.18 bits per weight. - Each of these packs loads and generates text.
- Weight widths go from 2 to 8 bits. Routed experts can also have half-bit widths (for example 3.5).
- The
h6_ng8packs hold an 8-bit n-gram table.CRUCIBLE_PLE_BF16(off by default) reads the n-gram table at
16 bits from a file instead. It needsCRUCIBLE_PLE_HOST=1. - The packs 3.05 and 4.05 stay the qualified packs. The README tables describe them.
Speed
- Several parts of decode and prefill are faster. Speculative decoding is still exact: the target model checks every draft.
- A draft chain now stops at a stop token and at the token limit.
- Long prompts prefill in launches of 8,192 rows. A short last part joins the last launch.
The tables compare v0.22.0 (A) with v0.26.0 (B). A is the engine of train 22. The releases v0.22.1 to v0.22.4 use the
same engine. Each cell is the median of 5 pairs. The v0.22.0 cell for 4 sessions on the 4.05 pack is the median of 4 pairs, because one pair gave no value. The ratio is B divided by A. For the rows in tok/s, a ratio above 1
means that v0.26.0 is faster. For the rows in seconds, a ratio below 1 means that it is faster.
Setup: Strix Halo with 128 GB. Each pack ran 5 pairs of A and B, in the order AB, BA, AB, BA, AB. A pair that failed a
rule (low GPU clock, high disk wait, or a missing value) was run again once. The 4.05 pack had 7 tries (14 runs) and the 3.05 pack had 5 tries (10 runs). The median GPU clock
was about 2,330 MHz in both builds. The server ran 4 sessions of 262,144 tokens, with MTP on and image input on. Only
the serial row has MTP off. The cold rows use prompts that no cache has seen.
4.05 bpw pack
| Row | A: v0.22.0 | B: v0.26.0 | B/A |
|---|---|---|---|
| Serial decode, short context (tok/s) | 30.9 | 32.0 | 1.036 |
| Decode at 32K, 1 session, code (tok/s) | 66.1 | 64.9 | 0.982 |
| Decode at 32K, 1 session, prose (tok/s) | 49.0 | 49.3 | 1.005 |
| Decode at 32K, 4 sessions, total (tok/s) | 110.8 | 105.2 | 0.949 |
| Cold prefill 32K, time to first token (s) | 19.74 | 19.04 | 0.965 |
| Cold prefill 130K, time to first token (s) | 84.15 | 81.94 | 0.974 |
| Re-send of the same 32K prompt, time to first token (s) | 1.872 | 0.021 | 0.011 |
3.05 bpw pack
| Row | A: v0.22.0 | B: v0.26.0 | B/A |
|---|---|---|---|
| Serial decode, short context (tok/s) | 35.4 | 36.5 | 1.029 |
| Decode at 32K, 1 session, code (tok/s) | 75.3 | 73.4 | 0.974 |
| Decode at 32K, 1 session, prose (tok/s) | 59.5 | 59.2 | 0.995 |
| Decode at 32K, 4 sessions, total (tok/s) | 129.6 | 129.2 | 0.997 |
| Cold prefill 32K, time to first token (s) | 18.75 | 18.78 | 1.001 |
| Cold prefill 130K, time to first token (s) | 80.39 | 80.77 | 1.005 |
| Re-send of the same 32K prompt, time to first token (s) | 1.728 | 0.021 | 0.012 |
What the tables show:
- A prompt that you send again gets its first token in about 0.02 s instead of 1.7 to 1.9 s.
- Cold prefill is 2.6% to 3.5% faster on the 4.05 pack. On the 3.05 pack it is equal within 0.5%.
- Serial decode is 3% to 4% faster.
- Decode with 4 sessions on the 4.05 pack is 5% slower: 110.8 to 105.2 tok/s (0.949). One of the 5 runs of v0.26.0
was low (86.4 tok/s). In the 4.05 test, 3 of the 14 runs gave no value for this row: 2 of v0.22.0 and 1 of v0.26.0.
This is a known fault of the measurement and is not new in this release. The 3.05 test had no failed run. - The 3.05 pack shows no loss with 4 sessions (0.997).
- Decode at 32K with 1 session varies more between runs in v0.26.0 on the code prompt. The medians are within 3%.
Conversation cache
- A prompt that you send again restores from the cache in more cases. This includes a prompt that the cache holds
in full. - A cache miss has a stated cause. The
crucible_turnline of the server log now hasmiss_reason. On a miss it also
hasclosest_shared_tokens,closest_age_sandclosest_outcome. - The slot, host and disk tiers count their own lookups and hits (
cache_slot_*,cache_host_*andcache_disk_*
on each turn line). Each host lookup writes onecache_lookup tier=hostline. - The newest state of a conversation stays in the cache.
- A slot can copy the state of another slot on the device when the host cache has a miss.
- The disk tier (
--state-dir) stores equal blocks once. It reads a state back during a run. A state that the host
cache refuses goes straight to disk. - The disk files have a new format. At start, the server removes the state files of v0.22 from the state folder.
The cache starts empty.
Server
GET /healthanswers{"status":"ok"}when the engine runs. Use it as a readiness check, for example in llama-swap.
This is already in v0.22.1.- The server accepts any
Hostheader by default. SetCRUCIBLE_ALLOWED_HOSTSor--allowed-hostfor a strict list.
This is already in v0.22.1. GET /v1/crucible/versiongives the version and the git commit.GET /v1/modelshascrucible_version.- The server flags
--temperature,--top-k,--top-p,--min-pand--default-max-tokens N|maxset defaults for
requests that omit those fields. A field in the request always wins. Without a flag, nothing changes.
A request for log-probabilities stays at temperature 0. This is already in v0.22.2. - During a speculative turn, the server writes a
crucible_decodeline with draft counters. The line holds counts
only. It never holds token IDs or text. - A completion with
max_tokens1, temperature 0 and no stream acceptstop_logprobs(0 to 20). - With
--serial, the engine stops a thinking turn at the point where the server closes the think block (the
thinking budget, or the end of the answer room), not atmax_tokens. Before, the engine decoded on tomax_tokens
after the close, and the answer waited behind it. In our GPU test of 2026-10-23, the wait after the forced close fell
from 12.0 s to 0.3 s, and the engine made no extra tokens.
Build
- The engine device code compiles in parallel parts (64 by default). An edit rebuilds only the parts that it changes.
make SPLIT=0uses plainhipcc.make isa-checkshows that the ISA is equal. - The number of parallel compiles follows
make -j. The default is 1.
Updates
- The file
VERSIONand the fileCHANGELOG.mdgive the release number and the notes. build/engine/crucible --versionand the server start line show the version.- The server asks GitHub once a day if a newer release exists, and prints a notice. It changes nothing. Turn the
check off with--no-update-checkorCRUCIBLE_NO_UPDATE_CHECK=1. make updatemoves your clone to the newest release, builds it, and runs a self-test. If a step fails, it restores
the previous release. If a kill or a power loss stops an update, the nextmake updateormake rollbackrestores the
previous release first.make rollbackundoes the lastmake update(once, with no local changes). It then holds the
release that you left:make update, and the timer, skip it until a newer release exists.- Automatic updates are off. "Updates" in the README shows how to turn on the timer.
How to update from v0.22
- Stop the server.
- Run
git fetch --tagsandgit checkout v0.26.0in your clone, thenmake. - Start the server. For later releases, run
make update.
The cache starts empty after the update. Your pack files and checkpoints do not change.
Needs your action, if you come from v0.22.3 or earlier: a thinking request with max_tokens below 1280 and no
thinking budget now gets no thinking. --min-thinking-tokens 0 restores the old behavior. If you use v0.22.4, you
have this change already.