Skip to content

v0.10.12

Choose a tag to compare

@github-actions github-actions released this 12 Nov 18:05
· 3 commits to master since this release
v0.10.12
dafd15c

Highlights

Plain-language summary of provider 0.10.12. The full technical changelog, the pull requests and the build's
verification details are on this page too.

Two-Mac pairs

  • Setting up a two-Mac pair is simpler, and a stopped pair starts again in its last mode.
  • A pair only takes requests whose first word it can deliver in time.
  • If one Mac of a pair drops out, the next request gets a clear error right away instead of hanging.
  • A pair Mac that runs short of memory ends only that request; the pair keeps serving.
  • A two-Mac pair that serves only you now restarts into the new version on its own after an update.
  • Gemma 4 pipeline pairs serve the model's full 262K context.
  • Gemma 4 tensor-parallel pairs serve 2 requests at once where memory allows; GLM-5.3 pairs serve as many as both Macs' memory holds.

Speed and stability

  • Drafting models (Qwen 3.5 9B, Nemotron Lightning, Qwen 3.8) no longer run out of GPU resources under load.
  • Long busy stretches no longer slow a Mac down (the 9B keeps 65 tokens/s after 15 busy minutes, where 0.10.11 fell to 9).
  • Faster prompt reading and generation for Gemma 4; a faster first word for Qwen 3.5.
  • Saving a prompt's cache no longer holds up the answer.

Models

  • Bonsai 2 27B reads prompts about 6.5x faster on a single Mac (back to its pre-0.10.5 speed).
  • Bonsai 2 27B generates at full speed again when two requests share a Mac.
  • Gemma 4 26B (QAT) drafts with exact acceptance, so it falls into fewer reasoning loops; Gemma 4 also reasons with tool_choice: none and accepts deadline text while reading images.
  • Qwen 3.8 drafts with exact acceptance on a single Mac; the Qwen 3.5 9B drafts and caches again; Qwen 3.5/3.6 cache hits are back.
  • Nemotron Lightning uses plain decoding.
  • Nemotron Super answers with JSON, tools and its settings alongside images and video, takes WebP, GIF and BMP images, and its prompt cache gives exactly the same answers.
  • Nemotron Super: an image that arrives while the Mac is still answering another image waits up to 3 seconds for it, when its deadline allows, instead of bouncing to another Mac.
  • GLM-5.3 answers in JSON with an image or video on one Mac, and an idle GLM Mac no longer looks full.
  • MiMo: a slow step no longer disables MiMo for good, and a stalled MiMo engine stops taking requests.
  • All models: answers that repeat themselves word for word stop early; tool_choice: auto returns tool calls as generated; nullable tool parameters accept null.

Requests and refusals

  • A Mac tells the coordinator why it refused a request.
  • Requests sent together are no longer wrongly refused as "memory" on a warm Mac.
  • A Mac that can't hold an image or video right now says so clearly (media_memory_unavailable), and that refusal no longer makes the Mac look full.
  • A request no longer waits forever behind the Mac's idle speed test, and stopping no longer waits for requests the coordinator abandoned.
  • A Mac whose old, slow speed reading keeps making it refuse re-checks its speed with one short idle test (at most once per model every 2 minutes; never on battery; DARKBLOOM_TTFT_REFUSAL_PROBE=off turns it off).
  • Models you selected reload automatically.

The Darkbloom app and command line

  • darkbloom start serves Darkbloom and your local API without asking; darkbloom local serves only you.
  • darkbloom status shows answers in progress, which early-ended requests are paid, and whether fan control is on; status, doctor and verify keep an older denial visible.
  • darkbloom status also lists answers in progress on a Mac that serves only you.
  • The models catalog and recommendations name the models this Mac could add, based on Macs with the same chip.
  • A restart after an update no longer rolls the update back, and the watchdog restarts a provider job that macOS lost.
  • Opt-in: answer before the cache save finishes.

Provider v0.10.12 (Swift CLI)

Source: dafd15c
Build and SDK qualification: https://github.com/Layr-Labs/d-inference/actions/runs/43272471348
Signing: https://github.com/Layr-Labs/d-inference/actions/runs/43272471348
Binary SHA-256: 3c0c5ff016ddc62decb0edeb127b4e4ab810e8563dd55d45ba6facc627be6472
CodeDirectory SHA-256: 5ce1bb36241604b729a213c1508cf7a714c3873d7046a9d73960ae61bdb15683
Bundle SHA-256: 0a353e90d5bfdfd854dbc8ec9a11ad1d38f84960872a56fcea8ffccdb0485dfb
Metallib SHA-256: 5b97893cc20e6e9651edf3284d970488af9a9bdd64820231e95fee760eec70ac
Signed and notarized: yes
macOS deployment target: 14.0 (build, tests and signed runtime smoke: macOS 27)

Install

curl -fsSL https://api.darkbloom.dev/install.sh | bash

What's new in v0.10.12

Upgrade and rollout notes

  • 0.10.12 makes two-Mac pairs sturdier and wider: a pair keeps serving when one of its Macs runs short of memory, a Gemma 4 pair's pipeline serves the model's whole context, and Gemma 4 and GLM-5.3-Flash pairs in tensor parallel serve more than one request at once where both Macs' memory holds them. The toolchain (Xcode 27.2, macOS 27.2 SDK), libs/mlx and libs/mlx-swift are 0.10.11's, so the GPU kernels (mlx.metallib) are unchanged; libs/mlx-swift-lm (Swift model code) moves to ad8a5860e5.
  • Behaviour changes to read first:
    • darkbloom start no longer asks how this Mac serves. On a Mac with no saved choice it serves the Darkbloom network and this Mac's local API (Both), and Darkbloom keeps serving while you use the Mac; it saves that choice ([local] mode = "both", me_first = false). Saved choices are never rewritten, so a Mac set to "Both, me first" keeps pausing. darkbloom local serves only you. Scripts and the desktop app's Start without its local API take the same default where nothing is saved (#2008).
    • darkbloom cluster is simpler: it no longer asks how the pair serves (always through Darkbloom, with the leading Mac's local API beside it, never pausing for new setups), picks tensor parallel or pipeline by itself for a new model (a pair's own model keeps its last mode, also when a stopped pair starts again; #2013), ends on one line ("✓ Serving on both Macs", details in darkbloom status), and on a terminal waits for "Press Enter to exit." darkbloom cluster local still asks, for a pair only you use (#2007).
    • Bonsai 2 27B on a single Mac reads prompts batched again, fast (about 620 tokens per second on an M5 Max instead of about 100; 0.10.5 to 0.10.11 ran every prompt row through the one-row kernel). Two requests decoding together on a single Mac run per-row again, at full speed (#2028); prompts and three or more requests stay batched, and an answer can still depend on the requests beside it, as in 0.10.11. Pairs keep the per-row exact arithmetic, so both Macs compute the same bytes (#2022).
    • Qwen3.5-9B has no default presence penalty and drafts with exact MTP acceptance: its requests speculate and cache their prompts again; in the 0.10.12 gate it ran away on none of 31 draws under exact acceptance (13 of 31 under typical). The 35B Qwen builds keep the card's 1.5. A request's own presence_penalty still wins (#1961, back in with the SDK's shared-event fix, #2021).
    • A pair keeps serving when one Mac runs short of memory partway through a request, in every pair mode (tensor parallel and pipeline; Qwen 3.8 27B, Gemma 4, GLM-5.3-Flash, MiMo V2.6, GPT-OSS). That request alone ends, on both Macs, with one Mac's memory 503 (request_exceeds_node_budget); the next request is served. Before, the pair's worker exited and the pair was down for about a minute (0.10.11's known issue D6). Proven on the M5 Ultra pair with the real memory check.
    • A Gemma 4 pair's pipeline serves the model's whole context again: prompts up to 262,143 tokens and answers up to 32,768, as in tensor parallel and on one Mac (0.10.11 capped it at 65,536). Each Mac's memory check still decides, request by request, whether a long one fits now.
    • More requests at once on a pair: a Gemma 4 26B (QAT 4-bit) pair in tensor parallel serves 2 requests at once, and a GLM-5.3-Flash pair in tensor parallel the widest of 8, 6, 4 or 2 that both Macs hold beside the model (it said 8 on any two Macs before). Each Mac reserves that width when the model loads; if the memory isn't there at that moment, a GLM pair loads with fewer and takes more requests as memory allows.
    • A pair takes the requests whose first token it can deliver in time, and only those. Once a pair had measured how fast it reads prompts, 0.10.11 could turn away every Darkbloom request as "deadline unreachable". It now estimates each request's first token from its own measured speed and the prompts it is still reading.
    • Typed refusals: a pair's API that is starting or stopping answers every request, the first included, with a 503 pair_api_unavailable and Retry-After: 10; an image or video sent to a model that doesn't take it is refused with unsupported_media; a video or audio decode fault is a retryable 503 with the reason media_memory_unavailable, which no longer marks the Mac's token budget as full.
    • Updates and the watchdog: a darkbloom restart (or stop then start) you run in the first 10 minutes after an update no longer counts as a failed start, so it can't roll the update back; a crash, a hang or a reboot still counts. The watchdog brings back a provider job that launchd lost while it was enabled, and says in ~/.darkbloom/watchdog.log when something restarts the provider so often that a new release is never promoted.
    • Selected models come back by themselves: a model you selected that didn't fit at start, or that an Autopilot operation unloaded, is retried when memory frees, never by evicting another selected model. Ten minutes after it began waiting, a model outside your selection that keeps it out only by serving Darkbloom requests finishes its requests and is unloaded. darkbloom stop, restart and start no longer need --force when a model's unload never finishes.
  • A GLM-5.3-Flash Mac no longer reads as full while it serves nothing: its heartbeat reports only the requests it holds, not the memory its engine keeps for reuse (which grew to 90–99 % and turned most GLM requests away from idle Macs), and a request waiting behind another's prompt counts once (#1964).
  • Nemotron Super with images and video serves JSON output (json_object, json_schema), tools and tool_choice, as with text, and takes WebP, GIF and BMP under any image label; other formats are refused as unsupported_media (#1922, #1962). The coordinator sends media with JSON or tools only to Super Macs on 0.10.12.
  • No more stuck MiMo requests behind the idle speed check: a request that arrives during the check stops waiting when cancelled, and after 30 s is refused as busy so Darkbloom sends it elsewhere; while the check is overdue the model reads crashed and gets no new requests (#1966). darkbloom stop, restart and update no longer wait out requests the coordinator already cancelled (at most 30 s after the cancel) (#1968).
  • A MiMo slot whose engine stops stepping stops taking requests (#1967): an engine whose native work failed reads crashed at once, and a native MiMo slot whose queue holds work with nothing running reads crashed after 30 s, so Darkbloom routes around it and the slot is retired.
  • Gemma 4 26B QAT drafts with exact MTP acceptance by default: each committed token is the model's own draw, so its outputs follow the model's distribution (exact in distribution, not bit-identical to decoding without MTP), at a speed cost of about 0–4 %. Only the catalog gemma-4-26b-qat-4bit changes; every other model keeps its rule, and mtp_acceptance_by_model with "gemma-4-26b-qat-4bit" = "typical" restores the old one (#1974).
  • An answer that repeats itself word for word stops early, as length: after about 2,000 repeated tokens instead of running to its output limit (up to 131,072 tokens), so the Mac is free sooner; only exact repetition counts, never in an answer's first 4,096 tokens, and tool calls and JSON answers are never cut. On by default, Nemotron Super's text requests included; DARKBLOOM_LOOP_GUARD=0 turns it off (#2005, #2020; window 16).
  • GLM-5.3-Flash answers a JSON response_format with an image or video on one Mac, as its text JSON mode does; before, every such request was a 400 (#2009). A pair still refuses it.
  • Qwen 3.5/3.6 requests that can't speculate cache their prompts again (a penalty, a stop string, logprobs, logit_bias or a token constraint): they record the MTP head's prompt history and save the same prefix checkpoints a speculating request saves, so the 35B builds, which keep the card's penalty, cache again (#1965 with mlx-swift-lm #398; window 8b: exact, memory accounted and capped).
  • Nemotron 3.5 Lightning serves plain decode under mtp_mode = "auto": its MTP head sped up only a lone request and halved decode beside a second one; plain decode keeps production's throughput (#1969, window 7).
  • A Mac whose old, slow speed reading keeps making it refuse re-checks its speed: after three refusals in a row that only the stale reading explains, it runs one short idle test of its own (at most once per model every 2 minutes, never on battery) and prices the next request from the new reading, instead of refusing for up to 30 minutes. No customer request is risked: none is admitted without a time estimate. DARKBLOOM_TTFT_REFUSAL_PROBE=0 turns it off (#2024, #1956 reworked).
  • darkbloom status lists the answers in progress on a Mac that serves only you (darkbloom start --local), as on a Mac that serves Darkbloom; a pair that serves only on its local API still lists none (#2030).
  • A two-Mac pair that serves only you restarts into the new version on its own after an update: the first part started from the newer release restarts the pair's cluster service once, before it takes a request (before, the service kept the earlier release until the next login) (#2033).
  • Nemotron Super: an image that arrives as the previous image finishes waits for its cleanup (up to 1 s) instead of being refused with 429 queue_full and sent to another Mac. Super's slot serves up to 8 requests at once; only images go one at a time. Memory use is unchanged (#2029).
  • Nemotron Super: an image that arrives while another image's answer is streaming waits for the slot (up to 3 s, and never past the time its first token needs), instead of being refused and sent to another Mac. Only one request waits at a time and it reserves no memory, so two images are never in memory at once; text never waits (#2035).
  • Drafting models no longer use up Metal shared events while several requests decode: each decode step of two or more requests together left one event alive, and after roughly 15–50 minutes of such a stretch the provider could abort (Failed to create Metal shared event). This hit Qwen 3.8 by default, the Qwen 3.5/3.6 builds with presence_penalty: 0, Lightning and Qwen4 Flash-Next. Tokens and drafts are unchanged (mlx-swift-lm #404, #2021).
  • Paged decode stays fast on a busy Mac: freed KV pages no longer scatter new requests' pages over many segments, so decode keeps one attention pass per layer instead of slowing to 0.30–0.40× its fresh speed a few minutes into a busy run (the 9B in production); outputs are unchanged (mlx-swift-lm #403, #2021).
  • A Gemma 4 pipeline checks its whole number of requests at load: a setup that saves several requests at once (2, 4 or 8) serves that many only if each Mac holds them all beside the model, and refuses the load otherwise; one without a saved number serves one at a time (it served four unchecked). Pipelines set up by darkbloom cluster keep one chunk of lookahead and one request at a time (#1937 via #2015).
  • Small local requests sent together are no longer refused as "memory" on a warm Mac: at the fallback width (heat or Low Power Mode) a streamed request waiting for its first token counted twice, so the third or fourth of four got a 503 with most memory free (#2018).
  • A media request this Mac's memory can't hold now is refused as media_memory_unavailable (it was capacity_busy), so Darkbloom sends it to another Mac without counting this Mac as full (#2019).
  • darkbloom status shows the answers in progress: model, running time, output tokens, current speed and a short request hash (never the prompt or answer); --json has the same list. A Mac writing long answers no longer looks hung (#2017).
  • A refused request tells the coordinator why: the inference_error frame carries refusal_reason and a fixed refusal_message (no request text; the requester's error is unchanged), and Qwen tool conversations name the check that refused them (#2016).
  • A stream that stores its prompt's checkpoint no longer waits for row-by-row coding: the SSD checkpoint export reads rows in batches and codes them on up to four cores, so the end of a stream (and its terminal) comes much sooner; the checkpoint bytes are unchanged (#1971, window 7).
  • Gemma 4's MTP assistant upgrade no longer races the serving engine: attaching the assistant to a model that already serves reuses the KV types observed at load instead of running a forward, which could crash the provider ("Attempting to eval an array without a primitive") when a request arrived during the upgrade (#1985, window 8b).
  • A long MiMo prefill no longer trips the engine watchdog: a native MiMo step may run up to 120 s (the engine's prefill progress lease) before the watchdog seals the engine, instead of 30 s, so slower Macs (M3 Ultras take three to four times an M5 Ultra's 15.6 s at 170k tokens) finish long prompts; a step that never returns is still caught (#1989; window 8c: sealed at the limit under a 5 s override, a 170k prefill with no crash, identical output).
  • darkbloom status lists early-ended requests by cause, paid or unpaid (#1980). The causes are exact once the coordinator sends the reason, which it does from the coordinator update of 11-11. One known gap until platform #792 ships: an owner's own request stopped mid-answer shows as "paid".
  • Nemotron Super's prompt cache restores exactly: it saves native-precision prefix checkpoints, so an answer from a cached prompt is the answer the same prompt gets without the cache; checkpoint files are about 15 % larger, and older Super checkpoints are skipped and age out within 30 minutes (#2002; window 11: 3/3 prompts exact).
  • Qwen 3.8 27B drafts with exact MTP acceptance on single Macs (#1999): its outputs follow the model's distribution (exact in distribution, not bit-identical to decoding without MTP). It costs about 4–5 % decode speed while a Mac serves one Qwen 3.8 request at a time and nothing with several; pairs are unchanged (they were already exact). mtp_acceptance_by_model with "EigenLabs/Qwen3.8-27B-4bit-mtp" = "typical" restores the old rule (window 8d: acceptance 0.73 vs 0.89–0.90, text matches MTP-off, no loops).
  • Gemma 4 no longer turns away text requests while an image request is being answered; it still waits while the image is read in (mlx-swift-lm #402; window 12: text beside a decoding image admitted 16/16 within its deadline).
  • Tool calls under tool_choice: auto come back as the model wrote them, as with OpenAI, instead of failing the request; strict, required and named tools are still checked. A refusal's error frame names the check that failed (tool_check) (#2003; window 13).
  • Faster first tokens and tool calls (the SDK, libs/mlx-swift-lm 093563be11): a Gemma 4 prompt's cold first token is about 7 % sooner on M5 Macs, with exactly the same output (row-mapped NAX prefill, on by default; DARKBLOOM_GEMMA4_NAX_ROW_MAP=0 turns it off); a Qwen 3.5-family first token is 10–12 % sooner under MTP for requests that send presence_penalty: 0; a tool parameter declared nullable now takes null. An opt-in early response before the SSD checkpoint write finishes ships off by default (DARKBLOOM_PREFIX_CACHE_SSD_TERMINAL_BEFORE_PUBLICATION=1).
  • Smaller changes: darkbloom models catalog ends by naming up to three models this Mac could select but hasn't downloaded, with their size (print only; --json unchanged; #1957); a pair's planner can't run out of stack (planning runs on its own 16 MiB thread); a pair's leading Mac backs off (30 s up to 10 min) while the other Mac is away and starts at once when it answers; darkbloom status says whether fan control is on, and keeps an older authorization denial visible (as do doctor and verify); truer words on a paired Mac after cluster leave and in doctor, and the split question names each mode's context; Gemma 4 reasons with tool_choice none; the desktop app recommends models from what Macs with the same chip earn.
  • Known issues (none blocks this release):
    • On a Mac of 48 GB or less, a very long Gemma 4 pipeline request can get the memory 503 partway through its prompt (past about 190,000 tokens at 48 GB, about 85,000 at 36 GB), for that request alone.
    • A GLM-5.3-Flash tensor-parallel request that ends at its first token for memory can leave its prompt's stored prefix on the other Mac until it is evicted; it is never served stale.
    • After darkbloom update --rollback to 0.10.11, a Gemma 4 or GLM pair set up on 0.10.12 must run darkbloom cluster again: 0.10.11 can't read the saved width.
  • Pair owners: both Macs of a pair must run the same version; a pair on 0.10.11 updates itself at its slot, or at once with darkbloom update on each Mac. darkbloom update --rollback puts back 0.10.11.
  • Pairs still on 0.10.10 stop serving through Darkbloom once 0.10.12 registers, until they update (0.10.12 and 0.10.11 hold 24 pair approvals each, the coordinator keeps 64).
  • The registration keeps 0.10.11's fields; its cluster runtime facts change where the modes do: Gemma 4's pipeline qualification serves the whole context again, and the tensor-parallel widths follow each Mac's memory. Deploy the coordinator that ships with 0.10.12 first.
  • The 68 entries below are the pending fragments rendered verbatim, except where the release notes reword one; CI-only entries change no provider behaviour.

status, doctor and verify keep an older denial visible

  • A provider's last coordinator authorization was shown only while it was at most 10 seconds old. An App Attest lease already stayed visible until it expires, but an older none (a denial) or legacy report vanished from darkbloom status, doctor and verify, so a Mac the coordinator had denied could look healthy. Such a report now stays visible as dated history: Last reported authorization: received 10m ago; not a fresh dispatch grant. … in status, and a last reported authorization line in doctor and verify, INFO for legacy and WARN for a denial.
  • History never stands in for a current grant or the public proofs: it offers no MDM removal or migration, and doctor and verify still run the trust and public attestation checks after it. darkbloom verify now fails on a WARN or FAIL line, not on an INFO line.

Bonsai 2 27B processes prompts at full speed again on a single Mac

  • Since provider 0.10.5, a single Mac ran every prompt row of Bonsai 2 27B through the one-row kernel. That kept each row's arithmetic identical to decode, but made prompt processing 2.3 to 6 times slower than in 0.10.4: about 100 tokens per second on an M5 Max instead of 620, and about 76 to 96 instead of 220 on M1 to M4 Macs. Image and tool prompts of 1,000 to 2,000 tokens or more then missed their first-token deadline, and OpenRouter saw 429s.
  • Now a single Mac processes a Bonsai 2 27B prompt with one batched matrix multiply per projection again, as in 0.10.4 (TF32 on M5, FP32 elsewhere). Prompt rows are no longer bit-identical to the decode arithmetic, as before 0.10.5. A request decoding alone is unchanged. When several Bonsai 2 27B requests decode at once on one Mac, a request's exact output can depend on the requests decoding beside it, as in 0.10.11; two requests decoding together run per-row at full speed (see the next entry).
  • Pairs keep the per-row arithmetic for prompt and decode rows, because both Macs must compute the same bytes. Their arithmetic receipt (bonsai-fp32-row-arithmetic-v2) is unchanged.
  • libs/mlx-swift moves to 1b3520fe94 (Layr-Labs/mlx-swift#70, on the fork's release/provider-0.10.12 = the previous pin c633d221d6). Only Swift sources change there, so mlx.metallib is unchanged.

Two Bonsai 2 27B requests decoding together run at full speed again on a single Mac

  • With prompts batched again (#2022), a single Mac also ran each decode step of several Bonsai 2 27B requests as one batched matrix multiply. At two requests on an M5 Ultra that was 9.5 % slower than 0.10.11's one-row kernel (71.1 against 78.6 tokens per second, steady state).
  • Now two requests decoding together on a single Mac are computed row by row again, as 0.10.11 computed them on every Mac, and run at full speed again (78.6 tokens per second on the M5 Ultra). Prompts and decode steps of three or more requests stay batched: batched prompts are 6 to 7 times faster than 0.10.11 at 1K to 8K tokens, and from three requests batched decode is as fast or faster (116.0 against 107.3 tokens per second at four).
  • A request's exact output can still depend on the requests decoding beside it, as in 0.10.11. Pairs are unchanged.
  • libs/mlx-swift moves to b1c94d29c5, the merge of Layr-Labs/mlx-swift#71 into the fork's release/provider-0.10.12. Only Swift sources change there, so mlx.metallib is unchanged.

A stream that stores its prompt's checkpoint no longer waits for row-by-row coding

  • When a request's prompt checkpoint is written to the SSD prefix tier at a quantized checkpoint precision (balanced, the default), the response's terminal waits until the checkpoint is written. Writing it read the checkpoint's attention rows one row at a time and coded each on one CPU core. On GLM-5.3-Flash every read also rechecked the native binding, so on an M5 Ultra the stream stayed open 63 s after a 200k-token prompt's last token and 17 s after a 57k-token one; in production on 0.10.11, Gemma 4 26B requests with 4–32k-token prompts waited a median 1.2–1.6 s, GPT-OSS 20B 8–32k 1.7 s and GLM-5.3-Flash ≥ 128k 51 s, and the Mac kept the request's memory reserved meanwhile.
  • The export now reads a head's rows in batches of up to 256 KiB, codes each batch on up to four cores (a quarter of the Mac's active cores) and rotates eight lanes at a time. The checkpoint's bytes are unchanged, so restores and their answers are exactly as before. CPU coding cost per 512-wide row falls from about 3 µs to about 0.6 µs on an M5 Ultra.

CI: a Darkbloom Sandbox without the Darkbloom cache uses no Actions cache

  • On a Darkbloom Sandbox that the plane gave no Darkbloom cache, actions/cache read and wrote GitHub's own Actions cache through the gateway. A master push on 2026-11-05 (job 127018716621) saved .git, a 1.69 GiB Swift build, the SwiftPM repositories and a Metal library to the refs/heads/master scope; dev pushes on 2026-11-07 and 2026-11-09 saved .git to refs/heads/dev.
  • Each routed job that uses a cache now first runs Choose this job's Actions cache. On a runner named darkbloom-* without DARKBLOOM_CACHE_URL it sets CI_ACTIONS_CACHE=off, and every actions/cache step and actions/setup-node cache then skips. Darkbloom jobs with the plane's cache, Blacksmith and GitHub-hosted jobs, and the Blacksmith-only release workflows keep their caches.
  • A restored .git is now discarded, before any git command reads it, if it is not a directory, holds a symbolic link or any hook but git's samples (also in .git/modules), or has a config that sets core.fsmonitor or core.hooksPath or includes another file. This holds on every runner.

CI: every Cluster GPU Tests shard asks Darkbloom first

  • Each of the 3 Cluster GPU Tests shards now asks the Darkbloom plane for a sandbox on a lane of its own (test-cluster-gpu-1, test-cluster-gpu-2, test-cluster-gpu-3; 4x24 while sizes are on). Each falls back alone to blacksmith-12vcpu-macos-27 when the plane does not grant it. A lane routes only when the approval's lanes names it; on 2026-11-10 it named test-cluster-gpu-1 and test-cluster-gpu-2 but not test-cluster-gpu-3.
  • Shard 2 no longer needs 42 GiB. The paged verify/prefill attention bench (paged-verify-bench) bounds MLX's freed-buffer cache to 1 GiB, as a ready Qwen tensor-parallel rank does. Before, the cache kept every prompt chunk's attention temporaries, and the bench's process reached 34.26 GiB. The bench also prints its peak active memory and cache size for each run.
  • A full CI run now asks for 15 macOS jobs instead of 14. The GPU lanes come last in the lane table, so the DARKBLOOM_MACOS_LANES cap drops them first.

CI: the MiMo audio codec download resumes instead of starting over

  • The provider's last unit-test shard downloads the 1.87 GB pinned MiMo audio codec when the runner's Darkbloom cache has no copy, which happens on some runners and not others. curl's own retries restarted every attempt from the first byte, so on a link of about 2 MB/s no 900 s attempt finished: CI run 43132089921 got 1.60 and 1.79 GB and failed after 30 minutes, and other pull requests' runs hung in the same step until cancelled.
  • scripts/prepare-mimo-audio-fixtures.py now makes up to eight attempts per file, each resuming the partial file (--continue-at -), with a growing pause between them. Each attempt ends after at most 900 s and never past 45 minutes from the first, so the step gives up within 45 minutes. A server that ignores the range request makes the next attempt start over, and a file whose digest is wrong is deleted, so no later run resumes it. The same script prepares the codec in the release build's qualification lane. scripts/test-prepare-mimo-audio-fixtures.py checks this offline.

CI: the cluster qualification checks build with the job's selected Xcode

  • libs/darkbloom-cluster-worker/Tests/QualificationChecks/run.sh now uses the Xcode and SDK the job's toolchain step selected (PROVIDER_SDKROOT), so it no longer fails on a runner whose default developer directory is the command-line tools (SDK "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk" cannot be located). Local runs keep xcrun's own selection. CI only; no provider behaviour changes.

Provider CI: smaller job sizes for the cluster runtime shards

  • The cluster runtime tests that drove the shards' memory now run with MLX's freed-buffer cache bounded: 4 GiB for the GLM-5.3 paged KV, row invariance, depth invariance and window census tests and Qwen's prefix-depth test, 1 GiB for the Gemma 4 paged synthetic stages and the GLM-5.3 prefix SSD restore. MLX's default cache kept every prefill frame's temporaries, which no later request reuses. The bound changes no result, no assertion and no shard's time.
  • Peak used memory: Cluster Runtime Tests shard 2 from 26.51 to 14.64 GiB, shard 3 from 13.21 to 10.71, shard 5 from 15.33 to 11.48 (the largest sample before and after).
  • While DARKBLOOM_JOB_SHAPES is on, the lane of runtime shards 2 and 3 (test-cluster-2) asks for 4x24 instead of 12x32, and the lane of shards 4 and 5 (test-cluster-3) for 4x16 instead of 4x24. The other lanes keep their sizes; docs/developer/build.md (Job sizes) has each job's sampled memory from 20 full dev runs.

CI: the desktop review package asks Darkbloom first

  • The manually dispatched Desktop App job that builds the unsigned macOS review package (unsigned-desktop-review) now asks the Darkbloom plane for a sandbox Mac on lane desktop-package. It falls back to blacksmith-12vcpu-macos-27 when the plane does not grant it.
  • The job is renamed from review-package to desktop-package, so that its lane has the job's name, as the lane of every unsharded routed job does.
  • Pull requests and pushes do not ask for this lane: the job runs only on manual dispatch.

Simpler two-Mac cluster setup

  • The first Mac's last line while it waits is set apart and stressed: "→ Now run darkbloom cluster on the other Mac" (darkbloom cluster local names its own command).
  • darkbloom cluster no longer asks how the pair serves: it always serves through Darkbloom with the leading Mac's local API beside it, and Darkbloom keeps serving while you use that API: once the pair switches, starts or is shown serving, the leading Mac saves [local] mode = "both" and, where no me_first is saved, me_first = false (a run that changes nothing saves nothing; a pair already serving Both with the pause on keeps it until its next start, and the screen says so). To have Darkbloom pause for you again, set [local] me_first = true on the leading Mac and run darkbloom restart. A pair saved as Only me or as the Darkbloom network alone switches when darkbloom cluster runs and says so in one line; darkbloom start, an update or a restart never changes how a pair serves. "Change how the pair serves" is gone from the menu; darkbloom cluster local stays the local-only command.
  • Where a pair can't serve through Darkbloom (no login, no approval), the screen says "To use the pair just for yourself, run darkbloom cluster local." instead of offering "Serve locally only".
  • darkbloom cluster chooses tensor parallel or pipeline by itself for a new model: tensor parallel unless it can't run on these Macs or this pair measured it slower. A pair's own model keeps the mode it last served in, also when a stopped pair starts again. The model list no longer shows modes. darkbloom cluster local still asks, with the same choice tagged Recommended and preselected (the pair's own model preselects the mode it last served in).
  • A start ends on one standout line, "✓ Serving on both Macs" (locally: "…, on your local API at http://localhost:"), and "darkbloom status for details", the same on both Macs. The pair's card (log, login, commands) is in darkbloom status.
  • On an interactive terminal, both Macs end with "Press Enter to exit." (q and Esc too); NO_COLOR, TERM=dumb, pipes, CI and runs without a terminal never wait, and the screen's lock is released first, so another darkbloom cluster or start isn't refused. The pair keeps serving.

A two-Mac pair takes the requests whose first token it can deliver in time, and only those

  • Once a pair had measured how fast it reads a prompt, it turned away every Darkbloom request as "deadline unreachable", even one it could serve at once. Darkbloom sends a first-content deadline with every request, and the pair estimated no first token at all.
  • Now the pair estimates each request's first token from its own measured prompt speed, behind the prompts it is still reading for the requests it already holds, since it reads them one after another. It takes a request whose first token comes in time and turns one away at once when it would not, so Darkbloom sends it to another Mac. Before, eight long prompts arriving together were all taken, and the last waited about a minute for its first token.

A pair's first answer after the other Mac's part stops is its typed 503, at once

  • When the part of the pair on the Mac that runs the pair's API stops or starts again (a stop, a restart, an update, a
    crash) while that Mac stays on the cable, every request, the first one included, now gets a 503 with code
    pair_api_unavailable and Retry-After: 10 at once, on that Mac and through the other Mac's port alike. Before, the
    first ones got {"error":{"type":"invalid_request_error","message":"Provider capacity is temporarily unavailable."}}
    with no code, once after 19 seconds. The pair's API now answers "The pair's API takes no requests right now: the pair
    is starting or stopping. Requests resume when it serves, …" while its pair doesn't serve, never holding a request until
    it does, and a request made during the pair's warm-up request is refused at once instead of waiting for it.
  • Both 503s a pair's other Mac answers (pair_mac_unreachable and pair_api_unavailable), and the pair's API's own,
    now end the same way: "…, or run darkbloom stop, then darkbloom start --solo, to serve this Mac alone (darkbloom start --solo by itself is refused while this Mac's part of the pair holds its serving lock: up to 15 minutes after
    the other Mac last answered over the cable)."
  • Through the other Mac's port, a key it can't judge (another key in the local key format, such as the leading Mac's
    own, which works through it while that Mac is there) gets the same 503 without the other Mac's name, instead of a
    401. A missing key, one without Bearer, or one that isn't a local key still gets the 401.
  • While the pair's API on the leading Mac starts again, the other Mac's port stays up and answers those 503s, instead
    of refusing connections until that API is back, and its darkbloom status, darkbloom local and darkbloom doctor
    say "Local API: http://127.0.0.1:/v1 on this Mac answers 503 for now." with the 503's words, instead of "not
    listening on this Mac yet".
  • A local request waiting for one of the pair's lanes when a stop or an update begins gets the same 503, instead of a
    429. A request without a key that reaches the pair's API while it stops gets the API's 401, instead of a 503.

A pair's leading Mac starts its part less often while the other Mac is away

  • While the other Mac of a pair doesn't answer (asleep, off, its cable out), the leading Mac keeps its part ready for 10 minutes so the pair re-forms when that Mac is back. Its part can't form the pair meanwhile, and each start reads its share of the model and registers with the coordinator again. A part that ended was started again every 30 seconds (every 48 seconds for a MiMo leader whose other Mac slept). It now waits twice as long as the last time, from 30 seconds up to 10 minutes (30 seconds, then 1, 2 and 4 minutes), and starts at once when the other Mac answers. While the other Mac answers, a part that keeps ending starts again as before (6, 12, 24, then every 30 seconds).
  • darkbloom cluster status on the leading Mac says when, under the line that it waits for the other Mac: This Mac's part starts again in 4 min, or as soon as the other Mac answers: while the other Mac is away, each start waits twice as long as the last, up to 10 min.

A pair's cluster service runs the installed release after darkbloom update

  • A pair that serves only on its local API (or a Mac whose cluster service
    serves alone) kept its cluster service on the earlier release after
    darkbloom update on both Macs: the service's supervisor does not drain
    such a pair for an update, so it never restarted into the installed
    release, and darkbloom restart restarted only the pair's parts (0.10.12
    W1). The parts ran the installed release; the service's own changes, and
    the version it reports to the other Mac, waited for the next login.
  • Now the first part the service starts from a newer installed release
    restarts the service's launchd job once, before it takes a request: the
    pair stops on that Mac and starts again on the installed release, as after
    a login. Only for the build's own job, only when its supervisor runs an
    older release than the part, and once per release in 6 hours; the part's
    log says so (cluster part: this Mac's cluster service still runs …). It
    runs in the parts of this release, so it also covers the update to it.

A stopped pair keeps its last mode on restart

  • darkbloom cluster local, and darkbloom start in a terminal on a pair that serves locally, preselect the mode the pair's own model last served in when its mode question comes up, so Enter starts a stopped pipeline pair as a pipeline again instead of switching it to tensor parallel. The Recommended tag stays on the automatic rule's pick; when the last mode is the recommended one, nothing changes. darkbloom cluster local <its model> starts the pair's own model in that mode too; a new model still gets the automatic rule.

Three deadline refusals on a stale prefill estimate re-measure the Mac

  • A deadline_unreachable refusal admits nothing, so it measures nothing. An idle Mac with no calibration anchor, whose prefill measurement was slow and is now older than 120 seconds, could go on refusing from it for up to 30 minutes.
  • After three refusals in a row that the stale estimate explains, the provider re-runs its idle calibration probe once: one uncached 512-token request of its own, alone, only while the Mac is idle, at nominal posture and on AC power. Any customer request preempts it. Its measurement replaces the estimate and the anchor, and the next request is priced from it. A loaded model probes at most once every 120 seconds.
  • It never runs on battery, with or without Low Power Mode; a full streak waits for AC power. The startup calibration is unchanged.
  • Only refusals the stale estimate explains count: the Mac was idle, the projection priced nothing but that request's prompt, the estimate was older than 120 seconds with no fresher measurement for that prompt size, it was below the calibration anchor if there is one, and the request still had at least 5 seconds left. Refusals at a fresh rate, late retries, refusals beside other work or for memory or capacity, reviewed calibrations, native-media requests and native block engines (which keep their own recovery probe) don't count.
  • A customer request is never admitted without a time projection because of these refusals, so none is admitted that is already out of time. The provider logs one line when the probe starts (Deadline refusals: <model> re-measures stale prefill evidence with one idle probe …).
  • No wire field changes. DARKBLOOM_TTFT_REFUSAL_PROBE=0 turns the probe off on its own, and DARKBLOOM_TTFT_ADMISSION_FIXES=0 turns it off with the other estimator corrections. Both reach the launchd provider job.

Recommended models from Macs with the same chip

The desktop app's "Recommended for your Mac" now ranks models by what Macs with
the same chip earned per day from each model over the last 24 hours (the typical
Mac first, then the average), when the coordinator publishes those figures in
/v1/stats model_earnings[].by_chip. Network-wide 7-day totals favor models
that many fast Macs serve, which can pay a slower Mac nothing. Without per-chip
figures, or when they cover no model this Mac can serve, the ranking falls back
to network totals as before.

Stopping no longer waits for requests the coordinator abandoned

  • Before, darkbloom stop, restart and update waited for every accepted request to finish, including requests the coordinator had already cancelled because their client left or their first token was late. A request stuck behind a stalled engine never finished, so the drain timed out after its full deadline and asked for --force.
  • Now a request the coordinator cancelled counts for at most 30 s after the cancel. Its answer has nobody to go to; if its task has not ended by then, the drain stops waiting for it. Memory and eviction still treat it as in flight until it ends, and a native MiMo slot still retires only after its last request.

A Gemma Mac decoding an image request accepts deadline text again

  • A Gemma 4 Mac refused every text request with a first-content deadline (deadline_unreachable, profile reason multimodal_work) while an image or video request was anywhere in its engine, including the image's whole decode, which is about 98 % of its time there. On OpenRouter this was 0.9–1.8 M refused attempts a day on 11-08..11-10, almost all on M5 Macs, and 16–59 K requests a day ended as 429s.
  • The SDK now refuses only while the image request still has prompt work (until it has sampled from its whole prompt). After that its decode is priced like any other decode row, so the text request is admitted when its first token fits the deadline. Image requests themselves, other models' media (all causal) and text-only engines are unchanged.

Gemma reasons with tool_choice none

  • A Gemma 4 request with tool_choice none and reasoning on could not open its thought: the sampler that keeps tool calls out also masked Gemma's reasoning delimiters, <|channel> and <channel|>. Such a request now reasons and answers as it does without tool_choice; before, it could repeat until it hit its output limit. Tool-call markers stay refused, and required or named tool_choice is unchanged.

A Gemma 4 pair's pipeline serves the model's whole context

  • A Gemma 4 26B pair in the pipeline now accepts prompts up to 262,143 tokens and up to 32,768 output tokens within 262,144, as in tensor parallel and on one Mac. Before, its pipeline served the 32,768 + 32,768-token request (65,536 tokens). A pipeline Mac whose memory refuses a running request ends only that request. Each Mac's memory gate still decides, request by request, whether a longer one fits now. With lanes (a setup that prefills serially), the gate decides when the request starts, and a long request can still be refused partway through its prompt when the Mac runs short.

A Gemma 4 pipeline checks its whole number of requests at load, and serves one at a time without one

  • A Gemma 4 pipeline set up by darkbloom cluster still reads each prompt with one chunk of lookahead and serves one
    request at a time. Serving several at once needs it to read each prompt one chunk at a time, and on two M5 Ultras a
    long prompt alone then started its answer about 1.9 times later, so the setup keeps the lookahead.
  • A pipeline whose setup saves one-chunk-at-a-time reading and a number of requests at once (2, 4 or 8) serves that
    many. Each Mac now checks at load that the whole number fits beside the model, each request charged its whole state
    at the standard request (32,768 prompt plus 32,768 output tokens), and refuses the load if it does not: "Resident
    load refused: N requests at once, each of C context tokens, do not fit on rank R now". Before, the load checked one
    request. A Qwen 3.5 or Gemma 4 pipeline widened automatically on a local pair is checked the same way.
  • A pipeline set up to read prompts a chunk at a time without a saved number now serves one request at a time. Before,
    it served four, which nothing had checked its memory for.
  • Each request still gets exactly the tokens it gets alone. A request cancelled or refused for memory ends alone, and
    the others keep running.

Gemma 4 tensor parallel serves 2 requests at once where both Macs' memory holds them

  • A Gemma 4 26B pair in tensor parallel served one request at a time. Through Darkbloom, every request beyond the
    one it served was turned away as "at capacity".
  • Now the setup sizes the QAT 4-bit pair the way it sizes Qwen 3.8 27B's tensor parallel: it serves 2 requests at
    once where both Macs can hold them now beside the model. Each request after the first is charged everything one
    request holds at the standard request (32,768 prompt plus 32,768 output tokens), about 2.6 GB on each Mac, and each
    Mac reserves that width when the model loads, the planner counting that reservation while the model is read.
  • Why 2: on two M5 Ultras every request got its own tokens at 2, 4 and 8 at once, but each request was served fastest
    at 2 (77 tok/s each and 95.5 in all, against 48.5 and 96.2 at 4). The 8-bit model still serves one request at a
    time; nothing has qualified its lanes yet.
  • The setup screen and the number the pair reports to Darkbloom are the same width. Where only one request fits,
    the pair saves no width and serves one request at a time, as before.
  • Each request still gets exactly the tokens it gets alone. A request cancelled or refused for memory ends alone,
    and the others keep running.

Gemma 4 compiled feed-forward dispatch

Adopt the native SDK's shape-specialized feed-forward graph for eligible Gemma 4
GPU decode and verification rows. A matched single-request M5 Max endpoint
comparison with 8K cached context measured a further 2.5–3.2% decode-proxy gain
over the prior dispatch change. Parameter updates, eager fallbacks and existing
attention, MTP, admission, prefix-cache and SSD-write behavior are preserved.
Broader concurrency and other-chip gains remain unqualified.

Gemma 4 native dispatch efficiency

  • Adopt the SDK's shared dense GeGLU reuse for eligible Gemma 4 models, preserving numerical expressions, parameter contracts, paged attention, prefix caching and SSD write policy. Gemma 4's decode command-buffer limits are unchanged by this entry.
  • An initial warm 8K, single-request endpoint screen on one M5 Max measured a 3.5–3.9% decode-proxy improvement with matching outputs, together with an earlier scoped command-buffer limit that has since been replaced, so the gain is not attributed to the GeGLU change alone. Broader concurrency, prefill, clustering and other hardware remain unmeasured; memory and admission safeguards are unchanged.

Gemma 4 row-mapped NAX prefill

Adopt the native SDK's shared plain-greedy metadata shortcut and guarded Gemma 4
row-mapped NAX prefill route, enabled by default within its existing hardware,
shape and evaluation guards. Exact DARKBLOOM_GEMMA4_NAX_ROW_MAP=0 restores the
original graph, with the override forwarded when creating a LaunchAgent job.
Same-app M5 Max comparisons in both orders measured 6–7% lower cold HTTP time to first generated
frame for uncached 4K/8K single requests, with exact paired outputs. A longer
warm decode control was neutral. These are not additive speedups or concurrency,
other-chip or clustering claims; attention, MTP, admission, prefix caching and
SSD-write policies are unchanged.

A GLM-5.3-Flash Mac no longer reads as full while it serves nothing

  • A GLM-5.3-Flash Mac's heartbeat now reports as used only the requests it holds, as its own admission charges them. It used to report the memory its engine keeps after every request and reuses for the next, which grew to 90–99 % of the budget, so Darkbloom turned most GLM requests away while the Mac sat idle. A GLM Mac can now take as many requests at once as its memory admits.
  • A GLM request waiting for another's prompt to finish is now counted once. It was counted twice on this Mac's own API, so a 256 GB M5 Ultra took two long requests at once instead of three.

GLM-5.3-Flash answers a JSON format with an image or video on one Mac

  • A GLM-5.3-Flash Mac now serves a JSON response_format (json_object or json_schema) together with image or video input. Before, it refused every such request with HTTP 400 Invalid inference request. (failure code invalid_request) before any media work.
  • It works as GLM's text JSON mode does: the instruction renders in the media prompt, reasoning streams on its own channel, and the answer is withheld until generation ends, then checked and exposed once. An answer that isn't the JSON asked for is a 422 with no content.
  • Still refused with 400: a required or named tool_choice with image or video input, and a JSON format with image or video input on a cluster pair (the pair's media routes don't withhold the answer).
  • Darkbloom's coordinator answers these requests itself, without dispatching, for Macs on earlier provider versions.

GLM-5.3-Flash tensor parallel serves as many requests at once as both Macs' memory holds

  • A GLM-5.3-Flash pair in tensor parallel served up to eight requests at once on any two Macs. Its load checked
    memory for one request only, so a pair with room for fewer still told Darkbloom eight.
  • Now the setup sizes GLM the way it sizes Gemma 4 26B and Qwen 3.8 27B in tensor parallel. It saves the widest of 8,
    6, 4 or 2 requests at once that both Macs can hold now beside the model, and each Mac reserves that width when the
    model loads. Each request after the first is charged what it adds at the standard request (32,768 prompt plus
    32,768 output tokens): about 9.7 GB.
  • The setup screen and the number the pair reports to Darkbloom are the same width. Where only one request fits, the
    pair saves no width and serves one request at a time.
  • When a Mac has less memory free at load time than at setup, the pair still loads: each Mac reserves as many of the
    saved requests as it can hold then, at least one, and takes the others as memory allows. On two M5 Ultras an
    8-request pair whose Mac had other programs using 38.68 GiB did not load at all; it now loads with 6 reserved.
  • A pair set up before this version is planned again by itself after the update, as after every Darkbloom update,
    and saves its width then.
  • Each request still gets exactly the tokens it gets alone.

Developer tooling

  • Add an offline GPT-OSS publisher checkpoint repacker that preserves MXFP4
    experts and BF16 non-expert tensors, with source/output verification and
    comparison receipts. Add frozen-request replay guards for paged attention,
    effective prefix reuse and complete streams. These tools do not change the
    registered model or establish candidate qualification.

A request no longer waits forever behind the idle speed check

  • Before, a request that arrived while the provider ran its idle speed check (MiMo repeats it whenever the Mac is idle) waited for the check to finish, with no time limit, and ignored its coordinator's cancel. If the check stopped making progress, every later request on the Mac was accepted and never started: the coordinator timed each one out, and darkbloom stop waited 600 s for them and gave up.
  • Now a cancelled request stops waiting at once, and any request stops waiting after 30 s and is refused as busy, so the coordinator sends it elsewhere. While the check is overdue the model reads crashed in the heartbeat, so no new requests are routed to it; it serves again as soon as the check finishes.

Nemotron 3.5 Lightning serves plain decode instead of its MTP head

  • Under the default mtp_mode = "auto" Lightning now serves plain decode instead of its embedded MTP head. The head speculates for one request only: beside a second request each step became a slower hidden-returning step, and production decode fell from about 110 to about 54 tok/s (11-09). It also charged about 45 KB of head state per context token, so a Lightning slot's token budget was about an eighth of its KV and many Macs could not admit one request with the default 32,768 output tokens. Plain decode commits a Metal command buffer every 100 operations instead of after almost every MoE layer. A single request at temperature 0 keeps the same output. Explicit mtp_mode = "on" keeps the head.
  • The gain is not universal, on an M3 Ultra: a single request at the artifact's default temperature 1.0 stays about even, text the head predicted very well (counting, repeats, tables) runs up to about 30% slower, and a row decoding beside another request's long prompt runs at about half the head's rate.
  • Sampled output follows the target's own distribution: the head's lossy typical acceptance no longer applies to Lightning.
  • Its complete checkpoints keep native rows by default, as they did beside the head, so a cache hit's output does not move. A global balanced in a saved provider.toml does not change that; k4v4, a per-model setting or the process switch do. Lightning's target-only artifact and explicit mtp_mode = "off" follow the same default.
  • Cache identity includes the MTP configuration, so a provider's existing Lightning checkpoints stop matching and age out within their 30-minute lifetime.

Small local requests sent together are no longer refused as "memory" on a warm Mac

  • While a Mac's heat or Low Power Mode holds a model to its fallback width (4 instead of the M5's 8, as often right after a start on a warm Mac), a streamed local-API request waiting for its first token counted twice. So the third or fourth of four small requests sent together got 503 "This Mac's memory can't hold this request right now" with most memory free. Each request now counts once, as the memory check already did; a fifth is still refused.

Answers that repeat themselves verbatim stop early, as length

  • An answer that starts repeating one block of text word for word now stops after about 2,000 repeated tokens instead of running to its output limit (up to 131,072 tokens, 25–40 minutes on one Mac). It ends like an answer cut at max_tokens: finish_reason is length, the usage counts the tokens produced, and the request is paid as usual. The Mac is free for the next request sooner.
  • Only exact repetition counts: the last 2,048 tokens or more must repeat one block of 8–512 tokens at least 8 times in a row, and never within an answer's first 4,096 tokens. A tool call, a JSON answer and a forced tool call are never cut.
  • On by default for Gemma 4, Qwen 3.5/3.6/3.8, GPT-OSS, Nemotron 3.5 Lightning and Super (text), GLM-5.3 and MiMo. DARKBLOOM_LOOP_GUARD=0 turns it off.

A media decode refusal no longer marks the provider's token budget as full

  • A video or audio reader that fails twice with a decoder, hardware or interruption fault, and a Nemotron Super video whose decode allowance runs out, are still refused as a retryable 503 (failure code capacity). Their error reason is now media_memory_unavailable, not capacity_busy.
  • The coordinator still sends the request to another provider, up to its three capacity refusals. It no longer reads the refusal as a full token budget, so the provider's budget for the model is not clamped to zero for up to 5 minutes, and the provider gets no capacity strikes.

A slow MiMo step no longer stops the MiMo slot for good

  • Before, a native MiMo step that ran longer than 30 s, such as a prefill step deep into a long prompt on an M3 Ultra, made the engine's step watchdog seal the engine. The engine never took another step, even after the slow step finished, and the slot stopped serving until MiMo was reloaded.
  • Now a native MiMo step may run up to 120 s, the time the engine already allows a prefill to go without progress. A step that never finishes is still caught, after two minutes instead of 30 s. Other models keep 30 s.

Models catalog names the models this Mac could add

  • darkbloom models catalog now ends with the catalog models that are not downloaded and that this Mac could select: at most three rows with id and download size, larger first, then darkbloom models download <id> and darkbloom switch. The note is printed only when there is at least one such model.
  • A model is named when the darkbloom start picker would offer it to this Mac (required runtime capabilities present, not a hidden build or one replaced under a public alias), it is an active text model, its listed min_ram_gb is at most this Mac's memory, and it fits the provider's load budget alone. Free disk space, the memory reserve and memory in use at load time, the coordinator's minimum provider version and the provider's trust status are not checked, and the note says so.
  • The note only prints: nothing is downloaded, selected or saved, and no request is added. The table's catalog request now also asks for the public aliases (include_aliases=1); the --json request and output are unchanged.

Gemma 4 26B QAT drafts with exact MTP acceptance

  • The catalog gemma-4-26b-qat-4bit now uses exact draft acceptance by default: each token a speculating sampled request commits is the model's own draw, so its outputs follow the model's distribution (exact in distribution; the tokens are not bit-identical to decoding without MTP). Every other model keeps its built-in rule; the 8-bit Gemma builds draft only with mtp_mode = "on" and stay typical.
  • Its assistant drafts most Gemma rows, and typical acceptance keeps drafts the model would rarely have sampled; once a repetition starts the drafter predicts it and it is kept. On OpenRouter's GPQA run of 10-31 every Gemma answer drafted under typical acceptance and 30 % ran to the 32,768-token cap with no answer.
  • Exact keeps fewer drafts, so MTP speeds a Gemma stream up a little less: a speed cost of about 0–4 % (0.96x of typical at one request in the 10-07 measurements, 0.99–1.01x in the 11-10 M5 Ultra A/B). mtp_acceptance_by_model with "gemma-4-26b-qat-4bit" = "typical" restores the old rule.

Qwen 3.8 27B drafts with exact MTP acceptance on a single Mac

  • The catalog EigenLabs/Qwen3.8-27B-4bit-mtp (and its head-less target EigenLabs/Qwen3.8-27B-4bit) now uses exact draft acceptance by default: each token a speculating sampled request commits is the model's own draw, so Qwen 3.8's outputs follow its distribution (exact in distribution; the tokens are not bit-identical to decoding without MTP). This applies on every single-Mac path: network serving, Only me / --local, and both MTP upgrades. Qwen 3.8 pairs never used typical acceptance, and every other model keeps its built-in rule.
  • On a single Mac a lone Qwen 3.8 request drafts, and typical acceptance keeps drafts the model would rarely have sampled. On Qwen3.5-9B typical acceptance made long answers loop to the output cap (13 of 31 draws, against 0 of 31 under exact). Qwen 3.8's own production records point the same way, though they are observational: while the fleet moved from 0.9.17 (exact) to 0.9.18 (typical), Macs on typical ran long Qwen 3.8 answers to their max_tokens about 3.6 times as often on the same traffic. Since 11-09, every one of the 229 single-Mac Qwen 3.8 answers of 8,192 tokens or more that ended at their cap had drafted; none of the 33 long answers that never drafted did.
  • Exact keeps fewer drafts, so Qwen 3.8 decodes about 4–5 % slower on single Macs, all of it while a Mac serves one request at a time (a lone request runs at about 0.93x of typical in the 10-07 measurements). A Mac serving several requests at once is unchanged. mtp_acceptance_by_model with "EigenLabs/Qwen3.8-27B-4bit-mtp" = "typical" restores the old rule.

Drafting models no longer use up Metal shared events while several requests decode

  • While two or more requests decoded together on one model, each decode step left one Metal shared event alive. The count fell only when one request drafted alone or the requests ended. This hit Qwen 3.8 by default; the 9B, the 35B and Qwen 3.6 when a client sent presence_penalty: 0; Lightning; and Qwen4 Flash-Next. Two Qwen 3.8 requests added about 2,000 events a minute. Shared events are a finite system-wide pool. After roughly 15–50 minutes of such a stretch, the provider could abort with Failed to create Metal shared event.
  • The SDK now drops each finished step's event from the arrays the drafting history keeps. Tokens and drafts are unchanged, and no GPU work is added.

MTP assistant upgrade: no forward on the model while it serves

  • When a provider attaches the verified MTP assistant to a model that is already serving (the idle upgrade), building the replacement engine no longer runs a forward on that model. It reuses the KV types observed when the model was first loaded.
  • A forward from the preparing thread could race the serving engine. With Gemma 4's compiled feed-forward (mlx-swift-lm 093563be and later), that race crashed the provider with Attempting to eval an array without a primitive when a request arrived during the upgrade, typically the first request after a start.

Nemotron Super serves JSON output, tools and its settings with images and video

  • An image or video request to Nemotron 3.5 Super now takes the same request plan as a text request. response_format (json_object and json_schema), tools, tool_choice (auto, none, required or a named function), parallel_tool_calls and min_p: 0 are served with media. Before, the Mac refused every such request as invalid_media (400).
  • Tool calls and JSON answers with media are withheld and validated as for text: an invalid JSON answer or a broken forced tool choice returns 422, or ends a stream with an error event.
  • Nemotron Super now advertises native_media_tools, so the coordinator sends it forced tool choice with media and media in tool results. It also advertises native_media_controls, which a coordinator can require before it sends a Super request with media and a JSON response format or tools, so Macs without this release do not receive them.
  • A media-route refusal names its cause by its failure code. A request control that the text route also refuses (logit_bias, n > 1, min_p other than 0, prompt_cache_key, top_logprobs outside 0 to 20) is invalid_request (400), as are a prompt whose visual tokens do not fit the context and text that names <image> or <video>. Media the route does not serve (a label or encoding outside the image format reference, audio, more than 16 images, images with a video, a second video, a codec other than H.264 or HEVC) is unsupported_media (the coordinator answers 415). Media above a size, pixel or frame limit is media_too_large (the coordinator answers 413). Only corrupt media stays invalid_media (400). Before, all of these were invalid_media (400), a video's own frame and pixel limits included.

Nemotron Super's prompt cache now restores exactly

  • Nemotron Super's prompt cache now restores exactly: an answer from a cached prompt is the answer the same prompt gets without the cache. Super now saves its prompt checkpoints at native precision. Before, they used the provider-wide balanced default, which stores older attention rows at 4 bits.
  • Measured on the production weights on an M5 Ultra, with 3 reasoning prompts of about 8,600 tokens restored from an 8,192-token checkpoint: balanced checkpoints changed all 3 answers, and native checkpoints matched the uncached answers bit for bit in all 3, tokens and logprobs alike.
  • Checkpoints are larger. At 8,192 tokens one is about 249 MB instead of 204 MB, so about 15 % more is written per prompt; at 30,720 tokens it is about 457 MB instead of 285 MB. The SSD cache keeps its shared disk budget, so it holds fewer Super checkpoints.
  • Super's own per-model setting or the process switch still selects another precision. A global setting, even an explicit k4v4, k8v4 or k8v8, no longer applies to Super. Existing Super checkpoints stop matching and age out within their 30-minute lifetime.

Nemotron Super takes WebP, GIF and BMP images, under any image label

  • Nemotron 3.5 Super now decodes WebP, GIF (the first frame) and BMP images as well as PNG and JPEG. The coordinator already fetched and inlined these formats; every Super Mac refused them.
  • ImageIO decodes them to the same RGB input as PNG and JPEG, under the same byte, pixel and resize limits, checked from the header before any pixel is decoded. A WebP's EXIF orientation is applied as for PNG and JPEG. Alpha is dropped; a fully transparent pixel reads as black.
  • Other formats (TIFF, HEIC and the rest) are still refused, now as unsupported_media.
  • An inline image is taken under any image label, in any case: the format comes from the bytes, so a JPEG sent as data:image/jpg (an alias some clients send) or data:image/JPEG is served. Before, only data:image/png and data:image/jpeg were taken, and every other label was refused as invalid_media (400). A label that is not an image, percent encoding or a remote URL is refused as unsupported_media.

A nullable tool parameter takes null

  • A tool parameter a client declares nullable ("type": ["integer", "null"], or anyOf with a null branch) reaches the model's parser as "nullable": true, which the SDK ignored. A model's null for it, as Qwen 3.5-family XML calls write it, stayed the string "null", failed the parameter's type, and the client got a tool error for a legal argument. It now becomes JSON null and passes validation for a nullable integer, number, boolean, object or array. A nullable string keeps the text null, and enum, const and combinators still apply.

Paged decode keeps one attention pass per layer on a busy Mac

  • Before, on the paged KV backend (the 9B, Qwen 3.6, gpt-oss-20b, the 35B, Gemma 4 and the other paged models), decode slowed while a Mac stayed busy and recovered only when the engine drained. Freed pages went back to the front of one shared free list, so new requests' pages spread over many segments, and the decode kernel, which binds at most 17 segments per pass, needed about one pass per 256 tokens of context per layer instead of one per layer. In production the 9B decoded at 0.40× its fresh speed 3–6 minutes into a busy run and 0.30× after 6 minutes.
  • Now each layer of a request takes its pages in runs of 16 consecutive pages, in as few segments as it can, and requests that start together take different segments. A CPU model of the allocator and the decode planner stays at one pass per layer through busy runs: Qwen 3.6 at 4 requests goes from about 930 passes per step to 10, the 9B under a production-like load from about 430 to 8.
  • Pages that the prefix cache still names are still reused last. Each attention partition is computed as before, so outputs are unchanged.

Truer words on a paired Mac, the split question names each mode's context, and refusals without a usage block

  • On the Mac that follows in a pair, darkbloom doctor now reads the pair as darkbloom status does, from the leading Mac's answer: it said "loading" for as long as the pair served. That Mac's own view (status --solo) says "started" there, since only the leading Mac's part knows whether the pair serves.
  • After darkbloom cluster leave on one Mac, the Mac left behind no longer says it leads and the other Mac follows: while the other Mac doesn't answer, status, cluster status, doctor and the cluster screen say "this Mac (…) is paired with the other Mac (…), which isn't answering". The cluster screen of a stopped pair whose other Mac is gone says it is stopped, as cluster status does, instead of "degraded: serving on this Mac alone … (its solo server is still loading)" with nothing running.
  • How both Macs split a model: each mode's line now says the context it serves, for example "context 262,144 tokens" for Qwen 3.8 27B's tensor parallel and "context 65,536 tokens, prompts up to 32,768" for its pipeline, and the recommendation puts context before speed ("the longest context here (262,144 tokens; the other serves 65,536)"). A mode that serves less is recommended only when it alone fits now.
  • An image or video sent to a model that doesn't take it, on one Mac's local API or a pair's, is now refused with the error code unsupported_media beside the same words.
  • On a Mac in a pair, every flag beside --solo is for that start only, and a solo start saves nothing in [local], which holds the pair's settings, but this Mac's one port. A mode flag (darkbloom start --solo --local) or the answer to --choose serves that start, and [local] mode, the pair's answer, is no longer replaced; --choose asks no route. --route, --me-first-idle, --no-me-first, or --me-first over a pair whose pause is off, would change the pair's setting, so beside --solo each is refused before anything starts, naming it. A plain darkbloom start --local on one Mac still saves its choice.
  • darkbloom local names the file that holds the key it shows (key file: ~/.darkbloom/local_token), and on a pair's API also the pair's key file, whose key works there too.
  • A tool set up beside a paired Mac's solo server (the other Mac away) names its models "(this Mac)", not "(your pair)".
  • A terminal darkbloom start on a stopped pair says "Press Enter to start it …" only while the other Mac answers. The app's start page on a pair that was started but can't be seen serving from this Mac says Start serving "Makes sure the pair serves …".
  • A script's darkbloom start on a pair with no model says only how to choose one, without "starting the pair" first. On a serving pair whose cluster service runs from a terminal session, it again moves the service to its login agent, so the pair comes back after a restart, instead of saying "the pair already serves" and leaving it.
  • A refusal a command makes once it runs now ends with its own words, without the generic "Usage: darkbloom / See 'darkbloom --help' …" block; mistyped flags still get the usage. Ctrl-C on darkbloom start --local --here no longer prints the local server's "Server shutdown error: Already closed" and "Waiting on child channel" lines.

A pair Mac that runs short of memory ends only that request

  • When one Mac's memory refuses a request while it runs, both Macs now end that request alone and keep serving, in every pair mode: the pipeline (Qwen, Gemma 4, GLM, MiMo V2.6, GPT-OSS) and tensor parallel (Qwen 3.8 27B, Gemma 4, GLM, MiMo V2.6), one request or several. A local request that isn't streamed gets one Mac's memory 503, "This Mac's memory can't hold this request right now. …". A streamed one gets the tokens already sent, then an error chunk with code request_exceeds_node_budget and the same words. Darkbloom traffic gets capacity with request_exceeds_node_budget. Before, the refusing Mac's worker exited, the client got an untyped 500 and the pair stopped serving for about a minute.

A MiMo slot whose engine stops stepping stops taking requests

  • Before, a native MiMo slot whose engine had stopped stepping, with a request queued and nothing running, still reported itself idle for as long as it stayed that way, so the coordinator kept routing requests to it. Each was accepted and timed out without a word.
  • Now an engine whose native work failed and was sealed by the SDK reads crashed at once, on any model. A native MiMo slot whose queue holds work with nothing running and no step reads crashed after 30 s. Either way the coordinator routes around it, and the provider then retires the MiMo slot, recording "Native MiMo slot failed; confirmed retirement or process restart required" in darkbloom status.
  • Other models keep their existing wedge rules: on them a queue that waits with nothing running is usually healthy.

A media request a Mac can't hold now is refused as media_memory_unavailable

  • A media request that this Mac's memory can't hold right now is refused as a retryable 503 with the reason media_memory_unavailable (it was capacity_busy), so Darkbloom sends it to another Mac without counting this Mac's capacity as full.

Qwen3.5-9B speculates and caches again, with exact MTP acceptance; the 35B Qwen builds keep their penalty

  • Since 0.10.5 the catalog Qwen3.5-9B, qwen3.5-35b-a3b and qwen3.6-35b-a3b-vl-mtp-mxfp8 builds have given a request that omits presence_penalty the model cards' 1.5. The engine drafts no row with a penalty, and these builds' MTP heads keep the prompt history, so through 0.10.11 such a row also staged no resident or SSD prefix checkpoint.
  • The 0.10.12 gate on the signed 0.10.11 decided each build. It ran temperature 1, top_p 0.95 and top_k 20 over #1652's four loopers and five tool transcripts at 32,768 tokens and nineteen everyday prompts at 8,192, three seeds each.
    • qwen3.5-35b-a3b: without the penalty it ran away under typical MTP acceptance (8 of 12 loopers to 32,768, 5 of 19 everyday prompts) and under exact (2 of 12 loopers).
    • Qwen3.5-9B: it ran away on 13 of 31 draws under typical acceptance and on none of 31 under exact.
  • Qwen3.5-9B now has no default presence penalty, and its built-in MTP acceptance is exact. Its requests speculate and stage prefix checkpoints: in the gate the penalty-free body cached 8,192 tokens on its second send, where the penalty body cached none. A [backend] mtp_acceptance_by_model or mtp_acceptance value still takes precedence.
  • qwen3.5-35b-a3b and qwen3.6-35b-a3b-vl-mtp-mxfp8 (one architecture) keep the 1.5, so their requests that omit it still run without MTP; they now save prefix checkpoints (see the entry on Qwen 3.5/3.6 requests that cannot speculate).
  • A request's own presence_penalty always wins: 0 on a 35B build speculates under typical acceptance and donates checkpoints, and any nonzero value on the 9B turns MTP off for that request, which still saves prefix checkpoints.

Qwen 3.5/3.6 requests that cannot speculate save prefix checkpoints again

  • From 0.10.5 through 0.10.11 the Qwen3.5-9B, qwen3.5-35b-a3b and qwen3.6-35b-a3b-vl-mtp-mxfp8 builds gave a request without a presence_penalty the model card's 1.5, and a request with a penalty cannot speculate. Their MTP head keeps each request's prompt history, and a request that could not speculate recorded none, so it saved no SSD or resident prefix checkpoint. A repeated prompt or the next turn of a conversation was prefilled again from the start, and the coordinator saw almost no cache holders for these builds.
  • Such a request now records the head's prompt history and saves the same checkpoints a speculating request saves. That covers the card's penalty (still the 35B builds' default), any other penalty, a stop string, logprobs, logit_bias and a token constraint. The penalty and its sampling are unchanged, and the request still never speculates.
  • The same holds on Qwen3.8-27B (the same embedded head) and on Nemotron 3.5 Super: a request with a stop string, logprobs, a penalty, logit_bias or a token constraint now saves prefix checkpoints too. Its prompt runs the prefill a speculating request's prompt runs, and its answer is unchanged.
  • A request that cannot speculate and restores one of these checkpoints answers exactly as without the cache, tokens and logprobs alike. A speculating request restores the same history its own prompt would record.
  • From the SDK pin libs/mlx-swift-lm at 505e6bd8 and later, which contain Layr-Labs/mlx-swift-lm#398 (it carries #397).

Qwen 3.5 MTP prompts project only their last position

  • With MTP active, every prompt chunk of a Qwen 3.5-family model (Qwen3.5-9B, Qwen3.5-35B-A3B, Qwen3.6,
    Qwen3.8-27B and Bonsai 2 9B) went through the 248,320-way output head at every position, though only the last
    row is used. The pinned SDK (mlx-swift-lm #302) now projects only the last position of the last chunk. On an
    M3 Ultra, time to first token with MTP fell by about a tenth at 4K–16K prompt tokens on Qwen3.5-9B and
    Qwen3.5-35B-A3B, with identical temperature-0 outputs. The first token's logits now equal the MTP-off path's
    exactly. Requests where MTP does not run see no change.

A refused request tells the coordinator why

  • The inference_error frame of a client refusal names its reason:
    refusal_reason, the closed reason darkbloom logs already names (such as
    missing_user_query, unmatched_tool_result or request_exceeds_context),
    and refusal_message, one fixed sentence for that reason. Neither carries
    request text. The error text the requester gets is unchanged ("Invalid
    inference request."); today's coordinator ignores both keys until it is
    updated to record or forward them.
  • Qwen 3.5/3.6/3.8 tool conversations name the check that refused them: a
    conversation with no real user turn is missing_user_query, and a tool
    message with no assistant tool_calls before it is unmatched_tool_result
    (before, both logged and sent no reason). The same history checks on every
    model, GPT-OSS's and Gemma 4's included, name theirs, and required or named
    tool_choice with image or video input is unsupported_forced_tool_choice.

Selected models recover their residency automatically

  • A model you selected explicitly (enabled_models, preload_models, the pinned model or Autopilot's selection) that the startup preload skipped or failed to load for memory, or left untried at the slot cap, is now retried automatically: after 30 s, backing off to every 10 minutes, at once after any unload or a saved enabled_models or Autopilot change, and when a placement hold, an idle threshold, a minimum dwell or a lease ends. Previously nothing retried it, so behind a busy model outside the selection it stayed cold until a restart. A model that loads but is then refused for memory three times (too little left to serve) waits for an unload or such a change instead of the backoff, since each of those retries loads the whole model. The queue lives in memory: after a restart only the startup preload queues a model again.
  • The same applies to a selected model an Autopilot operation unloads to load another model, for example Nemotron Super unloaded to place GLM-5.3-Flash: it waits from the moment it leaves residency, whether the operation then succeeds or fails, and comes back once the placement no longer protects GLM. An operation that only unloads models queues nothing.
  • A retry never evicts a selected model. Before loading it may unload only models outside the selection that no request could load straight back and whose memory makes the room, and only once such a model has no request in flight and no Autopilot hold, pin or per-model lease, has been resident for Autopilot's min_dwell_seconds, and has been idle (since its last request or its hold's expiry) for the longer of two minutes and min_idle_seconds. A model of the running daemon's selection, even one your saved filter dropped, is never evicted, so a recovery never alternates between two models. It waits while a live Autopilot lease manages residency, a drain runs, an add-on holds the Mac or you use the Mac under me first; one of these that begins while a retry unloads a model stops the retry before its load, and a hold that begins during the load cancels it.
  • darkbloom status prints Retrying <model> when memory frees; next check in … (or next check after an unload or a configuration change) with what blocked the last check, and Configuration change needs a restart to apply: enabled_models adds/removes … when the saved filter differs from the one the running selection applies: the filter the daemon started with, or the one the last darkbloom switch saved and applied, so a committed switch no longer asks for a restart.
  • darkbloom stop, restart and start no longer need --force when a model's native retirement never finishes: shutdown waits at most 15 s for each model's unload, so that model no longer keeps the process from exiting.
  • Policy (the owner kept it): ten minutes after the placement's hold ends or the selected model began waiting, whichever is later, and not before the placed model's minimum dwell or per-model lease ends, an adopted model outside the selection that keeps a waiting selected model out only by serving Darkbloom requests is released: it takes no new requests, finishes those in flight and is evicted. Holds, selected models, pins and a model one of your local requests is using are never released.

darkbloom start serves Both without a question; darkbloom local serves only you

  • darkbloom start no longer asks how this Mac serves, in a terminal or not. On a Mac with no choice saved it
    serves the Darkbloom network and this Mac's local API, and Darkbloom keeps serving while you use the Mac; one
    line says so (Serving Darkbloom and your local API at http://127.0.0.1:8001/v1 (`darkbloom local` for local only).) and the choice is saved ([local] mode = "both", me_first = false). With an account key saved, a
    request this Mac can't take goes to your other Macs, free. Scripts and the desktop app's Start without its local
    API take the same default where nothing is saved (they served the Darkbloom network alone before).
  • A saved choice is never changed: starts, restart, update, login and the watchdog serve it as saved, a pause
    saved by the earlier question included. --local, --network, --me-first, --no-me-first, --me-first-idle
    and --local-endpoint work as before. --choose asks nothing and says which commands to use.
  • darkbloom local starts Only me in the background where nothing serves, then prints the address and key. Where
    Darkbloom serves, it prints them (or that there is no local API) and, only in a terminal, asks Switch this Mac to local only (stops serving Darkbloom)? [y/N]; without a terminal it never switches. Where Darkbloom is stopped
    but saved (network or Both), it says so and asks the same in a terminal only; without one it changes nothing
    (exit 1). On a Mac in a two-Mac cluster it points to darkbloom cluster local. --model <id> names Only me's
    models; --json only reads.
  • A plain start no longer reaches the Autopilot question, which only a Darkbloom network start asks: use
    --network or darkbloom autopilot enable. The idle-memory question is skipped when standard output isn't a
    terminal, as it already was for standard input.

darkbloom status shows the answers in progress

  • Under the serving block, darkbloom status lists each answer the Mac is
    writing: the model, how long it has run, output tokens so far, its current
    tokens per second and a short hash of its request (never the prompt, the
    answer or the request ID), then "Credits post when a paid answer finishes;
    requests to your own machine are free." A Mac writing two long answers no
    longer looks hung while its served and token counters wait for them to
    finish. The engine's own calibration and warmup probes are not listed. On a
    two-Mac cluster the leading Mac lists the pair's answers under the pair.
    darkbloom status --json prints the same list as JSON.
  • The daemon reads them from counters its engines already keep, once per
    state-file write (about every 2 s), into running_requests in its state
    file. Nemotron Super's answers are not listed yet.

darkbloom status lists the answers in progress on a Mac that serves only you

  • A Mac that serves only you (darkbloom start --local) now lists its answers
    in progress under darkbloom status (and in status --json), as a Mac that
    serves Darkbloom does: before, its state file never carried them, so a long
    local answer looked like nothing was running (0.10.12 W1). Its standalone
    server samples the counters its engines already keep, at most every 2 s.
  • A pair that serves only on its local API (darkbloom cluster local) still
    lists none: its serving process writes no state file yet.

darkbloom status says which requests that ended early are paid

  • The line under Requests served said that every request that ended early "may be unpaid". Many are paid: when the requester leaves mid-answer, the coordinator pays for the prompt and the output your Mac had already sent. status now lists those requests by what stopped them and whether they are paid, for example:
      7 of them ended early:
        5 paid for the work done: the requester left mid-answer
        1 unpaid: another Mac answered first
        1 unpaid: the connection to the network dropped
    
    Your Mac knows itself when its connection dropped or when it stopped a request (a drain or an add-on hold). The other causes come with the coordinator's cancel from the coordinator update that sends them; until then those requests share one note: paid for the work done if the requester left mid-answer, unpaid if another Mac answered first or this Mac started too late.
  • With the coordinator update that sends the cause, a request whose answer had already finished when the requester left is no longer counted as ended early: it is paid as delivered. About 1 in 100 of the requests counted before were such answers. An unpaid stop still counts, finished answer or not.
  • The count now takes a line per cause, so a script that reads status line by line sees more lines.

darkbloom status says whether fan control is on

  • Fan control is opt-in (sudo darkbloom fan enable), and only darkbloom fan status said it was off, so a Mac with supported fans could serve hot under macOS automatic fan speed without anything in the normal flow saying so. darkbloom status now has a Fan control (experimental) line from the same report: the helper's mode when it runs; off, macOS automatic with the hottest GPU sensor and the enable command on supported hardware without the helper; and otherwise not answering, installed but not running (after fan disable, with no advice to enable), unsupported or unavailable. Fan control stays opt-in; nothing enables it.

Nemotron 3.5 Super: a request that arrives as the previous one finishes waits instead of bouncing

  • A Super Mac serves up to 8 requests at once, but only one image or video at a time. It answered another image at once with 429 queue_full, which sends the image to another Mac. That included images arriving after the previous answer had ended but before its cleanup (native retirement, and the release of its image memory) had finished. The same held for any request when the Mac's Super owner was at its request limit.
  • Such a request now waits up to 1 s for that cleanup and is then served. The previous request's memory is released before the next one reserves its own, so memory use is unchanged. A request that arrives while the previous answer is still being generated is still refused at once.

Nemotron 3.5 Super: an image that arrives while another image's answer is streaming waits for it instead of bouncing

  • A Super Mac serves up to 8 requests at once, but only one image or video at a time. An image that arrived while another image's answer was streaming was refused at once with 429 queue_full, so the coordinator sent it elsewhere or, after three refusals, gave up. Such an image now waits for the slot, up to 3 s, and is served when the image ahead of it finishes. An image's answer usually ends about 2 s after it starts streaming.
  • For Darkbloom requests the wait never spends the request's time limit for its first token. It stops early enough to leave room for this Mac's slowest recent time to first token on images (counted as at least 6 s), and is skipped when no room is left.
  • Only one request waits at a time, and a waiting request reserves no memory, so two images are never in memory at once. A waiting image never takes the last request slot a text request could use. Text requests never wait behind an answer still being generated, and an image never waits behind a text answer or behind an image whose prompt is still being processed.

Nemotron 3.5 Super answers that repeat themselves verbatim stop early too

  • Super's text requests now get the loop guard the other models have: an answer that starts repeating one block word for word stops about 2,000 repeated tokens in, as finish_reason: "length" with its usage, and is paid as usual. Super's requests take the owner's own route, which passed no guard before.
  • The same rules apply: never within the first 4,096 tokens; never for a JSON answer, a forced tool call, a tool call once it starts, or a request with images or video. DARKBLOOM_LOOP_GUARD=0 turns it off for Super too.

Opt-in: answer before the checkpoint write finishes

  • A request that donates a complete SSD checkpoint ends its response only
    after the checkpoint is durable, so the client waits for the write.
    DARKBLOOM_PREFIX_CACHE_SSD_TERMINAL_BEFORE_PUBLICATION=1 (default off)
    ends an unseeded donor's response as soon as the answer is complete. The
    provider still holds the request's rows, KV reservation and service
    allowance until the write settles, so other requests see no added
    capacity. A follow-up that arrives before the write finishes can't use
    that checkpoint yet: it restores an earlier one, if any, and prefills the
    rest. Seeded requests keep the old order.
  • For an installed provider, set it in the shell for darkbloom start,
    which writes it into the provider's launchd job; an ordinary darkbloom restart keeps the saved value. Each such request that retires at least
    1 ms after its answer is logged with that time, which for a donor is how
    long its write held its capacity after the answer (Early terminal: request … retired N ms after its answer). The 1 ms floor is a timing heuristic: on
    a busy provider a request that wrote no checkpoint can be logged too. The
    line is only in the provider log.

tool_choice: auto returns the model's tool calls as generated

  • With tool_choice: auto (or no tool_choice) and no function declared strict: true, the provider now returns the model's calls the way OpenAI and vLLM do. A call to a function the request did not declare, and arguments that do not match the declared schema, reach the client unchanged. When a streamed call's arguments are not valid JSON, the text already sent stays as the raw arguments string, the call is closed and the response finishes with tool_calls. Before, each of these failed the request with a 422 tool_noncompliance after its reasoning had streamed, so the client saw a 200 followed by an error and the coordinator could not send it elsewhere. This mostly affected GPT-OSS agent turns.
  • A function declared strict: true keeps the whole request validated, as do required, named and none tool choices, a request that declares no tools, and parallel_tool_calls: false. A call read from a frame that max_tokens cut off is still validated, since it may be missing arguments.
  • On a Mac's own API (--local) and for GLM-5.3-Flash, a strict request no longer refuses a valid value for a property that declares no type: the provider fills in type: "string" for such properties so the chat templates can render them, and it used to enforce that type too; strict calls are now checked against the schema as the caller sent it. Through Darkbloom's coordinator, strict requests to every other model still refuse a non-string value there, because the coordinator itself writes type: "string" into typeless properties before the request reaches the Mac.
  • A tool_noncompliance error frame now carries tool_check: which check failed (undeclared_name, schema, malformed, diverged, unterminated, parallel, response_format, forced_choice, native_output or other). The provider's local failure log line shows it too. It names the check only, never the tool or its arguments.

A restart you ask for after an update no longer rolls the update back

  • Until a new release has run for 10 minutes, three failed starts bring back the previous release and quarantine the new one. A darkbloom restart (or stop and then start) in those 10 minutes could count as one: the watchdog, started again by the command, looked at the provider the command had just replaced, saw no heartbeat from it, and counted a failed start, so a Mac restarted three times was rolled back and its release quarantined though nothing had failed. Now a provider that was asked to stop (by darkbloom stop or restart, or by its own restart into an update) records that before it exits, and the watchdog times the next provider's start from when that provider started: the restart is never a failed start. After a crash, a hang, a reboot, a kill or any other stop, whatever starts the provider again, its start is still timed from the update's launch, as before, so a release that crashes or hangs the Mac is still rolled back. A --force stop that has to kill the provider before it records its stop is timed from the update's launch too. A provider that sends no heartbeat in its first 5 minutes (longer with a raised startup_preload_timeout_secs) still counts too.
  • A new release is promoted only once one provider process has served it for 10 minutes. When something restarts the provider more often (a script, a cron job or a login item), the watchdog now says so in ~/.darkbloom/watchdog.log after 5 restarts, then after 10, 20, 40 and so on.
  • On a darkbloom cluster Mac, a clean stop of the cluster service that its supervisor didn't ask for (darkbloom cluster replacing a service started from a terminal, a kill) no longer counts as a failed start either. A crash, an exit with an error, a service that stops beating and a release that fails its start check still do.

The watchdog brings back a provider job launchd lost

  • When launchd dropped the provider's job while it was enabled (booted out, not stopped with darkbloom stop), the watchdog treated it as stopped and nothing restarted it until someone ran darkbloom start (#1104). It now restarts it after the usual grace, when auto_restart is on, the installed job is the provider's own and enabled, and no provider process is still alive, under the same lock as stop and updates. A job you stopped stays stopped, and nothing changes for a Mac whose pair runs here or for a pair member's job.
  • A pending update is handled as on a crash: a confirmed failed start counts once, the verified rollback and its retry backoff apply, and no update is downloaded on this path. An uncertain launchctl result keeps the crash-loop protection without counting a restart.

Pull requests in this release

9 pull requests merged into dev reached master with #2039, which promoted dev up to df36d81c3f:

  • fix(cluster, worker, provider): 0.10.12 — a pair Mac's memory refusal ends only that request, in every pair mode (D6) by @jonathan308 in #1932
  • CI (dev): route the build lanes and Cluster Release Build Checks to Darkbloom while job sizes are off by @davidtai in #1951
  • ci: Cluster GPU Tests shards 1 and 3 ask Darkbloom first by @Gajesh2007 in #1955
  • ci: route the desktop review package through Darkbloom (Blacksmith fallback) by @davidtai in #1986
  • ci: a Darkbloom Sandbox without the Darkbloom cache uses no Actions cache; refuse an unsafe cached .git by @davidtai in #1981
  • ci: gate desktop-package's npm cache like every routed job (follow-up to #1981) by @davidtai in #1990
  • ci: Cluster GPU Tests shard 2 asks Darkbloom (bound the paged bench's MLX buffer cache) by @davidtai in #1987
  • ci: bound MLX's freed-buffer cache in the peak cluster tests; runtime lanes 12x32 → 4x24 and 4x24 → 4x16 by @davidtai in #1988
  • 0.10.12 candidate d by @jonathan308 in #2038

1 pull request merged directly into master:

2 release pull requests, not listed above: #2039 prepared the release; #1970 prepared the release.

Full Changelog: v0.10.11...v0.10.12