Repository navigation
v0.10.13
Highlights
Plain-language summary of provider 0.10.13. The full technical changelog, the pull requests and the build's
verification details are on this page too.
- Gemma 4 no longer gets stuck repeating a thought: when its reasoning starts to loop, it stops thinking and answers.
- That answer now starts even when the Mac is busy with other requests, instead of timing out in the queue.
- When the Mac is full, that answer takes the stopped request's place instead of being turned away.
- Qwen 3.5 and 3.6 can do the same when their operator turns it on.
- For Qwen 3.5 9B and 35B it is now on by default.
- Nemotron 3.5 Super can do the same when its operator turns it on; it is off by default until it has been measured.
- When you set up a two-Mac pair, the second Mac no longer says "Stopped" while the first Mac downloads the model.
- GLM-5.3-Flash no longer turns requests away on an idle Mac after a run of long prompts: memory kept from finished requests is given back when a new one needs it.
- GLM-5.3-Flash and MiMo no longer turn long requests away on an idle Mac because a short one ran just before: the time a long prompt needs is no longer guessed from a few-token prompt.
- Qwen 3.5 9B gets its default presence penalty back, so fewer of its answers run on to their token limit.
- GPT-OSS answers that ask for JSON output are no longer lost: an answer the model marks as JSON reaches you instead of an empty reply.
- GPT-OSS honours
tool_choicerequiredand a named tool: the answer is a call to a tool (or to that tool). - An answer that reaches its token limit ends cleanly as
length, including a tool call cut off at the limit. - Qwen 3.5, 3.6 and 3.8 no longer lose a tool call because their thinking mentioned a tool.
- On request, tools and tool results reach the model exactly as Hugging Face Transformers renders them.
- A Mac notices a dead link to Darkbloom within seconds and reconnects at once.
- Earlier thoughts that a client sends back as
reasoningorthinkingreach the model again. parallel_tool_calls: falsereturns the first tool call instead of failing the answer.- Gemma 4 keeps a text value such as
01000as written instead of turning it into a number. - A GLM-5.3 answer that mentions a tool tag in passing no longer fails.
- Answers about images and videos that repeat themselves word for word stop early too.
- Optional: GLM-5.3 can return a JSON answer that misses the requested schema instead of an error, for the caller to judge.
- Gemma 4 answers in exactly the JSON format you ask for.
- Qwen tool calls marked strict always match their schemas.
- Bonsai 2 27B and Nemotron 3.5 Lightning answer in exactly the JSON format you ask for.
- Qwen JSON answers follow schemas written with Pydantic, OpenAI or zod.
- GPT-OSS no longer runs out of tokens repeating its reasoning on tool requests: it stops and answers or calls a tool.
- Security: the link between the two Macs of a pair is protected; both Macs need 0.10.13 to pair.
- Two-Mac pairs keep their link key in the Mac's Keychain instead of a file.
- Security: each Mac of a two-Mac pair proves its identity on the link.
- Security: a two-Mac pair checks the processes it starts.
- GPT-OSS stops repeating its reasoning and answers on ordinary requests too.
- GPT-OSS no longer drops the text it writes before a tool call.
- Qwen answers no longer show a stray .
- Nemotron Super answers two image requests at once.
- Qwen 3.8 strict tool and JSON answers decode faster.
- Gemma 4 tool calls give numbers where the tool asks for numbers.
- Hardened tool-call constraints for complex schemas.
- Security: stricter updater version checks.
- Clearer security information when pairing two Macs.
- Security: hardened request parsing.
- Security: tighter resource limits on queued requests.
- Steadier memory use when the settings are re-read.
- Gemma 4 no longer leaks a reopened thought into its answer.
- Nemotron Super and MiMo honour a reasoning token budget and still answer.
Known issue: a two-Mac pair that serves only you (no Darkbloom network) and is updated by hand can stay on the earlier release's cluster service after the update and stop pairing. To recover, run darkbloom cluster stop --local-only --service on each Mac, then darkbloom start on both. Pairs that serve through Darkbloom update together at their slot.
Provider v0.10.13 (Swift CLI)
Source: bfae049
Build and SDK qualification: https://github.com/Layr-Labs/d-inference/actions/runs/43615978527
Signing: https://github.com/Layr-Labs/d-inference/actions/runs/43615978527
Binary SHA-256: 79b317490ae2f97495a86aa50aeacd11fae5d6f89ff609e9e00b374938105d52
CodeDirectory SHA-256: 27eb19baf3d0d0587f7f1c1ba3f7deb60250ad40cb1ea0f9eca11bc1a842a1ba
Bundle SHA-256: 3c95bfb29197270e22f9320da93f036b872fa32733e02cbd85d7d9f944307316
Metallib SHA-256: 5b97893cc20e6e9651edf3284d970488af9a9bdd64820231e95fee760eec70ac
Signed and notarized: yes
macOS deployment target: 14.0 (build, tests and signed runtime smoke: macOS 27)
Install
curl -fsSL https://api.darkbloom.dev/install.sh | bashWhat's new in v0.10.13
Upgrade and rollout notes
- A Gemma 4 thought that repeats itself verbatim now ends in an answer: the loop guard closes the thought and the model answers in the same response (at most 8,192 tokens, within the request's budget), instead of ending as
lengthwith no answer. A loop in the answer still ends aslength. Gemma 4 only; never with logprobs or on a pair;DARKBLOOM_LOOP_EXIT_THINKING=0turns it off (#2040). - Setting up a pair no longer says "Stopped" while the leading Mac downloads: the joining Mac ends its screen only on a stop someone asked for, and such a stop now ends the leading Mac's start at once, also during the download or the load (#2042).
- Faster recovery from a dead coordinator link: the provider pings every 2 s and treats the link as dead after 6 s without a frame, so a link that died silently (for example on a coordinator restart) is noticed in 6 to 8 s instead of up to 40 s, and the first reconnect after a long session starts at once (#2047).
- Qwen 3.5 9B keeps its model card's presence penalty: a request that sends no
presence_penaltygets 1.5 again, as on the other Qwen 3.5/3.6 builds. Provider 0.10.12 dropped it, and the 9B's answers then ran on tomax_tokensabout three times as often; its JSON answers that ran out failed about twice as often. A request's own value still wins. Those requests decode without MTP, as on 0.10.11.DARKBLOOM_QWEN35_9B_PRESENCE_PENALTY=0restores 0.10.12's default (#2051). - Gemma 4 keeps a string argument as written: an unquoted
01000,1.10or a long id for a string parameter stays text instead of turning into a number, and astricttool no longer fails the call partway through the stream (#2050). - A past turn's reasoning sent back as
reasoningorthinkingreaches the model: clients that echo OpenRouter'sreasoningfield (orthinking) no longer lose the earlier thoughts that Qwen, Bonsai 2, MiMo and Gemma replay; the first non-null ofreasoning_content,thinkingandreasoningwins (#2048). parallel_tool_calls: falsereturns the first tool call instead of failing the response when the model makes a second one, streamed or not, on every model, as OpenAI and vLLM do (#2049).- A stray
<tool_call>no longer fails a GLM-5.3-Flash response: a tag the model only mentioned or drafted in its thought is returned as text instead of a 500; the calls are the well-formed frames that end the response. Required and namedtool_choicestill fail closed.DARKBLOOM_GLM_TOOL_TEXT_FALLBACK=0restores the 500 (#2052). - An answer cut at
max_tokensends aslength, not as an error: a JSON answer or tool call the limit cuts off now ends normally withfinish_reason: "length"and its usage (a cut call is left unclosed so no client can run it), and astrictcall that fails the provider's check once a stream has started returns as generated.DARKBLOOM_LENGTH_FINISH_AT_LIMIT=0andDARKBLOOM_STRICT_CALLS_AS_GENERATED=0restore each refusal (#2053). - The loop guard covers image and video requests: a media answer that repeats one block verbatim ends as
lengthinstead of running to its output cap, and Gemma 4's repeating thought on a media request ends in an answer. On OpenRouter's VGI-Bench rerun, 117 of 257 Gemma video answers had run to the cap (#2054). - Qwen 3.5 and 3.6 can end a repeating thought in an answer, as Gemma 4 does (off by default):
DARKBLOOM_LOOP_EXIT_THINKING=gemma4,gemma4_text,qwen3_5_moeturns it on for the Qwen 3.5/3.6 35B models (addingqwen3_5covers the 9B and Qwen 3.8 27B). When the loop guard stops a verbatim loop inside the thought, the provider ends the thought as Qwen's template does (</think>) and the model answers in the same response, instead of ending aslengthwith no answer. It is off by default because the A/B found too few Qwen thought loops to show that they answer. Never with logprobs or on a pair. For Gemma 4's exit (#2040), a stop string that ends the answer is now reported as the matched stop sequence (#2046). - Qwen 3.5, 3.6 and 3.8 parse tool calls after the thinking: a thought that mentions
<tool_call>no longer turns the real call into text, and a call written inside the thinking is no longer executed; a stray thought before a forced call is returned asreasoning_contentinstead of failing the request (#2057). - Tool JSON as Transformers renders it, on request: a chat request with
"_darkbloom_template_json": "transformers"shows Qwen 3.5/3.6/3.8, Bonsai 2, MiMo, GLM-5.3 and GPT-OSS their tools, calls and results exactly as Hugging Face renders them; every request that does not ask renders the same bytes as before (#2058). - GPT-OSS JSON answers are no longer lost: an answer that opens with Harmony header attributes, as in
<|channel|>final <|constrain|>json<|message|>, now arrives ascontentinstead of an empty reply; everything else in a GPT-OSS response is byte-identical (#2063). - GLM JSON answers: an optional release of schema misses. By default a GLM-5.3-Flash JSON answer that misses the caller's schema is still refused with a 422;
DARKBLOOM_GLM_JSON_SCHEMA_MISS=releaseexposes it instead when the model stopped by itself, so the caller's validator judges it (#2065). - GPT-OSS honours
tool_choicerequiredand a named function: the model reasons first, at the request'sreasoning_effort, then writes only a call to a declared function with arguments that match its schema, arriving astool_calls(before, these requests were refused with 400tool_choice_unsupported). The coordinator sends them only to Macs on this release, never to a pair.DARKBLOOM_GPTOSS_FORCED_TOOLS=offturns it off (#2045). - GLM-5.3-Flash and MiMo no longer refuse long prompts because of a short prompt's speed: an idle Mac priced every prompt's prefill from the last cold prompt it ran, whatever its size. A few-token prompt reads as 9-140 tokens per second (its fixed setup) on GLM Macs that read long prompts at 1,400-2,000, so they refused nearly every longer request as
deadline_unreachable(1,300 of the 1,452 requests of OpenRouter's Tau2 run on GLM) and never measured one. A prompt of up to 4,096 tokens may now raise that estimate but never lower it, and an idle Mac with no recent calibration or longer-prompt reading prices prefill at no less than its own calibration. GLM and MiMo only.DARKBLOOM_TTFT_NATIVE_PREFILL_EVIDENCE=0restores 0.10.12's behaviour (#2072). - GLM-5.3-Flash no longer refuses requests on an idle Mac whose KV pool kept its free memory: the pool now returns its free segments before retrying a refused lone request (waiting up to 10 s while the previous request's prefix checkpoint is published) and after 30 s idle. On 0.10.12 the pool kept every segment and each prefix-cache restore added more, until a request arriving at an idle 256 GB Mac failed as
capacity_busyright after admission (about 3,960 a day), which also put the Mac into the coordinator's capacity cooldown. GLM only.DARKBLOOM_NATIVE_SEGMENT_RELEASE=0restores 0.10.12's behaviour (#2060). - Nemotron 3.5 Super can end a repeating thought in an answer (off by default):
DARKBLOOM_LOOP_EXIT_THINKING=gemma4,gemma4_text,nemotron_h_omniturns it on. When the loop guard stops a verbatim loop inside Super's thought, the provider ends the thought (</think>, as Super's template does) and the model answers within the same request, instead of ending aslengthwith no answer. Off by default until its A/B. Never with logprobs, media or on a pair (#2061). - A GPT-OSS analysis that repeats itself now ends in an answer or a tool call on requests that declare tools: the provider closes the analysis and the model writes its final answer or call in the same response (on 126 Tau2 airline turns, turns with nothing fell from 74 to 38).
DARKBLOOM_LOOP_EXIT_THINKING=gemma4,gemma4_textturns it off for GPT-OSS (#2070). - Strict Qwen tool calls always match their schemas: on one Mac, a call to a function declared
strict: trueis constrained while it is sampled, so it can no longer miss its schema and fail the response partway. On by default for Qwen 3.8 27B, Qwen 3.6 35B and Qwen 3.5 35B (Qwen 3.5 9B when a Mac opts in). While such an answer decodes, drafting pauses on that Mac, so a lone strict answer on Qwen 3.8 decodes at about 50 instead of 120 tokens per second. Schemas the grammar can't hold exactly are served as before.DARKBLOOM_STRICT_TOOL_GRAMMAR=offturns it off on a Mac (#2069). - Gemma 4 JSON answers follow the requested format: with a JSON
response_formatthe answer after the thought is exactly one JSON value of that format, with no code fences or extra text, including schemas written with$defs/$refor optional fields (100 of 100 valid in our test, up from 87). Requests the grammar doesn't cover and pairs are answered as before.DARKBLOOM_GEMMA_JSON_ANSWERS=offturns it off (#2071). - Valid JSON answers on Bonsai 2 27B and Nemotron 3.5 Lightning: a JSON
response_formatrequest whose answer starts without a thought is sampled under the same grammar the Qwen 3.5 family uses, so the answer is one JSON value of the requested schema.DARKBLOOM_JSON_ANSWER_GRAMMAR=qwenkeeps the grammar to Qwen (#2075). - Qwen JSON answers follow Pydantic, OpenAI and zod schemas: schemas with nested models (
$defs/$ref), optional fields or value limits are now constrained too: the structure, types, enums and required keys are enforced, and a value limit the grammar can't hold is left to the caller; never a refusal. A long JSON answer decodes without drafting, so alone it is slower (about 52 instead of 123 tokens per second on Qwen 3.8).DARKBLOOM_QWEN_JSON_SCHEMA_REWRITE=offrestores today's behaviour (#2084). - Two-Mac pairs: the link between the Macs is secured (security fix). Each connection between the two Macs of a pair now uses its own fresh keys, so traffic on the link can't be read with the pairing secret alone. A pair links only when both Macs run 0.10.13: a pair with one Mac on 0.10.12 doesn't form until both have updated (pairs update together; there is no mixed-version pair and no downgrade path for the link) (#2096).
- GPT-OSS without tools too: a looping analysis on a request without tools (or with
tool_choice: "none") now ends by opening the final answer, instead of ending aslengthwith an empty answer.DARKBLOOM_GPTOSS_FINAL_EXIT=0keeps the exit to requests with tools (#2088). - A looping thought's answer no longer times out on a busy Mac: when the loop guard ends a thought, the answer's continuation is now admitted ahead of the requests waiting in the engine's queue. With more requests than slots it used to wait behind them and end as
lengthafter the engine's 120 s admission lease, with no answer (21 of 29 Gemma 4 exits on the 0.10.13 DevNet build at 12 requests on 8 slots).DARKBLOOM_LOOP_EXIT_THINKING=0still turns the exit off (#2100). - Gemma 4 prompt contract 2, dormant until a coordinator switch-on: a Mac can render the prompt vLLM-served endpoints render (so
usage.prompt_tokensmatches theirs: OpenRouter's "−1 prompt token" notice), but only for a coordinator request that asks for it. No request asks yet; until the coordinator switches it on, every request renders exactly as before.DARKBLOOM_GEMMA_PROMPT_CONTRACT_V2=offstops advertising it (#2098). - A looping thought's answer is no longer refused on a full Mac: the answer's continuation now takes the stopped request's place in the Mac's serving allowance. Work waiting for a slot used to take that place first, and the continuation was refused, ending
lengthwith no answer (2 of 7 Gemma 4 exits in a GPU check at 16 requests on 8 slots; 0 of 6 with this change).DARKBLOOM_LOOP_EXIT_THINKING=0still turns the exit off (#2115). - Qwen drafting around protected spans: on Qwen 3.8 27B strict tool and JSON requests, the ordinary reasoning and text may draft (MTP) again while tool frames, arguments and JSON values stay under the grammar.
DARKBLOOM_CONSTRAINT_SPANS=0turns it off (#2101). - native reasoning budgets for Nemotron Super and MiMo: a request's positive
reasoning.max_tokenscloses the thought at that many tokens and the answer continues within the same total output limit, on Nemotron Super and MiMo V2.6.DARKBLOOM_NATIVE_REASONING_BUDGET=0turns it off (#2099). - Gemma quoted numeric tool arguments: a Gemma 4 argument the model quotes is read as a number when the parameter's schema clearly asks for a number; ambiguous schemas keep the text, and strict calls are still checked.
DARKBLOOM_GEMMA_SCHEMA_COERCION=0turns it off (#2090). - Off by default — Qwen forced tool calls keep the caller's thinking setting: with
DARKBLOOM_QWEN_FORCED_TOOL_THINKING=1on a Mac, a Qwen request withtool_choicerequired or named keeps the thinking setting it asked for, and the reasoning is split off before the call is checked. Unset keeps 0.10.12's behaviour (#2091). - GPT-OSS commentary before a tool call reaches the content: GPT-OSS commentary written before a tool call and addressed to no tool is returned as
contentinstead of being dropped, streamed or not.DARKBLOOM_GPTOSS_COMMENTARY_CONTENT=0turns it off (#2092). - Qwen stray
</think>markers are absorbed: a stray</think>outside a thought no longer leaks into a Qwen answer; markers inside tool arguments stay literal.DARKBLOOM_QWEN_ABSORB_STRAY_THINK_CLOSE=0turns it off (#2093). - Nemotron Super decodes two image requests at once: a Super Mac serving at least three requests may decode two image requests at the same time (image preparation stays one at a time; video stays exclusive).
DARKBLOOM_NEMOTRON_SUPER_CONCURRENT_IMAGES=0turns it off (#2094). - Gemma reopened thought channels: a Gemma 4 thought the model reopens later in its answer goes to
reasoning_contentinstead of the answer, and stray channel-close markers are absorbed; tool arguments are never re-routed.DARKBLOOM_GEMMA_REOPENED_THOUGHTS=0turns it off (#2089). - Hardened Gemma forced tool-call constraints (#2111).
- Updater version checks (security) (#2117).
- Clearer pairing security information (#2129).
- Hardened inbound request handling (security) (#2130).
- Resource limits on queued coordinator requests (security) (#2135).
- Configuration parsing no longer accumulates memory (#2138).
- Two-Mac pairs: the link key moves to the Keychain (security). After a restart the pair links once someone has logged in to the Mac (
darkbloom cluster statussays so); a rollback to 0.10.12 through the updater keeps the pair working (#2137). - Qwen 3.5 9B and 35B now end a repeating thought in an answer by default, as Gemma 4 and GPT-OSS do: when the loop guard stops a verbatim loop inside the thought, the provider closes it as Qwen's template does (
\n</think>\n\n) and the model answers in the same response, instead of ending aslengthwith no answer. On GPQA's looping questions (OpenRouter's settings, temperature 0.5) the 35B answered 14 of 27 such thoughts (6 right) and the 9B 11 of 29 (3 right); the others looped again and still end aslength. Requests where the guard does not fire are unchanged. Only these two builds: Qwen 3.6, Qwen 3.8 27B and any other build keeplengthunlessDARKBLOOM_LOOP_EXIT_THINKINGnames them (…,qwen3_5_moe,qwen3.6);DARKBLOOM_LOOP_EXIT_THINKING=gemma4,gemma4_text,gpt_ossturns Qwen's off (#2125). - Off by default — GLM-5.3-Flash JSON answers under a grammar: with
DARKBLOOM_GLM_JSON_ANSWERS=1on a Mac, GLM's JSONresponse_formatanswers are sampled under the JSON grammar; unset keeps 0.10.12's path (#2114). - Two-Mac pairs: each Mac proves its identity on the link (security). Every connection between the two Macs of a pair now also checks that each Mac holds the key its pair pinned. Both Macs must run 0.10.13 to link; after a restart, the pair links once someone has logged in to the Mac (#2146).
- Two-Mac pairs: the processes a pair starts are checked before they run (security). A pair's worker and owner start only if they are exactly the installed, pinned executable (#2139).
- Rollout: a patch release over 0.10.12 with only the fixes below; nothing else from
devrides. Pairs: both Macs must run the same version, and a pair on 0.10.12 updates itself.darkbloom update --rollbackputs back 0.10.12. - The registration keeps 0.10.12's fields and cluster runtime facts;
libs/mlx(72dffa2d),libs/mlx-swift(b1c94d29) and the Xcode 27.2 toolchain are unchanged, somlx.metallibis unchanged. - The 50 entries below are the pending fragments rendered verbatim, except where the release notes reword one; CI-only entries change no provider behaviour.
- Known issue: a two-Mac pair that serves only you (no Darkbloom network) and is updated by hand can stay on the earlier release's cluster service after the update and stop pairing. To recover, run
darkbloom cluster stop --local-only --serviceon each Mac, thendarkbloom starton both; the pair comes back on 0.10.13. Pairs that serve through Darkbloom update together at their slot. - Known limitation (since 0.10.0, unchanged in 0.10.13): Nemotron 3.5 Super is served only by the provider daemon:
darkbloom start --local --here(the standalone server) skips it ("it loads only in the provider daemon"). To use Super on your own Mac only, run the daemon with its local endpoint instead (darkbloom start --foreground --local-endpoint).
A pair being set up is no longer called stopped while the leading Mac downloads
- During
darkbloom clustersetup, the Mac that joined ended its screen about 20 s into the leading Mac's download (or a long copy) of the chosen model with "Stopped (bydarkbloom cluster stopon one of the Macs): the pair isn't starting any more", though nobody had stopped anything and the leading Mac's start went on (0.10.9 to 0.10.12). It took the leading Mac's status, which reads "Stopped" until both Macs' parts start, for a stop. The leading Mac now counts each stop someone asks for (darkbloom cluster stopon either Mac, anddarkbloom stopordarkbloom restartonce the pair runs) and its status carries the count; the Mac that joined ends only on a stop asked since the start began, and then at once instead of 20 s later. A leading Mac on an older build sends no count, so the Mac that joined follows its start to the end. - On the leading Mac, such a stop now ends the start at once too: during the model's download or copy (what arrived stays, and the next start goes on from there; before, both Macs were started once it finished), while both Macs load the model (before, a ✗ with Retry after 15 minutes), and at the test (before, "✓ Serving" or "the test failed"). The registration no longer starts again a pair stopped during the test. The pair's own stops (the registration's restart, the cluster service's restart for a changed approval) no longer end the leading Mac's wait for the coordinator with that line.
Cluster launch check (security)
- The worker and owner processes a two-Mac pair starts run only if they are exactly the installed, pinned executable. Details will be published after the fleet has updated.
Cluster link: each Mac proves its identity
- Every connection between the two Macs of a pair now also checks that each Mac holds the key its pair pinned (security). Details will be published after the fleet has updated.
- After a restart, the pair links once someone has logged in to the Mac; it resumes by itself.
Cluster link: fresh keys for every connection
- The link between the two Macs of a cluster pair now uses fresh keys for every connection, from an X25519 key agreement made in memory on both Macs and authenticated with the pairing's key. This covers everything the link carries (worker commands including prompts, outputs, bootstrap rounds, leases, status), for
darkbloom cluster local/start --local --distributedand coordinator-formed pairs. Every reconnect makes new keys. - The link handshake is now version 3, with no fallback: a Mac still on an older release cannot join its pair until both Macs run this release (the pair's update cycle updates both); the refusal reads
protocolVersionMismatch.
Cluster pairing: the link key in the Keychain
- A two-Mac pairing's link key is now kept in this Mac's Keychain; an earlier release's copy moves there at the first start of this release.
- After a restart, the pair links once someone has logged in to the Mac;
darkbloom cluster statussays so, and the pair resumes by itself after the login. - Rolling back to 0.10.12 with
darkbloom update --rollback(or an automatic rollback) keeps the pair working; a 0.10.12 installed by hand asks to pair the Macs again.
Clearer pairing security information
darkbloom cluster pairlabels the other Mac's security details as what that Mac reports, not as verified facts.
Faster recovery from a dead coordinator link
- The provider now finds a silent coordinator link in 6 to 8 s and reconnects at once: it pings every 2 s and declares the link dead when no frame (pong, ping or message) comes in for more than 6 s. Before, a link that died with no reset and no close frame, for example on a coordinator node restart, took up to 40 s to notice.
- After a session that stayed open for 60 s or more, the first reconnect waits 0 to 0.25 s instead of 0.5 to 1 s; repeated failures still back off as before.
- A busy stream never trips the bound, because every inbound frame counts. A Mac that slept for 30 s or more reconnects once at wake, as before; a shorter delay of the ping timer is not counted as missed pongs.
Resource limits on queued coordinator requests (security)
- The provider bounds the coordinator requests it holds while they wait for admission; ordinary serving never reaches the bound. Details will be published after the fleet has updated.
Gemma 4 keeps a string parameter's unquoted 01000 as "01000"
- Gemma 4 sometimes writes a string argument without its quote marker. Every such value was read as JSON, so a string parameter's
01000reached the client as the number 1000,1.10as 1.1,007as 7 and a 20-digit id as an inexact number, and astricttool failed the call partway through the stream. - Such a value is now read by its parameter's type, at any depth, as vLLM does: a parameter that takes a string keeps the text the model wrote, and number, integer and boolean parameters read it as before. The type is the one Gemma is shown: a parameter declared without a type (or as
{}) is shown as a string and a union as its first member's type, so an unquoted5there now stays "5" where vLLM, reading the declared schema, returns 5; both are valid for the declared schema, except for anenumthat mixes types on a parameter shown as a string (untyped, or a union that lists the string first, asLiteral["a", 1]declares it), where an unquoted1now reads "1". A value in quote markers stays a string even for an integer parameter, as before. The SDK pin moves to mlx-swift-lm main with this change.
Gemma 4 JSON answers follow the requested format
- A Gemma 4 26B request with a JSON
response_format(json_object, orjson_schemawithin the provider's supported subset) is now sampled under a grammar: after the model's thought, if any, the answer is exactly one JSON value of the requested format, with no code fences, surrounding prose or trailing text. Schemas as Pydantic, OpenAI's strict mode or zod write them ($defs/$ref, optional fields asanyOfwith null,$schema) are enforced as written; ajson_schemathe grammar still does not cover is held to a JSON object when its root is an object. The thought is unchanged. - Requests the grammar does not cover (tools, a
tool_choiceother thanautowithout tools,stopsequences, images, other schemas) and cluster pairs are answered as before; an answer cut atmax_tokensstill ends aslength, never as an error.DARKBLOOM_GEMMA_JSON_ANSWERS=offturns the grammar off on a Mac.
Gemma 4 prompt contract 2 (vLLM's prompt), dormant until the coordinator asks
- A provider now registers Gemma 4 prompt contract 2 and renders it for a coordinator request that asks for it: the system message, and list-shaped user content, as the chat template's text parts, which end in a space before the system turn closes. For system and user content that is the prompt vLLM-served endpoints render, so
usage.prompt_tokensand the prompt the model reads then match theirs (OpenRouter's GPQA prompt: 88 tokens, not 87). Assistant and tool content renders as before. - No request asks yet: the coordinator asks only for models it switches on, and only for requests it planned and counted with contract 2. Until then every request renders exactly as before, as do the local API, media requests and other models.
DARKBLOOM_GEMMA_PROMPT_CONTRACT_V2=offstops advertising contract 2 on a Mac.
Gemma quoted numeric tool arguments
Adopt the SDK parser fix, on by default since 0.10.13 (measured on Gemma 4 26B in its ON state; the parser is shared by every Gemma 4 size). DARKBLOOM_GEMMA_SCHEMA_COERCION=0 restores 0.10.12's strings (the LaunchAgent forwards the switch to the installed service). Numeric marker strings become numbers at frame close only when the parameter schema unambiguously requests a numeric type. Potentially converted values stay withheld from argument deltas until the completed parse; unsupported or ambiguous schemas preserve their quoted text. The provider still validates strict calls.
Gemma reopened thought channels
On by default since 0.10.13 (measured on Gemma 4 26B QAT in its ON state; the change is in text parsing, the same for every Gemma build). A Gemma 4 thought the model reopens later in its answer goes to reasoning_content instead of the answer, and stray channel-close markers are absorbed. The collected path shares the streaming state machine. Native, JSON and escape-quoted tool argument data stays opaque to channel routing. The same parsing also changes four ordinary cases slightly: an empty thought adds no reasoning text; a bare thought prefix is stripped from streamed answers; collected answers keep text that precedes the first thought and absorb a close with no opener; and <turn|>/<eos> before any channel are dropped. DARKBLOOM_GEMMA_REOPENED_THOUGHTS=0 darkbloom start restores 0.10.12's parsing (the LaunchAgent forwards it; for a foreground process, set =0).
Hardened Gemma forced tool-call constraints
- Gemma 4's constraint for a required or named
tool_choiceis built within tighter resource limits for complex tool schemas.
GLM-5.3-Flash JSON answers can be grammar-constrained
- With
DARKBLOOM_GLM_JSON_ANSWERS=1, a GLM-5.3-Flash request with a JSONresponse_format(and no tools) is sampled under the same JSON grammar as Qwen, Bonsai and Lightning answers: the answer is one JSON value of the requested schema, so a missing required key, an invented key or a wrong type can no longer be generated (about 7 % of GLM's answers missed under batching, each a 422 after the streamed reasoning). - Requests the grammar does not cover keep today's prompt-steered, checked answer; a cluster pair is never constrained. The provider's MLX runtime now accepts token constraints on GLM's native engine.
- A constrained GLM request decodes without speculative drafting (MTP), and so does a request sharing a two-request decode step with it (GLM drafts with at most two); it also reuses no cached prompt prefix of another JSON request.
GLM JSON answers: an optional release of schema misses
- A GLM-5.3-Flash JSON
response_formatanswer is checked before it is exposed. One whose JSON value misses the caller's schema has been refused with a 422 after the streamed reasoning; it still is by default. DARKBLOOM_GLM_JSON_SCHEMA_MISS=releaseexposes that value instead when the model stopped by itself (fences and surrounding prose removed), so the caller's own validator judges it. Output with no complete JSON value, and output cut atmax_tokens, are handled as before. It applies to streamed responses, which every coordinator request is; a non-streamed local API request still gets the service's own schema check.
GLM no longer fails requests on idle Macs whose KV pool kept its free memory
- GLM's native paged KV pool kept every segment it ever committed until it was
unloaded, and each prefix-cache restore attached more. On a 256 GB M5 Ultra
that free memory stayed charged until a request arriving at an idle Mac could
not reserve its fixed memory and first step's scratch beside it and failed as
capacity_busyright after admission (about 3,960 GLM requests a day), which
also put the Mac into the coordinator's capacity cooldown. - The SDK now returns the pool's free segments before retrying such a request,
and after the engine has been idle 30 seconds; a request that arrives while
the previous one is still publishing its prefix checkpoint waits up to 10
seconds for it instead of failing.DARKBLOOM_NATIVE_SEGMENT_RELEASEsets the
idle seconds, and any value that is not a positive number below a day (0,
off) restores the former behavior; it is forwarded to the launchd provider
job.
A stray <tool_call> no longer fails a GLM-5.3-Flash response
- GLM's tool parser reads a response from its first
<tool_call>to the end as one batch of calls, and onautotool requests that includes the model's reasoning. A tag the model only mentioned or a call it drafted in its thought, or a text answer that mentions the tag, failed the whole response with a 500 (native GLM tool output is invalid). - Such output is now text instead of a failure, as vLLM's and SGLang's GLM parsers never fail a response on tool syntax, with a stricter rule for what is still a call: the calls are the frames that end the response, when all of them are well-formed; everything before them is text, which the service's reasoning parser splits at
</think>. A frame inside or after another frame's arguments, or after arguments that break the parser's grammar, is never a call. Output with no such calls (a malformed or cut-off final call, a malformed call among the final ones, text after the calls) is all text, and so is output cut off atmax_tokenswhose tool suffix does not parse. No call is completed or salvaged. - Still a 500, as before: output whose released text would hold a think marker from inside a tool frame (the reasoning parser would read it as the end of the thought), and a tool suffix over the parser's 1 MiB bound. Output the parser cannot read safely, such as a thought that spells out the tags in order (
<tool_call>,<arg_key>,<arg_value>) or a refused call with quotes before the final calls, recovers no call: it is text, or still a 500 when the thought's</think>comes after it. Required and namedtool_choicestill fail closed (422).DARKBLOOM_GLM_TOOL_TEXT_FALLBACK=0restores the 500 for every unparsed suffix;darkbloom startpasses it to the launchd provider.
GPT-OSS commentary before a tool call reaches the content
- GPT-OSS: a recipientless commentary preamble (Harmony's short message before a tool call) now reaches
content, streamed and collected, instead of being dropped. Tool calls are unchanged: a recipient before or after the channel marker stays a call, and incomplete, oversized, non-assistant and body-embedded commentary headers pass as before. On by default;DARKBLOOM_GPTOSS_COMMENTARY_CONTENT=0restores the 0.10.12 behavior.
GPT-OSS answers under a final header with attributes reach the client
- GPT-OSS can open its answer with Harmony header attributes, as in
<|channel|>final <|constrain|>json<|message|>, typically for JSON answers. The provider's Harmony split kept only an exactfinalheader, so such an answer was dropped and the client got emptycontent. It now arrives ascontent, as any other answer does. - Nothing else in a well-formed GPT-OSS response changes: reasoning, tool calls, preambles and plain answers are byte-identical.
GPT-OSS honors tool_choice "required" and a named function
- A GPT-OSS 20B request with
tool_choice"required"or{"type": "function", "function": {"name": …}}was refused (400tool_choice_unsupportedfrom the coordinator). It is now served: the model reasons first, at the request'sreasoning_effort, then the provider lets it write only a call to a declared function (the named one, or any declared one forrequired), with arguments that match that function's schema. The reasoning arrives asreasoning_contentand the call astool_calls, withfinish_reasontool_calls. - One call per response: GPT-OSS's format ends the response at its first call, so
parallel_tool_callschanges nothing. - The reasoning before a forced call ends at 16,384 tokens, whatever
max_tokensallows: a model still reasoning then writes its call, so a reasoning loop no longer runs to the context limit. - A named function that is not declared is still a 400, and a schema outside the forced-choice subset (for example
$reforanyOf) is still refused, as for the other models, before any generation.autoandnoneare unchanged. - The Mac advertises the capability for
gpt-oss-20b, and the coordinator routes these requests only to Macs on this release or later, never to a cluster pair.DARKBLOOM_GPTOSS_FORCED_TOOLS=off(orfalse,0,none,no,disabled) turns it off on a Mac (it also reaches a provider started bydarkbloom start).
Grammar-aware MTP
- Adopt SDK #409: constrained rows draft with MTP and every committed position is verified under the row's grammar. On by default since 0.10.13 for the measured Qwen 3.8 27B build only (
EigenLabs/Qwen3.8-27B-4bit-mtp), where constraint spans are on (measured together); Gemma 4 and Super keep target-only constrained rows unlessDARKBLOOM_CBV2_MTP_GRAMMAR=1.DARKBLOOM_CBV2_MTP_GRAMMAR=0(orDARKBLOOM_CONSTRAINT_SPANS=0) restores 0.10.12's target-only constrained rows. Forwarded by the LaunchAgent.
A past turn's reasoning sent back as reasoning or thinking reaches the model
- A client that sends an earlier assistant turn back as OpenRouter returned it carries that turn's thought in
reasoning; some clients usethinking. The provider read onlyreasoning_content, so the thought was dropped: Qwen 3.8, Bonsai 2 27B and MiMo, whose templates replay every earlier thought, saw an empty thought on each of those turns, and Qwen 3.5/3.6 and Gemma 4 lost the thoughts of the tool calls since the last user message. - Darkbloom requests and the local API now read such a turn's
thinkingorreasoningas itsreasoning_content, as Hugging Face's chat template rendering does: the first non-null ofreasoning_content,thinkingandreasoningwins. An empty or non-string value changes nothing.
Hardened inbound request handling (security)
- The provider applies stricter limits when it parses inbound requests and tool schemas. Ordinary requests are unaffected. Details will be published after the fleet has updated.
Valid JSON answers on Bonsai 2 27B and Nemotron 3.5 Lightning
- A tool-free JSON
response_formatrequest on Bonsai 2 27B or on Nemotron 3.5 Lightning, on one Mac, whose prompt
starts in the answer (thinking off, the JSON default on Bonsai), is now sampled under the JSON-answer grammar the
Qwen 3.5 family already uses: for schemas inside the constrained subset the answer is one JSON value of the
requested schema (no extra or missing keys, no raw newline inside a string). On our checks, Lightning answers with
thinking off missed the schema 8 times in 72 before and 0 times after. - Requests whose prompt opens a thought (Lightning's default), requests served by a cluster pair, and the Bonsai 2 9B
keep today's prompt-only steering. - Like Qwen's, these answers decode without MTP (a Lightning row sharing their decode step stops speculating
meanwhile), and the provider checks them when generation ends: an answer that misses the schema at a natural stop
is the typed422(tool_noncompliance); one cut atmax_tokensends as a normallengthfinish with its
content as generated. - The grammar keeps an answer inside the schema, not on topic: a model that wants a forbidden key may keep writing the
string it is in untilmax_tokens. DARKBLOOM_JSON_ANSWER_GRAMMAR=qwen(or0/off) keeps the grammar to the Qwen types.
An answer cut at max_tokens ends as length, not as an error
- When generation stops at the request's
max_tokens(or this Mac's limit) during aresponse_formatanswer or a tool call, the response now ends normally withfinish_reason: "length"and its usage, instead of atool_noncomplianceerror. Before, a streaming client got an error frame partway through the answer, which OpenRouter counts as a failed response, and a coordinator request withoutstreamgot a 422. - What was generated stays as generated: a streamed call stays open with its arguments so far, left unclosed so no client can run a cut call; a JSON answer leaves as written (Qwen, Flint, Super; GLM's normalized when it already holds a whole value); and of the calls held until the end, each one that no longer passes its checks is left out. On the Mac's own API, a request without
streamwhose cutresponse_formatcontent is not one JSON value still gets the 422 that saysmax_tokenscut it off. - A
stricttool call (tool_choiceauto) that fails the provider's own check now returns as generated, as a non-strict one does since 0.10.12, once a streaming response has started. Before anything was sent, and for a request withoutstream, it is still refused, so the request can go to another Mac. Forced choices,noneandparallel_tool_calls: falsekeep their checks. DARKBLOOM_LENGTH_FINISH_AT_LIMIT=0andDARKBLOOM_STRICT_CALLS_AS_GENERATED=0(oroff,false,no) restore each refusal; both reach a provider started bydarkbloom start(launchd). Telemetrytool_constraint_cut(by outcome) andtool_constraint_strict_as_generatedcount what replaced the refusals.- With or without the switches, a tool-call refusal at the end of an answer now comes before the last text the model produced instead of after it, so a refusal with nothing else sent stays one the request can be retried on another Mac.
A looping thought's answer no longer times out on a busy Mac
- When the loop guard ends a thought (Gemma 4 by default; Qwen 3.5/3.6, Nemotron 3.5 Super and GPT-OSS when their exit applies), the answer's continuation is now admitted ahead of the requests waiting in the engine's queue. On a Mac with more requests than slots it used to queue behind them and end
lengthafter the engine's 120 s admission lease, with no answer: 21 of 29 Gemma exits on 0.10.13's DevNet build at 12-on-8 concurrency.DARKBLOOM_LOOP_EXIT_THINKING=0still turns the exit off.
A looping thought's answer keeps its place on a full Mac
- When the loop guard ends a thought, the answer's continuation now takes the stopped request's place in the Mac's serving allowance. On a full Mac, work waiting for a slot took that place first and the continuation was refused, ending
lengthwith no answer: 2 of 7 Gemma exits in a GPU check at 16-on-8 concurrency. Other requests still wait for a free place. - Under memory pressure the engine evicts an ordinary running request before a continuation (that request recomputes its work so far).
DARKBLOOM_LOOP_EXIT_THINKING=0still turns the exit off.
A GPT-OSS analysis that repeats itself now ends in an answer on requests without tools too
- When the loop guard stops a verbatim loop inside GPT-OSS's analysis on a request without tools (or with
tool_choice: "none"), the provider ends the analysis and opens the final channel (<|end|><|start|>assistant<|channel|>final<|message|>), and the model writes its answer in the same response, instead of ending aslengthwith an empty answer. Left to pick its next channel (the exit for requests with tools), the model reopened its analysis and looped again on every GPQA question that looped. - On GPQA Diamond in OpenRouter's request shape (
reasoning_effort: high, temperature 0, 66 questions), unanswered questions fell from 39 to 0 and accuracy rose from 35 % to 58 %. Requests where the guard does not fire are unchanged (token-identical when sent one at a time). - The request's stop strings apply to the answer from its first token. The usage counts every token produced, as one completion. A loop in the answer still ends as
length. - On by default.
DARKBLOOM_GPTOSS_FINAL_EXIT=0keeps the exit to requests with tools;DARKBLOOM_LOOP_EXIT_THINKING=gemma4,gemma4_textturns GPT-OSS's exit off altogether.
A GPT-OSS analysis that repeats itself verbatim now ends in an answer or a tool call
- As for Gemma 4: when the loop guard stops a verbatim loop inside GPT-OSS's Harmony analysis channel, the request no longer ends as
lengthwith no answer and no tool call. The provider ends the analysis message (<|end|>) and starts the next assistant message (<|start|>assistant), as the model does after its own analysis, and the model writes itsfinalanswer or its tool call in the same response. The usage counts every token produced, as one completion. A loop after that still ends aslength. - In OpenRouter's Tau2-Bench airline run on 0.10.12 (
reasoning_effort: high, temperature 0), 27 % of GPT-OSS's turns ended that way, at 4,097 tokens with nothing but reasoning. - On by default for
gpt_ossrequests that declare tools, the shape of Tau2's turns: on a GPU A/B of 126 Tau2-Bench airline turns atreasoning_effort: high, temperature 0, the turns that ended with nothing fell from 74 to 38 (calls 31 → 46, replies 21 → 35), and 7 more got a retryable memory-admission 503 after their retries (the same build needed such retries with the exit off too). Without tools the model mostly opened a new analysis after the exit and looped again, so those requests keep thelengthstop. Never with logprobs or on a cluster pair.DARKBLOOM_LOOP_EXIT_THINKING=gemma4,gemma4_text(a list withoutgpt_oss) or0restores thelengthstop for GPT-OSS.
A Qwen 3.5 thought that repeats itself now ends in an answer
- When the loop guard stops a verbatim loop inside a Qwen 3.5 9B or Qwen 3.5 35B thought, the provider now closes the thought as the model's template does (
\n</think>\n\n), and the model writes its answer in the same response, instead of ending aslengthwith no answer. Gemma 4 and GPT-OSS already did this. - On GPQA Diamond at OpenRouter's settings (temperature 0.5), on the questions whose thoughts loop: Qwen 3.5 35B answered 14 of the 27 thoughts the guard ended (6 right) and the 9B 11 of 29 (3 right); the others looped again in the answer and still end as
length. Requests where the guard does not fire are unchanged (token-identical when sent one at a time). - The request's stop strings apply to the answer from its first token. The usage counts every token produced, as one completion. A loop in the answer still ends as
length. - On by default for these two builds only. Qwen 3.6, Qwen 3.8 27B and any other build of these model families keep
lengthuntil their own A/B, whatever list a Mac sets, unless the list also names the build (…,qwen3_5_moe,qwen3.6).DARKBLOOM_LOOP_EXIT_THINKING=gemma4,gemma4_text,gpt_ossturns Qwen's exit off. - A request whose strict tool calls the provider constrains to their schemas keeps today's
lengthstop: the answer after the thought would be written without that constraint.
A Qwen 3.5 or 3.6 thought that repeats itself verbatim can end in an answer (off by default)
- Once turned on, as for Gemma 4 in 0.10.13: when the loop guard stops a verbatim loop inside a Qwen 3.5 or 3.6 35B thought, the request no longer ends as
lengthwith no answer. The provider ends the thought as Qwen's template does (\n</think>\n\n), and the model writes its answer in the same response. The usage counts every token produced, as one completion. A loop in the answer still ends aslength. - Since 0.10.12, 0.16 % (Qwen 3.6 35B) and 0.83 % (Qwen 3.5 35B) of GPQA-shaped requests ended that way with no answer.
- Off by default: the A/B on Qwen 3.5/3.6 35B found too few thought loops to show rescues (Qwen 3.6 never looped in 198 greedy GPQA draws; Qwen 3.5 35B looped 3 times, once on the candidate, whose answer looped again).
DARKBLOOM_LOOP_EXIT_THINKING=gemma4,gemma4_text,qwen3_5_moeturns it on for the 35B models;…,qwen3_5adds the 9B and Qwen 3.8 27B;0turns all of it off. Never with logprobs or on a cluster pair. - Client stop strings apply to the answer from its first token, and a stop string that ends the answer is reported as the matched stop sequence (Gemma 4's too).
A Nemotron 3.5 Super thought that repeats itself verbatim can end in an answer (off by default)
- Once turned on, as for Gemma 4 in 0.10.13 (and the Qwen 3.5/3.6 35B models, when listed): when the loop guard stops a verbatim loop inside Super's thought, the request no longer ends as
lengthwith no answer. The provider ends the thought (</think>, as Super's template does), and the model writes its answer in the same response. The usage counts every token produced, as one completion. A loop in the answer still ends aslength. - Since 0.10.12, 3.0 % of GPQA-shaped Super requests ended that way with no answer.
- Off by default until its A/B:
DARKBLOOM_LOOP_EXIT_THINKING=gemma4,gemma4_text,nemotron_h_omniturns it on beside Gemma 4's (the default); addingqwen3_5_moeturns on the Qwen 3.5/3.6 35B models' too;0turns all of them off. Never with logprobs, media or on a cluster pair.
A Gemma 4 thought that repeats itself verbatim now ends in an answer
- When the loop guard stops a verbatim loop inside Gemma 4's thought, the request no longer ends as
lengthwith no answer. The provider ends the thought (<channel|>), and the model writes its answer in the same response. The client sees the thought close and the answer start. The usage counts every token produced, as one completion. A loop in the answer still ends aslength, as in 0.10.12. - The answer gets what is left of the request's output budget, at most 8,192 tokens. If it cannot start (under 256 tokens left, the stopped row not released within 5 s, a full Mac, a cancel), the request ends as
length, as before; if it fails after its first token, the answer ends there aslength. - Gemma 4 only (
gemma4,gemma4_text). Never with logprobs or on a cluster pair.DARKBLOOM_LOOP_EXIT_THINKING=0turns it off. DARKBLOOM_LOOP_EXIT_THINKINGand 0.10.12'sDARKBLOOM_LOOP_GUARDnow reach a provider started bydarkbloom start(launchd). Before, both worked only in the foreground or with--local.
The loop guard covers image and video requests
- Image and video requests on one Mac to Gemma 4, Qwen 3.5/3.6/3.8, Qwen
Flash-Next and GLM-5.3 now get the text path's loop guard: a media answer
that repeats one block verbatim ends aslengthinstead of running to its
output cap. MiMo's and Super's media and the pairs' media routes are
unchanged. - For Gemma 4, a repeating thought on an image or video request ends the
thought and the model answers, over the same media input (exit-thinking). - On OpenRouter's VGI-Bench rerun of 11-13, 117 of 257 served Gemma video
answers ran to the 117,8xx-token cap, 116 of them with at most 71 tokens
outside the thought; the median served answer was 110K tokens long (27
minutes).
GLM-5.3-Flash and MiMo Macs no longer refuse long prompts because of a short prompt's speed
- An idle GLM or MiMo Mac priced every prompt's prefill from its last short prompts. A prompt of a few tokens takes about as long as its fixed setup, so it read as 9-140 tokens per second on GLM Macs that read long prompts at 1,400-2,000; the Macs then refused nearly every longer request as
deadline_unreachable(1,300 of the 1,452 requests of OpenRouter's Tau2 run on GLM), and a refusal measures nothing, so the estimate never recovered. - A prompt of at most 4,096 tokens may now raise that estimate but never lower it, and readings from prompts of up to 1,024 tokens no longer lower the estimate for a prompt of their size either. An idle GLM or MiMo Mac with no calibration or longer-prompt reading in the last two minutes prices prefill at no less than its own calibration, instead of re-measuring with a short prompt. A fresh slow reading from a longer prompt, or a busy Mac, still prices below it.
- Other models are unchanged.
DARKBLOOM_TTFT_NATIVE_PREFILL_EVIDENCE=0restores the previous estimate (it reaches a provider started bydarkbloom start).
Native reasoning budgets for Nemotron Super and MiMo
- Add native reasoning budgets for supported local Nemotron Super and MiMo routes. Positive
reasoning.max_tokensnow reaches the SDK sampler, closes the thought at its token limit and continues the answer within the original total output ceiling. Unsupported routes and conflicting thinking-off controls remain refused. On by default since 0.10.13 for Nemotron 3.5 Super and MiMo V2.6 (each measured in its ON state);DARKBLOOM_NATIVE_REASONING_BUDGET=0restores 0.10.12 for every family. A Super cluster pair cannot hold a budget: there the request is refused at the engine bridge (unsupported output control) instead of by the prompt contract.
parallel_tool_calls: false returns the first tool call instead of failing the response
- With
parallel_tool_calls: false, a model that made a second call failed the whole response: a streamed answer ended in an error after its first call had already been sent, and a complete answer was a 422. Under automatic tool choice the prompt does not ask for a single call (only a GLM pair's prompt does), so these were requests the provider should have served. - The response now carries the first call, as OpenAI and vLLM return it, and the calls the model makes after it are dropped, streamed or not, on every model, Nemotron Super's path included. A later frame that does not parse keeps the existing rule for malformed output: GLM, MiMo, Flint and Nemotron Super fail the response, as for any malformed frame, and a frame that never reads as a call (an undeclared name on Qwen or Gemma) stays text.
Qwen constraint spans
- Add
DARKBLOOM_CONSTRAINT_SPANSsupport for eligible Qwen strict-auto tool requests and JSON reasoning, on by default since 0.10.13 for the measured Qwen 3.8 27B build only (EigenLabs/Qwen3.8-27B-4bit-mtp, measured there in its ON state; every other build stays off unless set to1;0restores 0.10.12). Ordinary reasoning and content may draft; tool framing/arguments and JSON values remain protected. Text after a completed call may draft again. - Forward the flag into the installed provider service. Preserve existing grammar admission, masks, engine eligibility and off behavior. No full-model speedup is claimed.
Qwen forced tool calls can keep the caller's thinking setting
- Added opt-in
DARKBLOOM_QWEN_FORCED_TOOL_THINKINGfor qwen forced-call thinking. Forced required/named tool requests retain the caller’s resolved thinking setting. Nested reasoning.enabled takes precedence over enable_thinking and effort. The existing native router separates reasoning before validating a trailing tool call. Default-off behavior is unchanged.
Qwen JSON answers follow Pydantic, OpenAI and zod schemas
- A Qwen 3.5, 3.6 or 3.8 request whose
json_schemathe JSON-answer grammar used to decline is now constrained: Pydantic's nested models ($defs/$ref),Optionalfields (anyOfwith null),$schema, and value limits such asminimum,maximum,pattern,format, length or item bounds and map-typedadditionalProperties. A schema that can be rewritten into the grammar exactly is enforced as written; otherwise the structure, types, enums and required keys are enforced and the value limits are left to the model (the provider never refuses a streamed answer for them); an object schema the grammar still cannot shape is held to a JSON object. These answers used to be unconstrained and sometimes came back fenced. DARKBLOOM_QWEN_JSON_SCHEMA_REWRITE=offrestores declining these schemas to prompt-only steering.
Qwen 3.5, 3.6 and 3.8 parse tool calls after the thinking
- On Qwen 3.8, Qwen 3.6, Qwen 3.5 35B and Qwen 3.5 9B, tool calls were parsed before the thinking was split off. A
thought that mentioned<tool_call>once (for example "I'll wrap it in<tool_call>tags") made the response's real
call come back as text, with no call; a call the model wrote out inside its thinking was executed. The thinking is
now split off first, as on Bonsai 2, Nemotron 3.5 Lightning and MiMo: tool syntax in the thinking stays in
reasoning_content, and the calls after it arrive astool_calls. - A call the model writes at the very end of a thought it never closes (a known Qwen 3.5 habit) still arrives as a
call, once the output ends, and its text also stays inreasoning_content. A call followed by more thinking or by
the end of the thinking does not, nor does an earlier call that text separates from the last one, nor any call in a
thought the token limit cut off. autocalls still stream while they are generated, and requests without tools are unchanged. Forced calls
(requiredor a named function) still run with thinking off; a stray thought the model writes before one is now
returned asreasoning_contentinstead of failing the request.- A local
stream: falseresponse for these requests keeps the whitespace around its content and reasoning, as the
streamed response does.
Qwen stray markers are absorbed
- Qwen with thinking off: a stray
</think>in the answer no longer leaks intocontent. A complete stray close outside a thought is absorbed by native channel splitting, as on Nemotron 3.5; a closed-thought prompt selects that route for Qwen streams that had none (no tools, or a Qwen type outside the reasoning-before-tools split), on text and media paths; Qwen3-VL keeps its route. Partial markers at the end of a stream and markers inside tool arguments stay literal, and a missing or disabled reasoning parser gets no inferred thought state. On by default;DARKBLOOM_QWEN_ABSORB_STRAY_THINK_CLOSE=0restores the 0.10.12 behavior.
Strict Qwen tool calls always match their schemas
- On Qwen 3.8 27B, Qwen 3.6 35B and Qwen 3.5 9B and 35B, a tool call to a function declared
strict: truecould miss the caller's schema (a quoted number, a wrong enum value, a missing required argument, a value inside a$refdefinition of the wrong JSON type); the provider checks such a call when it closes, after the answer started streaming, and the response then failed partway. On one Mac the provider now constrains these calls while they are sampled, by default on Qwen 3.8 27B, Qwen 3.6 35B and Qwen 3.5 35B, and on Qwen 3.5 9B when the Mac opts in (DARKBLOOM_STRICT_TOOL_GRAMMAR=on): only a declared function, its declared parameters (each at most once, every required one), and values of the declared types and enums can be written, in the template's call format. Reasoning and text around the calls stay free (ordinary text, the reasoning and call markers and the end of the answer; other special tokens such as<|im_start|>are not sampled), text that only mentions<tool_call>stays text, and the choice whether to call, the prompt and the streamed call deltas are unchanged. Withparallel_tool_calls: falsean answer opens at most one call. - It applies to
tool_choiceauto(or absent) requests that declare at least onestrictfunction; every call of such a request is held to its schema, as the check already holds it. Local$ref/$defsare inlined,anyOfof a type andnullis supported, and a request whose schemas use a keyword the grammar cannot hold exactly (pattern,minimum,minLength, unions of several types, recursive references, ...) is served unconstrained, as before, never refused. Cluster pairs keep the previous behavior. - As with constrained JSON answers, MTP drafting stops while a constrained answer is decoding: it drafts nothing itself, and on these models no other answer decoding beside it on the same Mac drafts either. Qwen 3.8 and Qwen 3.5 9B draft while up to 8 answers decode at once, so their decode slows while a strict-tool answer is running (on Qwen 3.8 a lone strict answer decodes at about 50 instead of 120 tokens per second; with two or more answers decoding the Mac barely drafts anyway, and at four the answers beside a constrained one decode about 13 % slower): Qwen 3.8 takes the grammar by default because strict calls that missed their schemas were its main remaining failure on OpenRouter, the 9B only when a Mac opts in. Qwen 3.6 and 3.5 35B, whose cards set a presence penalty, do not draft anyway unless a request sets
presence_penalty: 0. - The coordinator can turn it off for a model on every Mac at once by forwarding the model's tools without
strict(EIGENINFERENCE_STRICT_TOOLS_UNMARK_MODELS); those calls are then also not checked against their schemas, and it does not reach a request body above 4 MiB or a Mac withDARKBLOOM_STRICT_TOOL_GRAMMAR_ALL_TOOLS=on.DARKBLOOM_STRICT_TOOL_GRAMMAR=offturns it off on a Mac,onturns it on for every one of the four builds, and a list of model types or model-id parts (qwen3.8-27b,qwen3_5_moe) picks the builds.DARKBLOOM_STRICT_TOOL_GRAMMAR_ALL_TOOLS=onalso constrains the calls of requests withoutstrictfunctions. Both reach a provider started bydarkbloom start.
Qwen 3.5 9B keeps its model card's presence penalty
- Provider 0.10.12 dropped the default
presence_penaltyof 1.5 from the catalogQwen3.5-9Bfor requests that send none. In production its answers withmax_tokensof 8,192 or more then ran to the limit about three times as often (5.5 % against 1.7 % on 0.10.11), and JSONresponse_formatanswers ran out and failed about twice as often (51 % against 23 % in the same 10-minute windows). 0.10.13 restores the model card's 1.5 for those requests, as on the other catalog Qwen 3.5/3.6 builds. - A request that sends its own
presence_penaltyis unchanged. Withpresence_penalty0 the 9B still speculates under exact MTP acceptance. A penalty row does not speculate but still donates prefix-cache checkpoints. DARKBLOOM_QWEN35_9B_PRESENCE_PENALTY=0(oroff) restores 0.10.12's default.darkbloom startpasses it to the launchd job. A coordinator registrypresence_penaltyarrives as the request's own value and wins, so while the registry sets one for the 9B the switch changes only local requests; rolling routed traffic back also needs the registry value removed or set to 0.
SDK feature defaults
- Add provider defaults for feature switches the SDK reads (
SDKFeatureDefaults). An unset switch follows the release's default, which is on only where it was measured in its ON state: per exact model id for constraint spans and grammar-aware MTP (grammar only where spans are on), per family for native reasoning budgets, process-wide for the Gemma parser switch. An operator value always wins;<KEY>=0restores 0.10.12 on every model. Injected defaults are recorded inDARKBLOOM_SDK_DEFAULTS_INJECTEDand never written into a service plist, so an installed service follows the running release.darkbloom start(andbenchmark, on stderr) printsSDK feature defaults on: …. Nothing is on by default unless its switch is listed.
Concurrent Nemotron Super images
- Nemotron Super serves two image requests at once on owners serving at least three requests (the managed slot serves up to eight). Vision preparation stays serial; native decode runs concurrently. Each full media reservation stays charged through settlement, and video stays exclusive with other media.
- On by default;
DARKBLOOM_NEMOTRON_SUPER_CONCURRENT_IMAGES=0(forwarded by the LaunchAgent) restores the 0.10.12 single-media admission.
Configuration parsing no longer accumulates memory
- Reading the provider's configuration file no longer leaves memory behind each time; the provider uses a fixed copy of its TOML library.
Tool JSON as Transformers renders it, on request
A chat request that carries "_darkbloom_template_json": "transformers" now
shows Qwen 3.5, 3.6 and 3.8, Ternary Bonsai 2, MiMo v2.6, GLM-5.3 and GPT-OSS
their tool declarations, earlier tool calls and tool results exactly as Hugging
Face Transformers renders their chat templates: members in the order the client
sent them, ", " and ": " separators, non-ASCII text and / as written,
null arguments and strict kept, and Python True/None where a template
prints a value with string. Before, they read sorted, compact JSON with
escaped non-ASCII text and without null arguments: on the Tau2 airline
prompts 523 to 666 tokens fewer than the reference. Nemotron 3.5 Lightning
takes the order. A request from the network asks only when the coordinator did
not plan it for its prompt cache, so a planned request always renders the
prompt of its plan. On the local API, DARKBLOOM_TEMPLATE_JSON=transformers
asks for every chat request. Media requests, batch requests, /v1/responses
and every request that does not ask render the same bytes as before;
DARKBLOOM_TEMPLATE_JSON_LEGACY=1 turns it off.
Updater version checks (security)
- The self-updater checks a release's version more strictly before it installs it. Details will be published after the fleet has updated.
Pull requests in this release
7 pull requests merged into dev reached master with #2086, which promoted dev up to fe3f1625e9:
- Wait for the MiMo media reservation settlement in the native audio and media HTTP tests by @davidtai in #2001
- 0.10.13 candidate (phase A) into dev, for the DevNet build by @jonathan308 in #2079
- 0.10.13 candidate (phase B) into dev, for the DevNet build by @jonathan308 in #2085
- 0.10.13 candidate (phase C: #2096 2088@001996379f 2100@2e241534b5 2098@d76366162a) into dev, for the DevNet build by @jonathan308 in #2123
- 0.10.13 candidate (phase D1) into dev, for the DevNet build by @jonathan308 in #2134
- 0.10.13 candidate (phase D2) into dev, for the DevNet build by @jonathan308 in #2136
- 0.10.13 candidate (phase E1) into dev, for the DevNet build by @jonathan308 in #2145
2 release pull requests, not listed above: #2086 prepared the release; #2044 prepared the release.
Full Changelog: v0.10.12...v0.10.13