Skip to content

Releases: Lexus2016/LLM-Fusion

v0.1.28 — Synth scratchpad-leakage fix + fusion-coder 4-model panel

Choose a tag to compare

@Lexus2016 Lexus2016 released this 08 Jul 20:55

Fixed

Fusion synth no longer leaks its own scratchpad reasoning into the answer. The synth prompt told the model to "write the single best final answer" but never forbade it from narrating its own synthesis process ("Expert 1 says X, Expert 2 says Y, I will provide...") into the visible response. On harder tasks that scratchpad consumed the entire token budget, leaving the real answer truncated or missing outright.

Reproduced on 6/15 benchmark tasks across two independent fusion-coder panel configurations — dragging fusion's bench average to 23.60/30, below two of its own panel members. On the 9/15 tasks where the leak did not fire, fusion averaged 29.56/30, beating every panel member including solo glm-5.2 (28.11/30) — confirming the panel/judge/synth mechanism itself is sound; only the narration habit needed fixing.

Verified live against the three clearest prior failures (fs.watch platform caveats, LRU-TTL cache, Postgres isolation levels) — all three now return complete, direct answers.

Changed

fusion-coder panel: 4 models, glm-5.2 drops to judge+synth only. New panel: kimi-k2.7-code, deepseek-v4-flash, gemini-3-flash-preview, qwen3-coder-next. kimi-k2.7-code is this panel's weakest voice — prone to long rambling generations that eat its own token budget; no per-panel-member request_overrides exists yet to throttle it. gemini-3-flash-preview's previously-documented thought_signature 400 on foreign tool-call history did not reproduce in a live spot-check, but this isn't exhaustive.

Added

bench/fusion-bench.mjs hardening: --resume support, a JSON-Lines scorer with fault isolation (a bad line no longer loses a whole task's score), per-answer length capping, and per-task running stats. Full benchmark result files committed for provenance.

See CHANGELOG.md for complete details.

v0.1.27 — mistral-large-3 coder panel (262K advertised window) + canonical [DONE] framing

Choose a tag to compare

@Lexus2016 Lexus2016 released this 08 Jul 13:29

Follow-up release to v0.1.26: both changes came out of cross-provider post-release reviews of that release.

Changed

  • fusion-coder panel: gpt-oss:120bmistral-large-3:675b. gpt-oss (131K ctx) capped the advertised fusion-agents/fusion-coder context window at 131K while every other member has ≥262K — the advertised window is now 262144 (the kimi bound). Selected against the live Ollama Cloud roster with four filters: ctx ≥ 262K; Western lineage (the panel keeps its 2-Chinese-labs + 1-Western decorrelation — Mistral); accepts foreign tool-call history (panel-shaped probe: 200 in 7 s, correct minimal-fix answer; gemini-3-flash still 400s on thought_signature); deliberation calibre (675B MoE). Rejected: nemotron-3-super (120 s probe timeout), gemma4:31b (calibre); runner-up: devstral-2:123b. fusion-researcher intentionally keeps gpt-oss. Honest open item from the review: the quality delta vs gpt-oss stays unmeasured until the fusion-lift bench is re-run — bounded meanwhile (one voice of three, weighed by the judge; the synth writes the final code). Preset tables (README en/ua/ru) and the example config synced, including the example's stale pre-v0.1.23 kimi synth.

Fixed

  • Tool-turn guard emits canonical single-[DONE] framing. When the upstream ended its stream with [DONE] but no finish_reason chunk, the guard forwarded that [DONE] and then appended its own after the terminal-less recovery. Production clients were shielded by the downstream usage-injection transform, so no corruption was observable — but the guard's own output must be canonical regardless of what sits behind it. The upstream [DONE] is now swallowed; every finish branch appends its own. +1 framing test (307 total, green).

v0.1.26 — tool-turn guard: the agent loop no longer stalls on the simple route

Choose a tag to compare

@Lexus2016 Lexus2016 released this 08 Jul 12:44

Field-debugged release: three live-validated failure modes made fusion-agents stall the OpenCode agent loop ("does one step, then waits for continue"). All three are now detected and recovered on the smart simple passthrough — the route ~87 % of agent steps take.

Added

  • Single-route tool-turn completeness guard (src/strategies/tool_turn_guard.ts). Detects, on tool-carrying requests: (1) narrate-and-stop — the model describes the next action in prose and ends the turn with finish_reason:stop and no tool_calls; (2) length-cut tool calls — large-file writes truncated mid-arguments (finish_reason:length, unrunnable JSON); (3) upstream stream termination — Ollama Cloud kills generation streams at ~5 minutes. Recovery is ONE live-streamed retry (nudged to emit the tool call and write large payloads in chunks), failing open to the original response. The stream wrapper is reader-driven rather than a pipeThrough TransformStream, because flush() never runs when the source errors — a transform-based guard is structurally blind to failure (3).
  • Terminal-state instrumentation. Every tool-carrying single-route stream logs one tool-turn terminal state line (finish_reason, tool-call count, guard verdict, lengths, tail) — real-session stalls are diagnosable from the log alone.
  • Per-model request_overrides (single strategy + smart inline simple slot). Extra request-body fields merged into every upstream call; core keys (model, messages, stream, tools, tool_choice) are protected. A/B-measured flagship use: reasoning_effort:"none" stops glm-5.2 deliberating for minutes on mechanical steps (reasoning 1692→0 chars, 6 s→2 s, tool-calling intact; think:false and "low" are ignored by Ollama Cloud).
  • Fusion synth: agentic tool-action directive — when the request carries tools, the synth is told the panel's prose is deliberation, not the final shape: if the best next step is an action, emit the tool call. Inert on the tool-less research/report path.

Changed

  • fusion-agents simple route now sends reasoning_effort:"none" via the new request_overrides. The stale simple-target comment now documents the two real constraints: foreign tool-call-history compatibility and a large context window (a real session overflowed kimi's 262K — do not swap kimi in for speed).

Post-release hotfix on main

  • 8942eb7 — the guard now swallows the upstream [DONE] so its framing is canonical (exactly one [DONE], its own) even when the upstream ends without a finish_reason chunk. Found by a cross-provider post-release review; production clients were shielded by the downstream usage-injection transform, so no observable corruption occurred.

306 → 307 tests green; typecheck clean. Full details in CHANGELOG.md.

v0.1.25 — independent BinEval evaluator for the coder pipeline

Choose a tag to compare

@Lexus2016 Lexus2016 released this 07 Jul 18:40

Changed

  • fusion-coder BinEval evaluator: glm-5.2deepseek-v4-pro. After v0.1.23 glm-5.2 became the coder synth, so a glm evaluator would grade its own answers. deepseek-v4-pro sits outside the coder pipeline entirely; BinEval runs rarely (non-streaming responses only), so sharing the concurrency-capped background-traffic model is fine. fusion-researcher keeps glm (its synth is kimi — independent).
  • README (en/ru/ua): the recovery-fallback description now reflects the v0.1.24 behavior (judge, or a panel member when the judge IS the synth). Pilot/smoke benchmark result files committed for provenance.

v0.1.24 — restore synth-recovery insurance for judge===synth

Choose a tag to compare

@Lexus2016 Lexus2016 released this 07 Jul 18:38

Fixed

  • Synth-recovery fallback works again when judge === synth. v0.1.23 made glm-5.2 both judge and synth for fusion-coder, and the recovery fallback (judge !== synth ? judge : null) silently lost its cross-model insurance — an empty synth answer had no second model to recover on. The fallback now picks the first PANEL member that differs from the synth in that case. Stale "kimi is the fusion-coder synth" wording in docs/claude-code.md and the fusion-claude launcher comment updated to the post-v0.1.23 reality.

v0.1.23 — glm-5.2 synth: the fused model now beats every member

Choose a tag to compare

@Lexus2016 Lexus2016 released this 07 Jul 17:44

Changed

  • fusion-coder synth: kimi-k2.7-codeglm-5.2 — the first configuration change driven by the project's own fusion-lift benchmark (bench/, analysis in docs/llm-fusion-audit.md). As synth, kimi emitted 17–19k-char draft-style output on factual/reasoning tasks and dragged the always-on fusion composite to 0.71 — below every solo panel member. With glm-5.2 as synth the same pipeline scored 0.99 vs 0.89 for the best solo member (n=14, blind 3-axis scoring by a model outside all conditions; per-answer methodology control kept solo-glm at 0.89). The fused model now exceeds every individual member — the project's core promise, measured. kimi stays on the panel; fusion-agents inherits the change via its fusion-coder reference.

v0.1.22 — partial_coverage judge dimension

Choose a tag to compare

@Lexus2016 Lexus2016 released this 07 Jul 13:17

Added

  • partial_coverage judge dimension. The judge now also names the aspects of the request that SOME expert answers cover and others miss (and which expert covered each), and the synth is instructed to take each such aspect from the covering expert instead of averaging it away. Closes the one dimension missing versus OpenRouter Fusion's five-dimension judge (accepted item #2 of the comparative audit, verification 2).

v0.1.21 — think-tag stream filter (no more <think> leakage)

Choose a tag to compare

@Lexus2016 Lexus2016 released this 07 Jul 12:48

Fixed

  • <think> tags no longer leak across SSE fragment boundaries. The think-tag stripper ran per stream fragment, so an SSE boundary splitting the tag itself ("<th" + "ink>") leaked the literal tag into the visible output, and a block whose body arrived in later fragments leaked the private reasoning as answer text. A stateful stream filter (createThinkTagStreamFilter) now carries a possible partial tag across fragment boundaries, suppresses everything inside an open block until its close tag, strips orphan close tags, and surfaces a false-partial tail (e.g. a literal "<tho…") as text instead of swallowing it. Wired into both streaming paths — the reasoning-promotion transform (OpenAI clients) and the Anthropic stream transform (Claude Code) — with separate filter instances per source field, so an unterminated block in reasoning can never suppress real content; a reasoning-phase false partial is merged before the first content fragment, preserving order. The synthetic tail chunk before [DONE] is framed as its own blank-line-closed SSE event and carries the stream's id/model metadata (pre-tag adversarial review findings).

v0.1.20 — README refresh + Russian and Ukrainian translations

Choose a tag to compare

@Lexus2016 Lexus2016 released this 07 Jul 12:21

Added

  • Russian and Ukrainian READMEs. Full translations (README.ru.md, README.ua.md) pinned to v0.1.19, with a language switcher in all three files.

Changed

  • README refreshed for v0.1.16–19. Fixed the stale model lineup (gemini-3-flash-previewgpt-oss:120b in the fusion-coder panel; glm-5.2 on the fusion-agents simple route), documented the agent-loop reliability features (synth completeness guard with judge fallback + SSE keepalive, honest max_tokens stop_reason, per-model concurrency budgets, separated background-traffic class), updated concurrency/timeout numbers to the shipped presets, added FUSION_SYNTH_RECOVERY_PING_MS to the environment table, removed a dangling Docker cross-reference.

v0.1.19 — per-model concurrency budgets

Choose a tag to compare

@Lexus2016 Lexus2016 released this 07 Jul 12:08

Added

  • Per-model concurrency budgets. All three consilium advisors independently flagged head-of-line blocking: one global p-limit meant a burst of background small-model calls occupied the whole FIFO queue ahead of interactive fusion turns. Every real upstream model now gets its own gate in front of the global limiter (resilience.limiterFor(model)), with strict model-gate → global-slot acquisition order (uniform ordering — no lock cycle; confirmed deadlock-free in a pre-tag adversarial review). A saturated model holds at most its budget of global-queue positions. Config: upstream.per_model_concurrency (map, keyed by real upstream model name) and upstream.per_model_concurrency_default; both unset = behavior identical to the previous single global limiter. The shipped fusion.yaml caps deepseek-v4-pro (the Claude Code background-call carrier) at 2. All strategy fallbacks wire the same budgets via resilienceForUpstream, and /v1/models capability discovery is keyed too.