v0.34.1-dynres — fold of upstream v0.34.1

Fork fold of upstream ollama v0.34.1 (tag 38fdb5dd5, 23 first-parent commits) on top of v0.34.0-dynres. Upstream's release notes describe upstream's work; this lists only what the fork adds on top, and what it measured. Record: docs/maxusai/tasks/upstream-sync-0.34.1.md. PR #302.

Payload pins — all three moved: llama.cpp b10760 → b10864 (5d806aa25), MLX ce916dbb → d9add9d1, MLX-C c74db530 → ebc88f10 plus upstream's carry patch mlx/compat/0001-mlx-c-qmm-global-scale.patch, which lifts the #4458 wall the 0.34.0 fold stopped at. A native rebuild was required; no binary swap was valid for the gates.

Decisions recorded in the fold

• D1 — upstream's pins and its MLX-C carry patch, as upstream carries them.
• D2 — port to upstream's scoped array lifetimes (Scoped, ScopedEval, held scopes) instead of reintroducing Pin/Unpin/Sweep; three fork files needed it. The runner logs one memory line per request: peak, and held — MLX's active memory once the request's scope ended and the cache was cleared.
• D3 — accept upstream's deletion of convert/; GGUF conversion is llama.cpp tooling.
• D4 — the memory measurement moves to held. The per-array trace listing is gone with the lifetime API; runnerlog.py reads the memory line, and summarize_retained_memory.py prints held, its step, and — when the trie's trace line is logged — held − trie.
• D5 — the drafting-under-grammar knob stays (OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0, ADR 0033); see the memory finding below.
• D6 — x/structured retired. The pure-Go constrained sampler (ADR 0009/0013, 3,829 lines) was tested against upstream's xgrammar engine: 108 verdicts, 0 regressions, one finding — an allOf with several branches enforces nothing on xgrammar 0.2.5 — pinned by x/mlxrunner/xgrammar/engine_behaviour_test.go and recorded in ADR 0033's amendment. Deleted.
• D7 — v0.34.2 is its own fold; it moves the MLX engine out of x/. Its 5-line ec3cc2307 (release MLX's freed-buffer pool on crossing a 256-token boundary — speculative rounds step over an exact multiple) is carried here on evidence, below.

Ten conflicted files, in short

x/mlxrunner/client.go (kept the fork's admission and NewClient, took upstream's integrated-GPU bound into admit()), pipeline.go (fork's guardClose teardown and stop sequences on upstream's scoped loop), runner.go, llm/llama_server.go (upstream's token-repeat abort: limit 30 → 100 and an HTTP 500 instead of a silently truncated 200), server/routes_*_test.go (gguftest fixtures), prefix_cache_test.go, the qqmm bench, and llama/compat/004-llama-cpp-gemma4-budget-fill.patch, re-cut for b10864 — upstream changed gemma4's set_limit_image_tokens(40, 280) to (70, 1120), exactly the endpoints of the fork's budget ladder; the fork's flags already passed those, so the served budgets are unchanged (preflight token_ladder 5/5, pinned_image_token_budget 560 → 529, ceiling 1120). Lesson recorded: apply the patch series to a checkout of the new pin before building; this fold lost a build to skipping that.

Harness and records

• docs/maxusai/retirement-register.md — every carried item with what retires it and the test that gates its deletion; README's "What differs from upstream" rewritten per capability against v0.34.1 / b10864.
• runnerlog.py: the memory line, in slog's real unquoted form (msg=memory) — the quoted-only regex read nothing from the first real log; pinned by a test. summarize_retained_memory.py: held − (trie active + paged out) when the trie line is present. 134 tests.
• Preflight expectations pin llama_cpp_build = 5d806aa25, mlx_build = d9add9d1… (full sha).
• ADR 0033 amended (parity results, the allOf limitation); ADR 0035 is the whitespace bound (#301, renumbered).

What tested green

All tables rendered by the ADR 0012 generators, one runner at a time beside production, 16 GiB overhead, cold restart per cell. Renders in preflight-runs/ on the CUDA host.

• Preflight cuda-dynres-903, from the fold worktree: PASS 21 / SKIP 7 — same as the deployed 0.34.0 image.
• GGUF (llama-server), eight models, against the 0.34.0 fold: no scored cell regressed. gemma4 e2b and e4b — flagged as regressions on 0.34.0 — recovered (e2b: scene IoU 0.061 → 0.496, invoice 1/5 → 5/5, fine text 0/0/0/0/0 → 4/4/3/1/0; reproduced 4/4). Two deterministic output changes on the b10864 payload, both 4/4 across repeats and neither a scored-cell regression: e2b's multi_3img_anchored now loops past upstream's raised repeat limit and returns the new HTTP 500; gemma4:26b-a4b lost the bcreasoning and bcpinned contract flags (four flags gained elsewhere).
• Qwen2.5-VL, six GGUF cells, against v0.34.0-dynres: every quality cell identical, contract matrices identical.
• MLX-CUDA, five nvfp4 models, against the deployed 0.34.0 build: every scored cell equal within 0.003 IoU, contract matrices identical row for row. The one cell that moved at n = 1 (qwen3.6:35b-a3b name_bbox 0.613 → 0.504) is bimodal across cold loads on the new image — 0.504, 0.504, 0.613, 0.613 over four reps — with the deployed value one of the modes.
• No OOM, no unconverged cell in any valid campaign. Throughput columns were not read: the campaigns shared GPU0 with foreign work, and the first model into a fresh PTX cache pays the JIT.

Preflight, from the recorded run of the deploy candidate:

surface	Build identity	Image size ladder	Pinned image budget	thinking on/off	Output quality	fp16 overflow canary	Runner isolation	measured on
cuda	green	green	green	green	skipped	green	green	0.34.0-dynres-6-gfb18f5c
mlx-cuda	not run	not run	not run	not run	not run	not run	not run	—
mlx-metal	not run	not run	not run	not run	not run	not run	not run	—
apple-silicon-mlx	not run	not run	not run	not run	not run	not run	not run	—
rocm	not run	not run	not run	not run	not run	not run	not run	—
cpu	not run	not run	not run	not run	not run	not run	not run	—

Generated by release_matrix.py from recorded preflight runs. A surface with no run for this release reads not run — absence is shown, never assumed green. A group is reported at its weakest check, so one skipped probe does not read as a pass.

The preflight and the campaigns ran on 0.34.0-dynres-6-gfb18f5c, the candidate built from this fold. This tag differs from it by x/structured's deletion (dead code, no importers), ec3cc2307, a comment, and records.

Memory: the drafting finding, re-measured, and what changed

Four trace-level runs of the same suite (qwen3.6:35b-a3b-nvfp4, 28 requests), resident MLX memory after each request's teardown minus the prefix trie's own accounting:

build / arm	resident − trie, growth over 28 requests
deployed 0.34.0, drafting on	+1.8 GiB
this fold before ec3cc2307, drafting on	+5.3 GiB
this fold with ec3cc2307 (carried), drafting on	+2.4 GiB
this fold, OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0	+0.1 GiB, flat

The trie is exonerated on both builds (same 5.6 GiB everywhere). The free-pool cadence upstream fixed in v0.34.2 was the part new in 0.34.1, so ec3cc2307 is carried; what both builds share — drafting under a grammar retaining live memory on the recurrent-state models — remains, and the knob removes it at the cost of drafting speed on grammar requests only. Quality is unchanged either way. Six custom GPU kernel backend disabled … reason="no source" warnings on CUDA are upstream's shape (Metal-only kernel sources), not a fold gap.

Not in this release

• The Metal half: MLX and MLX-C moved, so the Metal payload changes too; held.
• Any deployment. The fork's production endpoint still runs v0.34.0-dynres; this tag's image is built and gated separately before it replaces it.
• Upstream v0.34.2 (MLX engine out of x/, llama.cpp b10969): a structural fold of its own.

v0.34.0-dynres — fold of upstream v0.34.0

Fork fold of upstream ollama v0.34.0. Upstream's release notes describe upstream's work; this lists only what the fork adds on top, and what it deliberately did not take.

Payload pins: llama.cpp b10760 (0f3a71be1) and MLX-C c74db5307cc8 are unchanged from v0.33.3-dynres; MLX moves to ce916dbbcaa8. MLX_VERSION is the only pin file that differs between the two tags, and no patch changed — 002/004/005/801/903 apply unchanged, as CI's patches jobs assert on every platform.

Decisions recorded in the fold (task doc docs/maxusai/tasks/upstream-sync-0.34.0.md)

• D1 — a structured-output request drafts again, as upstream intends, and the fork's gate becomes a knob. Upstream 4986e923 lets a constrained session run speculative decoding; the fork's speculation.open refused to. Everything else in the fold matches main on these gates, so the choice came down to that one line, and it went upstream's way. OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0 restores the fork's gate; an unrecognised value keeps the default and warns rather than silently halving structured-output throughput. ADR 0033 records the divergence.
• D2 — MLX at ce916dbb for the CUDA completion-worker busy-loop fix (ml-explore/mlx#4452). Before it an idle MLX runner held a full core at 100 % with the GPU at 0 %. The pin cannot advance past ollama#4458 without an MLX-C bump: gather_qmm gained a global_scale parameter that MLX-C c74db530 does not pass.

Three merge conflicts, and how they went

• server/sched.go, runnerRef.LogValue — kept the fork's leaf logMu over upstream's refMu.TryLock(), which drops name, inference, pid and num_ctx at the eleven log sites that already hold refMu. Upstream's slices.Clone of the GPU list inside the lock adopted. The fork's own test caught this one.
• x/mlxrunner/client.go — took upstream's requestGrammar, which sends structured output as an xgrammar structural tag, per ADR 0033.
• x/mlxrunner/speculate.go — took upstream's accept(), kept the fork's comments, and rewrote the park() note that claimed a constrained session never drafts. The parked step must still be masked, and it still is.

Harness

• cuda-dynres-903 now pins mlx_build, and the check can assert it on a CUDA run: a CUDA preflight loads llama.cpp models, so the MLX engine-init line it used to read never appears, and pinning alone turned a skip into a failed run. probes.mlx_build_payload reads the version out of the shipped libmlx.so, with the live line still preferred because only a load catches binary/payload skew, and the verdict names which source answered.
• The drafting and memory analysis from this fold are vision-suite generators now (summarize_drafting.py, summarize_peak_memory.py, summarize_retained_memory.py, summarize_output_lengths.py), sharing one runner-log parser.
• ADR 0012 §4 amended: on MLX, think-off is not bit-reproducible either. Prefix-cache state, cold loads and timing-driven drafting each move a think-off cell, so an MLX cross-build claim needs three or more repeats on both builds. A single-run fold gate had duly read six single-run differences as a regression.
• ADR 0012 rule 11 added: a counted cell states its scale, and takes it from the data.
• ADR 0034 amended: the admission headroom cannot price untracked MLX memory.

What tested green

Quality, against main, with every table rendered by the ADR 0012 generators:

• GGUF (llama-server), eight models: every quality cell identical to main — scene grounding, document extraction, OCR tiers, multi-image, and answer-token counts. The contract matrix is identical. Only throughput and latency differ, and that campaign shared its host at a load average near 100.
• MLX-CUDA, five nvfp4 models: the contract matrix is identical to main, 50 of 50 cells. Two scene-IoU cells move by ≤ 0.003 and one answer length by 12 tokens — inside the spread two runs of a single image already show.
• No OOM, no error, nothing left unconverged in either campaign. Structured output behaves: "json" and a JSON Schema return 200, "yaml" returns 400, no format returns 200.

Preflight, from the recorded run:

surface	Build identity	Image size ladder	Pinned image budget	thinking on/off	Output quality	fp16 overflow canary	Runner isolation	measured on
cuda	green	green	green	green	skipped	green	green	0.33.3-dynres-28-gfbedf50
mlx-cuda	not run	not run	not run	not run	not run	not run	not run	—
mlx-metal	not run	not run	not run	not run	not run	not run	not run	—
apple-silicon-mlx	not run	not run	not run	not run	not run	not run	not run	—
rocm	not run	not run	not run	not run	not run	not run	not run	—
cpu	not run	not run	not run	not run	not run	not run	not run	—

Generated by release_matrix.py from recorded preflight runs. A surface with no run for this release reads not run — absence is shown, never assumed green. A group is reported at its weakest check, so one skipped probe does not read as a pass.

The preflight and the campaigns above ran on 0.33.3-dynres-28-gfbedf50, the deploy candidate built from this fold. Its runtime differs from this tag only by the D1 knob (x/mlxrunner/speculate.go), which is off the default path unless OLLAMA_MLX_DRAFT_UNDER_GRAMMAR is set. The matrix reports the build it was measured on rather than the tag, because that is what was measured.

Known caveat, not new in v0.34.0

Speculative drafting leaves MLX memory that no tracked array owns, on the recurrent qwen3.5 family — one whole recurrent state (48 layers × 3 MiB = 144 MiB) at a time, growing across requests and invisible to the admission headroom. Main's own drafting does the same on format-less requests, so v0.34.0 does not introduce it; D1 extends its reach to structured output. The knob limits that reach; it is not a fix. The mechanism is narrowed but not yet pinned to a line, so nothing has been filed upstream.

Not in this release

• The Metal half: the mlx-metal profile, its vision goldens, and the Metal gates.
• Any deployment. The fork's production endpoint still runs v0.33.2-dynres.1.
• MLX past #4458, which needs the MLX-C bump described in D2.

v0.33.3-dynres — fold of upstream v0.33.3

Fork fold of upstream ollama v0.33.3. Upstream's release notes describe upstream's work; this lists only what the fork adds on top, and what it deliberately did not take.

Payload pins: llama.cpp b10760 (0f3a71be1), MLX 37c26e5755da, MLX-C c74db5307cc8. Upstream's re-cut 001 hooks patch taken; the fork's 002/004/005/801/903 apply unchanged at the new pin (verified on a real checkout and again by CI's patches jobs). 903 stays: llama.cpp#27044 is still open, and b10760 widens MMQ to MoE on pre-DP4A cards. The qwen2.5-vl fp16-overflow gate (GGML_CUDA_CUBLAS_COMPUTE_TYPE=f32, ollama#18070 still open) survives the bump and passed its canary on the new payload.

Decisions recorded in the fold (task doc docs/maxusai/tasks/upstream-sync-2026-09-04.md)

• D1-A — the fork's gemma4 MLX vision stays. Upstream v0.33.3 ships its own gemma4 MLX vision and audio tower, but its PrepareMedia has no per-request image-token budget (a fixed per-checkpoint soft-token set), which would make image_max_tokens a silent no-op on MLX gemma4 — what ADR 0021 forbids. Upstream's audio.go, process_audio.go, process_image.go (+ tests) are excluded; gemma4 audio is not shipped and stays suppressed in /api/tags/show. Adopting upstream's tower with the budget seam re-grafted is a separate measured spike.
• D2 — ADR 0010's arithmetic on upstream's plumbing. Upstream's IncludeIntermediateMetrics / firstPassMetrics / prompt_eval_cached_count are all in; the thinking→content transition prefers the runner's own pass-one report (reportedPassMetrics) and falls back to ADR 0010's textual reconstruction; done-time summing stays ADR 0004's (upstream's reclassification of the continuation prefill as generation time was not adopted, so recorded think+format tok/s do not move).

MLX runtime

• Upstream's single-buffer MLX-C error contract replaces the fork's mlxCall; mlx.ClaimOSThread is gone — ADR 0017's guarantee is now carried by upstream's mlxthread.Start (ADR 0017/0021 carry dated status notes).
• The fork's VRAM/cache ceiling (OLLAMA_MLX_MEMORY_LIMIT, MemoryLimit/SetMemoryLimit/SetCacheLimit) ported onto the new contract, symbols confirmed in the regenerated bindings; the $ORIGIN RPATH and libquadmath bundling that make the CUDA payload loadable are kept alongside upstream's new cusolver/cusparse/nvJitLink.
• The dead pure-Go constrained-sampling layer (ADR 0033's follow-up) is deleted (#267).

Harness and identity

• ADR 0032 amendment: point tags between folds (v0.33.2-dynres.1 marks what was actually deployed); the two lineage preflight profiles admit -dynres.N stamps (#265).
• release_matrix.py's "Output quality" column keyed on a check name the harness never emits and read green for every run; fixed with tests, README matrix regenerated honestly (#266).
• cuda-dynres-903's payload_pin moved to 0f3a71be1 after the three arch ladders were re-measured byte-identical to the b10630 rows.

What tested green

CUDA preflight on the tagged build (full run, --quality): see the matrix below. Gate 5 spot-check (five think-off nvfp4 cells vs the sync15 CUDA baselines): no systematic movement; four cudaMallocAsync OOMs under production traffic on the shared GPU re-ran 12/12 clean with OLLAMA_MLX_MEMORY_LIMIT=40 GiB (#272 documents the knob).

Not in this release

• The Metal half (mlx-metal-0-33-3 profile, vision goldens, mlx0333nv1) — handoff in the task doc.
• /api/generate intermediate metrics (#271), the D1-B spike, and the qwen3.5-MoE MMQ functional gate at b10760 (source-inspected only; Blackwell outside the widened path).

What tested green (generated, from preflight/runs/full-0333-tagged.json)

surface	Build identity	Image size ladder	Pinned image budget	thinking on/off	Output quality	fp16 overflow canary	Runner isolation	measured on
cuda	green	green	green	green	skipped	green	green	0.33.3-dynres-0-g0c4f09d
mlx-cuda	not run	not run	not run	not run	not run	not run	not run	—
mlx-metal	not run	not run	not run	not run	not run	not run	not run	—
apple-silicon-mlx	not run	not run	not run	not run	not run	not run	not run	—
rocm	not run	not run	not run	not run	not run	not run	not run	—
cpu	not run	not run	not run	not run	not run	not run	not run	—

Generated by release_matrix.py from recorded preflight runs. A surface with no run for this release reads not run — absence is shown, never assumed green. A group is reported at its weakest check, so one skipped probe does not read as a pass.

v0.33.2-dynres — fold of upstream v0.33.2

Fork fold of upstream ollama v0.33.2. Upstream's release notes describe upstream's work; this lists only what the fork adds on top.

Payload pin: llama.cpp b10630. All seven compat patches revalidated at the new pin.

Vision correctness

• qwen2.5-vl fp16-overflow gate. The vision tower's f16-weight matmul at v.blk.31.ffn_down runs through cuBLAS with 16F compute on CUDA; a handful of elements out of millions reach inf, NaN the image embeddings, and the caption collapses into one repeated glyph. The launcher injects GGML_CUDA_CUBLAS_COMPUTE_TYPE=f32 for qwen25vl runners. Offered upstream as ollama#18070; this drops from the fork when that lands.
• Public reproducers for the class — generated checkerboards and one public-domain photograph, each with a healthy paired control differing by a single parameter, covering both the clip path and the pre-0.30 Go engine. No customer data involved.

Instrumentation

• 801-clip-node-stats-meter (new 8xx diagnostics band): an env-gated per-node meter over the clip graph, inert unless OLLAMA_CLIP_NODE_STATS is set. It exists because this class of fault is invisible from the product surface — garbled text out, a few overflowing elements in.
• Preflight poison_probe now corroborates at the node when the meter is enabled, catching overflow that happens not to garble the text.

Carried fixes

• 903-fix-mmq-ids-padding — MMQ ids-path tail padding sized from ne11 instead of the flattened row count; under MoE broadcast the buffer gets no padding and the kernel overruns. Still required at b10630 (verified); tracked as ggml-org/llama.cpp#27044.

MLX

• Adopts upstream's MLX grammar engine, retiring the fork's own constrained-sampling layer (ADR 0033). The fork's version was measured inert; upstream's is simpler and async. The guarantee it protected — a format the runner cannot honour is an error, never a silently dropped constraint — is retained and re-tested.

What tested green

Built as maxusai/ollama:sync-0.33.2, stamping 0.33.2-dynres-5-g2b95b4a.

surface	Build identity	Image size ladder	Pinned image budget	thinking on/off	Output quality	fp16 overflow canary	Runner isolation	measured on
cuda	green	green	green	green	green	green	green	0.33.2-dynres-5-g2b95b4a
mlx-cuda	not run	not run	not run	not run	not run	not run	not run	—
mlx-metal	not run	not run	not run	not run	not run	not run	not run	—
apple-silicon-mlx	not run	not run	not run	not run	not run	not run	not run	—
rocm	not run	not run	not run	not run	not run	not run	not run	—
cpu	not run	not run	not run	not run	not run	not run	not run	—

Generated by release_matrix.py from recorded preflight runs. A surface with no run for this release reads not run — absence is shown, never assumed green. A group is reported at its weakest check, so one skipped probe does not read as a pass.

Generated by release_matrix.py from recorded preflight runs. A surface with no run for this release reads not run — absence is shown, never assumed green. A group is reported at its weakest check, so one skipped probe does not read as a pass.

CUDA: VERDICT PASS — 20 PASS / 1 SKIP. The remaining skip is qwen35, which
is structurally unable to pin an image budget rather than untested. The pin move forced a deliberate
re-baseline: preflight first returned PASS=18 / FAIL=1 with payload_pin as the
sole failure, and the ladders came back byte-identical to the b10488 rows, so
only the payload identity changed — no expected value was edited.

This is also the first CUDA run to exercise the fp16 canary, and the first
anywhere with node-level corroboration: the meter was enabled and reported
clean over 33 nodes, so the overflow class is asserted at v.blk.31.ffn_down
itself rather than inferred from the decoded text.

Still unvalidated: every other surface. mlx-metal is being run separately;
mlx-cuda, rocm and cpu have no run against this release and the table says
so rather than leaving them blank.