History / Home

Wiki review: remove the dependency framing; flag the plan pages as stale Removed every non-historical 'sister project' reference (Home, Glossary, two Architecture rows, the Roadmap objective, Testbed's dependency section) and the two tracker rows that presupposed adopting another project's code - DC-085 and DC-123. ttd is built from scratch and is MIT, so there is no upstream to track and no code to adopt. Flagged rather than rewritten: Roadmap.md, the milestone pages and most tracker rows still describe the retired DatacenterEngine plan, and Architecture.md's sharding section says expert parallelism while what shipped divides layers. That is a planning decision, not a cleanup.

Pummelchen committed Sep 19, 2026

Home: drop the sister-project row, so this wiki no longer points at the runtime repository

Pummelchen committed Sep 19, 2026

The LM head comes off the device every step -- and three of the round's ideas were losses (D109) A fresh profile put the decode step at 1769 ms (mix.read 805, load 361, head 255, attn.core 202) and 0.565 tok/s. LOST AND RECORDED AS LOST: the GPU unpack for the dense tensor(named:) path (load 353 -> 534 ms); the fused int4 expert path, a wash (2.29 s/step with no slab cache, 1.575 with a 1 GB one against a 1.69 s control -- 640 per-slab dispatch-and-wait pairs eat what the kernel saves); and the first head-residency attempt, which took the node into D106's swap failure twice (31 concurrent blocks each reading the whole 1.017 GB, and UncachedFile.readData building every read twice -- 2.03 GB transient for one 1.017 GB block). Both bugs are fixed. KEPT: the LM head on the GPU from its stored bf16 (MetalBf16Matmul) -- head 255 -> 118 ms, step 1.718 -> 1.580 s in an alternated A/B, digest unchanged -- plus three small measured fixes. Default now 1.567 s/step, 0.638 tok/s, peak RSS 3.10 GB, 266 tests, 0 failures. Tests 259 -> 266; DC-117 and DC-127 carry the D109 progress; open tasks unchanged.

Pummelchen committed Sep 18, 2026

The fused int4 matmul works on the GPU and is 1.2-2.4x faster -- and the unpack was never the bottleneck (D108) MetalInt4Matmul dequantises inside the matmul, so the ~6.5 GB of Float a token materialises never exists. It is bit-identical to dequantizeInt4 followed by Ops.orderedMatmul over the D107 grid, and in release it beats unpack-then-matmul 1.23x / 1.45x / 2.36x on the real gate_up, down and head-width shapes -- 0.114 s of a 1.74 s step, about 6.6%, the same order as the 7% the CPU probe bounded. mix.read is the disk read, so the route to 7 tok/s is residency, and this kernel is its precondition. The grid also found that an Apple GPU flushes a denormal product to zero where the CPU keeps it, for this kernel and for the MetalMatmul already in the tree, and no math mode changes it -- now a named test. Home and Tests: 253 -> 259 Swift tests. Tracker: DC-113 and DC-117 carry the D108 progress, DC-127 is the wiring row (the provider must hand over a packed payload, and the slot bank stops holding fp32), and open tasks 35 -> 36.

Pummelchen committed Sep 18, 2026

The packed slab cache works and loses: this node's constraint is memory (D106) 45% hits and 1.08 -> 0.65 GB/step at 1 GB of cache, mix.read 0.80 -> 0.54 s -- and the step gets WORSE (1.931-2.022 against 1.735) because load nearly doubles and head grows on a machine swapping 1.94 of 3.07 GB. The fourth mechanism to reach that verdict after D89, D98 and D105, which makes the pattern the finding: every cache-more route on an 8 GB Mac mini is now measured and closed. Default 0, SHARD_SLAB_CACHE_MB is the knob. The remaining CPU lever is not materialising fp32 at all (DC-120), which costs no residency.

Pummelchen committed Sep 18, 2026

The predictive prefetch is built and it loses -- because the reads are saturated (D105) On 1.859/1.816 s/step against off 1.854/1.721, with mix.read LARGER when it is on (0.830/0.813 vs 0.790/0.760) and the digest identical. D101's fan-out removed the latency problem, so the device is saturated and a second read stream takes bandwidth from the first. SHARD_EXPERT_PREDICT=1 enables it; the default is off. DC-120's direction is sharpened: the bank holds [Float], so 512 MB buys ~61 of the ~773 slices a token asks for -- the same bytes packed as int4 buy four times as many, which is the next lever.

Pummelchen committed Sep 18, 2026

The contract matmul is threaded on width-aligned chunks: 0.443 -> ~0.55 tok/s (D104) Every chunk boundary is a multiple of four, so a non-final chunk covers whole four-wide groups with no tail and the final chunk performs the serial body's own out % 4 tail -- the rule the first attempt was missing. attn.core 0.493/0.495 -> 0.209/0.202, mix.gateup 0.224/0.225 -> 0.077/0.073, mix.down 0.102 -> 0.049/0.047, step 4.124/4.103 -> 1.866/1.797 s, digest identical in all four arms. DC-122 is done, both halves.

Pummelchen committed Sep 18, 2026

The head's blocks fan out: 1.7x on the phase, 0.443 tok/s (D102) A whole vocabulary block of rows is the unit, so nothing is regrouped and only the thread changes -- legal where the general matmul's column-range split was not. head 0.434/0.439 -> 0.259/0.255 s/step, digest identical in all four arms; step 2.246 s = 0.443 tok/s. DC-124 is upgraded from unexplained to reproduced-on-the-baseline: the heavy gate-test pair fails when adjacent and passes alone, identically with and without D101/D102.

Pummelchen committed Sep 18, 2026

The expert misses are fanned across threads: 0.296 -> 0.400 tok/s (D101) preload(experts:shape:) is called once per layer where the router's choices are known, and the adapter fans the misses into the bank as staging. The warm reads do not count as requests but DO count their volume. Bank 512 MB + fan-out: 3.383/3.382 -> 2.503 s/step, mix.read 1.671 -> 0.761, digest identical in all four arms; the bank alone is 4.109, so the win is the concurrency. New row DC-124 records an intermittent abort seen twice and never reproduced, with the isolation attempts the disposition rests on.

Pummelchen committed Sep 18, 2026

The element-wise passes are threaded: 0.242 -> 0.290 tok/s, digest identical (D99) decodeRaw is a map, so threading it cannot change a value: 4.119/4.162 -> 3.448/3.465 s/step, load 1.097 -> 0.356, trace digest identical in all four runs. The matmul was attempted in the same round and REVERTED -- it moved bits and the repo's own reused-buffer comparison caught it, so DC-122 carries the failure rather than the speed-up.

Pummelchen committed Sep 18, 2026

The expert bank has a lifetime now, and the measurement says capacity is not the lever (D98, DC-119) ExpertBank holds slices for the whole generation with hits, misses, elements read and resident peak reported for the first time. Then five alternated 24-step runs gave 0.0% hits at 0, 512 and 1024 MB with identical 29,192.4M elements read: 773 requests per step of 12.5 MB is a 9.66 GB working set against a 537 MB bank, a reuse distance nine times its capacity. Default is 0, as D89 did for the layer cache; the levers are hiding the reads and cutting their latency.

Pummelchen committed Sep 18, 2026

v1.0.0 published and verified, and one blemish recorded rather than rewritten (D95, DC-115) The release is live with the archive and its checksum; the notes quote b6ebfd1b861a1ac4..., the .sha256 beside the archive matches, shasum -c passes on the downloaded bytes, and all three shipped binaries answer 1.0.0 and report arm64. The archive also carries two TEST bundles because the first version took whatever the build directory held; the selection is now plan-driven and DC-115 says so.

Pummelchen committed Sep 17, 2026

First release v1.0.0: the identity, the release script and the measured numbers (D95) VERSION is the authority and Version.swift is generated from it; every tool answers --version; the release script runs the gates, does a clean scratch build with a warning scan, asserts lipo -archs on the binaries inside the archive, and refuses notes that do not quote the digest it computed. Measured: 0.23 tok/s prefill and 0.23 tok/s decode on one Mac mini M2, 0.41 tok/s decode across four (1.7x).

Pummelchen committed Sep 17, 2026

The head is vocabulary-parallel: 1.13x -> 1.36x (D93) Each node computes its own vocabulary rows with the block decomposition unchanged, then the slices are gathered so every node holds the identical full logits array. Step 4.790 -> 4.021 s, head 1.045 -> 0.26-0.32 s, all four nodes identical. The gap to 1.5x is 0.36 s/step; the exchange counter still does not reconcile with the phases.

Pummelchen committed Sep 17, 2026

The exchange is 99.6% waiting, and the waiting is the farm (D92) Segments: encode 0.002 s, send 0.007 s, receive 1.11-2.31 s, merge 0.002 s, covering 100.0% of the total. Four interleaved contiguous/round-robin runs (1.06/0.96/0.86/0.88x) falsify the imbalance hypothesis: the slowest cluster step is the run with a peer at load 18.7. A 1.5x figure needs a quiet window.

Pummelchen committed Sep 17, 2026

The cluster's cost is the exchange (D90); the sister's ANE/GPU split studied (D91) Decode GEMV removes load rather than caching it; ANE prefill is opt-in, fp16 and must verify the device it got; the exchange blocks on peers one at a time, forty times per token. DC-113 added.

Pummelchen committed Sep 17, 2026

The decoded-layer cache works and on this node it loses (D89) Alternated on one binary: 256 MB neutral, 1 GB 5.46 s/step, 2 GB 5.60 s/step against 5.33 uncached, while load falls as designed. Memory is the constraint, so the default budget is 0 and SHARD_LAYER_CACHE_MB is the knob.

Pummelchen committed Sep 17, 2026

The cached step's own breakdown: 83% is materialising weights, a third is really I/O (D88) mix.read 33.0%, load 30.5% (CPU dequantisation of constants, not I/O — the payload is read once), head 19.2%. Perfect free sharding of the expert reads is worth 1.09x on the measured step, so the attack is DC-052's decoded-weight budget.

Pummelchen committed Sep 17, 2026

The first phase breakdown: reads are 64.5% of a forward (D86, D87) mix.read 37.6%, attn.core 23.6%, head 13.5%, load 13.5%. Perfect free sharding of the expert reads is worth only 1.39x, so the 3x target cannot come from the expert plan; the lever is the replicated reads, the attention core and the read path.

Pummelchen committed Sep 17, 2026

M3's first measurement (0.93x) and the staged-path bug it found (D83, D84) The documented cluster command staged the install under one name and launched with another, so it failed on every peer after copying 21.7 GB to each. Fixed with one shared rule and four tests. The measurement, once fixed: bit-identical on all four nodes, 0.93x, with the exchange budget that explains it.

Pummelchen committed Sep 17, 2026

Documented commands are checked against the tools now (D81) 45 documented flags, all valid, and no dead code among the 50 non-test tools. The new gate reports a tool it cannot ask as NOT CHECKED rather than as a pass.

Pummelchen committed Sep 17, 2026

Verify the install's 693 payload digests, which nothing had compared (D79) 21.7 GB in 26.69 s at 29.4 MB peak, zero problems, disk and swap unmoved. The milestone check now runs it by default, with --sample to bound it and the verifier's output inherited as the evidence.

Pummelchen committed Sep 17, 2026

Gate the install verifier; fix the false positive it found (D78) verify_install only ever ran by hand, and the milestone check now runs it before trusting a digest. The first run reported the checked-in fixtures as mis-tiled: the rule demanded exact contiguity while the format is 64-byte aligned. ALIGNMENT now has one home and the rule is the format's.

Pummelchen committed Sep 17, 2026

Home.md carried a false status; the claims gate now discovers its documents (D77) Three narrownesses hid it: the page was not a claim document, its phrasing did not match the patterns, and the claim was wrapped inside a blockquote. The widened gate found both false counts before they were corrected, and now checks 148 claims against 126.

Pummelchen committed Sep 17, 2026

Tracker holds open items only; the dated record moves to News The tracker had grown to 82 task rows of which 43 were Done, plus a 200-line dated log inside a page whose stated job is what is NOT finished. It is a tracker again: 39 open rows across the phases, each phase keeping a one-line index of what closed, and the record -- 112 dated entries, newest first -- is on the new News page, linked from Home and the sidebar. Corrected in the same pass, because these were wrong rather than merely stale: the tracker's own header still claimed "no source code exists at all"; its status row still quoted the RETIRED 0.0191 tok/s and a 0.0000 cache-hit rate as if current; its counts disagreed with its own rows; P1 was listed In progress although M0 is complete; and DC-004's status paragraph still described the warn-and-skip CI that DC-103 replaced with a hard failure.

Pummelchen committed Sep 16, 2026

Tracker: DC-104 -- M0 is complete, and six documents said otherwise docs/m0-gate.md records status: passing on Qwen/Qwen3.5-2B with 40,683,520 bytes of trace data identical to the contract and every discrete decision matching the reference. Every M0 task is Done, so the answer to "what is next for M0" is nothing -- and the work became correcting my own status rewrite, which called it "fixture scale" without reading the gate doc. The correction also names the 2B-vs-~1B deviation and the renegotiated oracle half.

Pummelchen committed Sep 16, 2026

Tracker: DC-103 -- the toolchain is enforced, and the old check was a no-op check_toolchain.py refuses anything but Xcode 27 / Swift 6.4, the CI job has no skip branch (expected red until an image carries Xcode 27), and Home says so. The bug: the replaced step read `swift --version | tail -1` -- the target line -- so its pattern could never match and it skipped on every runner, correct toolchain included.

Pummelchen committed Sep 16, 2026

Wiki: the status claim corrected on Home and Roadmap, and it now names the toolchain Home said "no source code, no build system and no release" -- the extreme AGENTS.md had already recorded as wrong. Roadmap said "nothing on this page is built yet ... design phase". Both now state what is measured: Swift 6.4 / Xcode 27, clean build, 105 Swift tests, 163 stdlib Python tests in CI; M0 met at fixture scale, M1 run on the real model with its gate open; no releases, no tags. DC-102 records it, and the trap entry now requires any status claim to name the toolchain.

Pummelchen committed Sep 16, 2026

Baselines measured, target models verified, and two arithmetic hazards DC-019: the numbers the plan depends on, measured rather than assumed, plus the three target models read out of their own checkpoints. Measured (Testbed, "Measured baselines"): - 4 KB round trip on the 1 Gbit segment: 645 us over TCP, 687 us over UDP, against a 515 us empty ICMP round trip. The transport is base-RTT-bound, not bandwidth-bound, so the 4 KB payload is nearly free and UDP buys nothing. - Internal SSD: 1161 MB/s sequential read, but 108 MB/s for random 16 KB reads at queue depth 1 — 9-11x slower. That gap is the entire case for a deep read queue and expert caching, and it is the number the bytes-per-token maths has to use. - ~2.5 GB reclaimable memory on an idle 8 GB node; no external NVMe attached on any node, so the assumed 3 GB/s has no hardware behind it yet. Two probes were thrown away before this: the first reported 13 us round trips (a dead echo server returning EOF instantly) and the second reported 13 GB/s reads (a 2 GB file that fit in the 8 GB machine's cache). Both are recorded as discarded, not published — a plausible wrong number is worse than none. Target models (new page): Qwen3.6-35B-A3B, DeepSeek-V4-Flash and Qwen3.8-Flash-Next read from their own config.json at a named revision, with what the brief got right and the three things it does not say — Qwen3.6 is 30 linear + 10 full attention layers, not conventional; DeepSeek has 3 hash-routed layers, a 64-head indexer and 128x128 FP8 blocks; the Qwen n-gram table is a tri-gram table of 20M entries. Two hazards, both demonstrated with numbers rather than argued: - R10: bit-identity across shard counts does not follow from "accumulate in fp32". Pre-summing each node's own experts and combining the partials is a different association order: 11185 of 20000 random top-8 draws (56%) sum differently, by exactly 1 ULP. The M2 gate would fail on arithmetic alone. Nodes must exchange per-expert contributions and accumulate in fixed expert-id order — determinism needs a fixed order, not a ring. - R11: bf16 router logits flip the top-k set. I3 keeps routers at "bf16 or higher"; over random 256-way routers the top-8 index set changed within 22 draws (expert 102 at 5.424064 rounded to expert 85's 5.413122). Routers go through fp32 logits and fp32 top-k, with a documented tie-break. The sync-cost table on the Target models page turns R4's latency into a per token budget: on the measured link a ring is 77 ms/token at four nodes over 40 MoE layers, against 26 ms for recursive doubling.

Pummelchen committed Sep 15, 2026

Bootstrap the wiki: project pages and the task tracker The wiki held a one-line Home stub. It now carries the project's plan, its status and its design, so that no later commit has to invent them. Pages: - Home — what the project is, the expert-parallel thesis, the milestones at a glance, non-goals, the sister project and the licence. - Roadmap — P0..P6 (M0..M5) with the objective, the deliverables and the gate of each phase; the bit-identity gate at M2 is named as the real one, and the rule that a gate is renegotiated openly, never quietly. - Project Tracker — the tracker the repository did not have: 51 stable DC-nnn tasks across seven phases, a phase/status board, nine risks and traps, seven open questions, the closed record and a parked list. DC-001 (this bootstrap) and DC-082 (the README/wiki/tracker sync rule) are the only two closed items; no source code exists yet. - Architecture — the design as written: expert versus pipeline parallelism, the four invariants (bit-reproducibility, discrete decisions, faithful ports, transcode-don't-requantize), the role-based IR and thin importers, the shard policy, transport, and the stack. - Testbed — the target configurations, what a node must provide, a readiness checklist and the sister project as a dependency. - Glossary — model, distributed, runtime and project terms. Also the sidebar and footer navigation, in the style of the sister project's wiki. Deliberately absent: host names, addresses, credentials and any measured number. Nothing in the repository runs yet, so every page states that the claim is design intent, and the risks say which ones are unproven.

Pummelchen committed Sep 15, 2026