Engineering Notes: Metal needs no M3 compiler flag
Shaders are compiled at runtime on the device, so they are already generated for
the GPU in the machine -- there is no apple-m1-baseline analogue on the GPU side,
and the runtime options are already at their fast settings. Measured with an
inf * 0 discriminator: with languageVersion 4.0 and mathMode unset the result is
0.0 (MTLMathModeFast behaviour) where .safe and .relaxed keep nan;
optimizationLevel offers only Default and Size, so there is no runtime -O3.
Precompiling would save the cold compile alone (0.86 s for the 15-module library,
0.00 s afterwards from the driver cache). Real M3 GPU gains are kernel work:
apple9 is true and apple10 false here, so the tensor-op prefill path stays gated
off, and bfloat16_t, simdgroup_matrix and simdgroup_multiply_accumulate are the
available tools.
d824afc
Engineering Notes: -O2 is the project default for the C kernels
The previous entry recorded the 1.24x the C optimization level is worth and left
it as a decision; Package.swift now sets .unsafeFlags(["-O2"]) on
TinyTitanKernelsC, so the portable build carries it and tools/build-native.sh only
adds the CPU selection. Measured default against the previous -Os artifact:
2.35 -> 1.94 ms per pass on the CPU int8 GEMV, min of six interleaved rounds,
1.21x, checksum 390266.62 both. The cost is that .unsafeFlags stops the package
being usable as a dependency, acceptable because it is an application package with
no dependents.
cae8cc0
Engineering Notes: the release build targets apple-m1, and C at -Os
Records the compiler reality behind the new tools/build-native.sh: no -mcpu means
clang's default CPU for arm64-apple-macos*, which is apple-m1, so an M3's BF16 and
I8MM go unused; and SwiftPM's swiftbuild system compiles every C target at -Os
where the older native planner used -O2. Measured on the M3, interleaved, six
rounds an arm, CPU int8 GEMV at 8 threads with an identical checksum: 2.06 min /
2.40 median ms per pass portable, 1.66 / 1.73 at -O2 on the M1 baseline (1.24x),
1.64 / 1.69 at -O2 plus apple-m3 (1.26x). The win is the C optimization level,
not the CPU selection. Making -O2 the default would need .unsafeFlags, which
would stop the package being usable as a dependency -- the same reason -O3 was
rejected in Package.swift.
c757437
Runtime-Controls: --ram is a process target with a 4G floor
`--ram-budget` (and the launcher's `--ram`) names what the whole server may
hold, not just the expert cache: the cache gets the target minus the resident
weights and a measured 512 MiB runtime reserve, and the slot count is the largest
rung that fits, so real RSS stays under the number. Measured on Qwen3.8 4-bit:
`--ram 8` is now 7.83 GiB peak instead of 11.63, and `--ram 12` is the flag that
reproduces the old 64-slot cache. Minimum 4G -- the weights plus the 8-slot
minimum cache are ~4.7G, so 1G/2G/3G are refused by the runtime and the launcher
rather than silently overshot -- and the interactive menu offers 4/8/16/32 or the
install's own profile. The Cookbook example moves to `--ram 12` so it still
selects the configuration it meant.
f54cb50
Runtime-Controls: the losing decode knobs are gone, not just off
TINYTITAN_PARALLEL_IO and TINYTITAN_PREFETCH_TOP_M are removed from the controls
table with their engine code: the parallel fill measured a wash (+0.4%, 2/3) and
the ring depth lost at every value but 1, so the profile row's depth is now the
only source and a fill is always parallel. The prefetch row records that the ring
is one read a layer and that every alternative allocation of that read was
measured negative.
Also drops the early-expert-hits clause from the decode-execution row: that path
(and the pooled layout it needed) measured a wash and is deleted, so
gpu-residency no longer has a profile that selects it on its own.
dbc8dcc
Runtime-Controls: the RAM-budget curve, and the clamp is the measured optimum
Two corrections from the 1-16 GB `--ram` sweep (Qwen3.8-Flash-Next 4-bit, short
prompt, 256 tokens, 2026-09-21). The clamp is a third of physical memory, not a
half, so this 24 GiB Mac runs 8 GiB / 64 slots rather than the row's 12 GiB - and
64 slots is the measured optimum (4.55-4.60 tok/s) while 96 gives 4.00 and 128
gives 2.26, despite the hit rate rising 0.757 -> 0.866 and reads falling
80.8 -> 44.5 GiB. The old text also called 96 slots "the fastest", which the
curve contradicts. 1 GB fails outright rather than running slowly: top-10
routing needs ten experts and 8 slots cannot hold them; the floor is 2 GB /
16 slots (2.87 tok/s).
5358728
Runtime-Controls: the bounded-I/O opt-out is rejected, not an option
The RAM budget stays enforced; the +4-5% decode the opt-out measured is not
available at the cost of an undeclared footprint. Recorded on the row.
c241898
Runtime-Controls: bounded I/O opt-out is +4-5% decode, nothing for prefill
Long-prompt rounds added (3.977 -> 4.261, 4.064 -> 4.090) and the prefill result:
no gain in 2/2 rounds, with swap growing in the page-cache arm.
3e16f4c
Runtime-Controls: bounded I/O opt-out is worth about +5% decode
Measured 2026-09-21 on Qwen3.8-Flash-Next 4-bit: three paired rounds of the
shipped bounded path against TINYTITAN_BOUNDED_IO=0 give 4.414/4.727,
4.481/4.765, 4.444/4.560 tok/s (+5.4% mean, 3/3 positive, output identical,
swap flat). The row now carries that number and the reason it is not the
default - the page cache then holds expert data the declared budget does not
count - plus the fact that read advice only earns its keep in that mode.
876ae9a
Runtime-Controls: read advice is family-dependent, so do not flip it globally
Measured 2026-09-21 with the same paired, interleaved design on both families:
turning the read advice off gains about 1% on Qwen3.8-Flash-Next 4-bit and loses
about 3.5% on Qwen3.6 35B-A3B 4-bit. Sampling `iostat -d disk0 1` during
generation shows the advice adds no device traffic at all (108.6-109.4 GiB for
~80.7 GiB of demand expert reads in every arm), so its cost is the syscall and
scheduling work of one call per missed expert, not I/O. The row now says the
default stays `default` and why, rather than inviting a global flip.
72a4abb
Runtime-Controls: Qwen3.8's prefetch rows are depth 1, re-measured 2026-09-21
The per-family table still carried 'off' for both Qwen3.8 widths, with the Qwen
3.6 series' +21.3% figure pasted into the 4-bit row's Measured column. The 4-bit
row is depth 1 and the 8-bit row inherits it: re-measured on the current engine
the ring gives 3.993 -> 4.621 tok/s (+15.7%) on a 7-token prompt and 3.627 ->
4.158 (+14.6%) on a ~500-token one, output byte-identical, after the 2026-09-05
session had found it losing on 512-token story runs. The prose no longer dates
the ring's repair to that lossy measurement, and says what the ring actually
does: reads land before demand and become hits rather than merely warming pages.
305908f
Runtime-Controls: the sampling table is per family and Qwen3.8 has two rows
Replaces the stale 'presence penalty 0.0, nonzero not implemented' line with the
real table: Qwen3.8 thinking 1.0/0.95/20/0/0/1.0 and instruct
0.7/0.80/20/0/1.5/1.0, chosen by the request's mode; min-p is 0.0 everywhere and
the filter is not implemented.
827c0e4
Tracker: TT-035 carries the three-run contract evidence, not the discarded pass
The row quoted a single run of the pre-fix instrument (13/24 carryable, 9/12
foundation). It now carries the final three-runs-per-arm numbers: memory carryable
67/71/46% (mean 61.3) and foundation 33/33/83% (mean 49.7) against the summary's
75/54/79% (69.3) and 67/58/67% (64.2), stale 15 vs 16, with the note that the
defect is structural in the logs while its score impact is noisy.
a586aa4
Tracker: TT-035 — memory keeps an amended fact beside the original under a new key
The contract master-prompt run distilled mga/* in session 1 and the amendment
under msa/calder/* in session 2, with later sessions distilling 1/0/0/0 facts, so
both values stayed live and the model answered either. Evidence and the two
candidate fixes are in the row.
539b93f
Changelog 5.9: the T7 hint is queued before the search answers
f2278d4
Changelog 5.9: the GDN pair at the slot's width, and the master-prompt benchmark
f7efe26
Notes: the GDN a/b validator refused the slot's own width (issue #16)
Recorded with the reproduction (the override added to a qwen38flash manifest,
receipt updated so the load is not refused for the edit), the fix (the check now
takes attentionBits and accepts the slot width or bf16), and the lesson: a rule
that names a slot has to be given that slot, and naming the slot's width is a
description rather than an override needing honour.
09d8aaf
Tracker round 35: TT-020's upstream ask is answered
The gate is the webserver schema, not the CLI guard; an interface literal would
also need its address in resolveLanTrust's trustedHosts. Still upstream's call,
so the row stays Blocked, with the SSH/tunnel workaround named.
1c5e456
Tracker round 34: TT-018's catalogue half is done
PR #5396 merged 2026-09-19 by fkysly (merge commit 4d136c1). The row now says
what is left: the npm publish (operator), and an inert fork that needs the
delete_repo scope to remove.
b3c4e36
Tracker: drop the Task numbers section
The numbering rule and the type/status/size legends were a second copy of what
`docs/task-table-standard.md` already defines, and the intro links it. The page
is now the table and nothing else, which is what a queue should be.
ef9fb1d
Tracker: adopt the one-table standard (Owner column, Blocked means external)
The table now matches `docs/task-table-standard.md`: same columns, plus Owner,
and the same status vocabulary. The only status that moved is TT-018, from Open
to Blocked -- under the standard an Open task is startable in this checkout, and
neither the catalogue merge (a maintainer) nor the npm publish (the operator) is.
Owner names who must act: `maintainer / operator` for TT-018, `upstream` for
TT-020. Nothing is actionable here right now, so the table is honestly two
blocked rows rather than a section marked Open.
33961e8
Tracker: one task table with a status column
Open, Blocked and Parked were separate tables with three different column sets:
Blocked had no Size and no Next step, and a task changing state had to move
between sections. They are one `## Tasks` table now, with a Status column
carrying Open/Blocked/Parked, so state is a field on the task and every row
reads the same. A blocked row states its blocker in the next-step cell.
No status changed in the merge: TT-018 stays Open (a maintainer or the operator
can act) and TT-020 stays Blocked (upstream schema change), and both rows carry
their current evidence -- PR #5396 open and CLEAN, discussion #7111 unanswered.
929dcbc
Tracker round 33: TT-018's fork goes when the PR resolves
The `Pummelchen/awesome-dsh-plugin` fork is only PR #5396's head, so TT-018 now
carries its cleanup: delete the fork once the PR is merged or closed. Recorded
rather than acted on -- the PR is open, CLEAN and waiting on a maintainer, so
removing the fork now would close the very PR the task depends on.
6681bd6
Tracker round 32: TT-025's closure made canonical
TT-025 is now a first-class entry under "Closed by measurement -- do not
re-propose", not a sentence in that section's intro: the conversion-time
question (which precision-sensitive tensors to keep at 8 bits rather than 4),
the plan that named k_proj/v_proj at 16 MB, the two end-to-end results that
close it (18/20 vs 18/20, and -0.009727 +/- 0.006665 nats, t -1.46), what was
therefore not built, and an explicit do-not-revive-without-a-125B-machine note.
The first check's "what it does not establish" now points at the sharper
perplexity check instead of saying it was never run, which was the last
sentence that could read as if the task were still open.
139e6c1
Tracker round 31: TT-025 closed by measurement
Both quality instruments now agree. The twenty-prompt checkable suite is 18/20
against 18/20 for the promotion and its --no-promote control, and the paired
held-out perplexity A/B puts the 16 MB promotion 0.0097 +/- 0.0067 nats ahead
(t -1.46) and the whole-slot 8-bit build 0.0012 nats ahead (t -0.09) over 1,023
positions -- in precision's favour and below the instrument's ~0.013-nat
floor. The runtime half ships and is tested; the converter policy and the GDN
a/b kernel's int8 branch are not built, and the Notes now say so where someone
looking for a closed quantisation idea will find it.
The Open table is TT-018 alone, TT-020 is the only blocked item, and Parked is
empty.
c0da11d
Tracker round 30: TT-033's caller lands, TT-025's sharper check finds the same floor
TT-033: `MemoryRetrievalHinter` is T7's caller -- a background sweep that runs
only in the idle window and leaves a ranking hint for a later search, never a
request. The Notes record the design, its three bounds (64 facts a question,
16 questions tracked, a value fingerprint against staleness) and the seven
behaviours the unit suite pins. The row is closed and deleted from Parked,
which is now empty.
TT-025: `TinyTitanBench cpu35ppl` scores a fixed held-out text through the CPU
forward pass, and the paired comparison over 1,023 positions gives the 16 MB
k_proj/v_proj promotion 0.0097 +/- 0.0067 nats ahead of its --no-promote
control (t -1.46) and the whole-slot 8-bit build 0.0012 nats ahead (t -0.09).
Both point estimates favour precision and neither clears the instrument's
~0.013-nat floor, so the task stays open with that recorded; the Notes also say
what a paired t over correlated positions does not prove.
Also in this round: TT-009's side-engine model policy is written down (2B
unused, 4B 4-bit the default and verification instrument, 9B optional on
quality benchmark results), and TT-021, TT-022 and TT-023 are closed -- no
other machines for chip validation or ANE across generations, and no disk for
the ~360 GB bf16 reference long-context parity needs -- with the caveats they
existed for left standing in Blocked work.
5df1fd9
Tracker round 29: TT-025's quality case is measured, and it is not there
The measured question was whether a per-tensor promotion inside a 4-bit install
buys anything end to end. On a twenty-prompt checkable suite: the 4B's own 16 MB
`k_proj`/`v_proj` promotion is 18/20 against its uniform control's 18/20 with
identical outputs; 4-bit vs 8-bit is 18/20 vs 17/20 on the 4B and 18/20 vs 18/20
on the 9B. The failures are reasoning, not precision.
The mechanism still ships (the runtime half), and the Engineering Notes entry
records the instrument, the controls and what a wash at this resolution does not
prove. What is left of the task has no measured justification, and the 125B that
would test the slot-reading tensors needs a 360 GB bf16 source this machine
cannot hold.
02d938b
Tracker round 28: TT-033's lexical half is fixed; the remainder is semantic
The token ranking's failure was measurable and, it turns out, cheap to half-fix.
It was never failing on questions that use the store's words (10 of 10 at rank
1) — it failed on paraphrases (1 of 4), and by handing rank 1 to a common word:
"does it ever rain in this town?" matched `setting/town`'s key for 3 and the
rain rule's value for 1 while "town" appears in two facts and "rain" in one.
Weighting each term by its inverse document frequency, and a value match at two
rather than one, takes paraphrase recall@1 to 3 of 4 with no regression on the
mechanical set. The last miss shares no term in any form, which is the boundary
of what lexical scoring can do: from here it is an embedding scorer or T7.
The row now says that, rather than parking on price alone.
c001042
Tracker round 27: TT-025's runtime half ships — the three slot-reading families take a per-tensor width
The row said the runtime sizes every tensor in a slot and rejects an override.
That was true for exactly three families, and worse than the row claimed: the
loader never validated those tensors' widths at all, so an override on them
would have been *unguarded* as well as unhonoured — packed at one width, read at
another, no error.
They now resolve through the manifest's overrides (`hyperConnectionWeightBits`,
`pleKeyWeightBits`, `qsaIndexerWeightBits`), each with a role so a partial
promotion is refused, their pipelines are built, and the quant-support guard
checks the resolved width rather than the slot. That is what makes the ~10 MB
`precision_probe.py` flags affordable, against +2.10 GB for the whole attention
slot.
Left on the row: the converter policy (per tensor — the 8-bit indexer alone was
measured and rejected), the GDN a/b pair (int4-or-bf16, refused by name when
quantized), and an install carrying one of these overrides to verify on the 4B
and 9B. Both GDN points are the same gap seen from the two ends.
2ef3ea6