Results

Results

Concise answers first; the evidence is one link deeper.

Question	Current answer
Does top-1 routing increase stored capacity efficiently?	Yes: 1.86 times the parameters at 1.02 times the active parameters per token
Do experts specialize?	Yes at this synthetic scale: 0.61 against a 0.25 blind baseline
Is top-2 clearly better?	No: better fit, first held-out MLPL answers, worse validation loss, twice the work
Does more training fix low-data behavior?	No: doubling epochs at 120 examples worsens validation loss
Does more data help?	Yes: eightfold data cuts validation loss by two thirds
Does sparse dispatch save work?	Yes: 13,440 to 3,360 expert row evaluations, outputs exactly equal
Is sparse dispatch faster yet?	No: dispatch bookkeeping outweighs the tiny matmuls skipped
Are cheap experts worth it?	Sixteen rank-4 deltas cost half of four full experts and give the best validation losses so far

flowchart LR
  Q[Quality: exact match by family, validation loss] --> F[Findings]
  E[Economics: parameters, active per token, bytes] --> F
  R[Runtime: expert evaluations, latency, throughput] --> F
  D[Diagnostics: entropy, balance, KL, specialization] --> F

Loading

Pages

• Current findings: seven claims, each as evidence, interpretation, and limitation
• Resource economics: sizes, expert cost, stored versus active, quantization projections, residency, transfer estimates, each labeled measured, derived, or estimate
• Quality: exact match by task family and validation loss, including the data-scale sweep
• Generation benchmark: time to first token, throughput, tail latency
• Full results table: one row per run
• Docent results table: the campus docent runs, with the held-out paraphrase columns where value over the deterministic matcher is measured
• Live demo verification: what was checked on the published page after each deploy

Headline sizes

Model	Parameters	Active per token	Validation loss
DN01 dense	3,812	2,996	3.79
MX01 top-1 of 4	7,096	3,064	3.76
MX02 top-2 of 4	7,096	4,136	3.92
LD01 shared plus 16 deltas	6,132	3,396	3.46

See Experiments for the runs and Learn for the mechanisms.