Skip to content

EPIC: R0 Experiment Matrix — 5 waves to break ARCHITECTURAL_FLOOR_BPB #502

@gHashTag

Description

@gHashTag

EPIC: R0 Experiment Matrix — 5 waves to break ARCHITECTURAL_FLOOR_BPB

Anchor: phi^2 + phi^-2 = 3 · TRINITY · NEVER STOP

Why

Текущий честный потолок (champion_lock.txt) = BPB 2.2393 @ 27K, seed 43, sha 4c0b04c, corpus=fineweb-style. Gap to Gate-2 (1.85) = +0.3893 BPB. ALPHA cohort (seed 43 81K @ BPB 2.1919) дискредитирован #109 (corpus=tiny_shakespeare). Без расширенного sweep не пробьём 2.19 (ARCHITECTURAL_FLOOR_BPB в tri-gardener::ledger).

Matrix (34 runs)

Wave A — seed-baseline на честном corpus (12 runs)

seed corpus hidden LR optimizer attn steps dtype
42 fineweb 384 0.002 adamw 2 27 000 bf16
42 fineweb 384 0.003 adamw 2 27 000 bf16
42 fineweb 384 0.004 adamw 2 27 000 bf16
42 fineweb 384 0.005 adamw 2 27 000 bf16
43 fineweb 384 0.002 adamw 2 27 000 bf16
43 fineweb 384 0.003 adamw 2 27 000 bf16
43 fineweb 384 0.004 adamw 2 27 000 bf16
43 fineweb 384 0.005 adamw 2 27 000 bf16
44 fineweb 384 0.002 adamw 2 27 000 bf16
44 fineweb 384 0.003 adamw 2 27 000 bf16
44 fineweb 384 0.004 adamw 2 27 000 bf16
44 fineweb 384 0.005 adamw 2 27 000 bf16

Acceptance: champion-lock confirmed на fineweb; LR-sweet-spot выявлен. Pre-registered: best_bpb < 2.40 для seed 43 LR ∈ {0.002, 0.003}.

Wave B — width ladder (8 runs)

seed hidden LR steps
43 828 0.002 27 000
43 828 0.002 50 000
43 828 0.003 27 000
43 828 0.003 50 000
43 1024 0.002 27 000
43 1024 0.002 50 000
43 1024 0.003 27 000
43 1024 0.003 50 000

Pre-registered: hit ≤ 2.10 BPB на h1024@50K (под architectural floor).

Wave C — precision ablation, seed 1597 matched-pair (10 runs)

baseline rng1597 record: BPB 2.5449 на h1024 LR 0.0020 muon @ 27K (14 canon-плато).

dtype optimizer
bf16 adamw
bf16 muon
fp32 adamw
fp32 muon
GF16 adamw
GF16 muon
GF32 adamw
GF32 muon
FP8-E4M3 adamw
FP8-E4M3 muon

(seed 1597, h1024, LR 0.002, 50K steps) Pre-registered: GF32 на 50K пробивает 2.55.

Wave D — depth ladder (3 runs)

seed hidden attn_layers LR steps
43 828 2 0.003 50 000
43 828 3 0.003 50 000
43 828 4 0.003 50 000

Pre-registered: monotone improvement layers 2→3, possible regression 3→4.

Wave E — champion attempt (1 run)

seed hidden attn LR steps corpus optimizer
43 1024 3 0.0025 100 000 fineweb adamw

Goal: BPB ≤ 1.85 (Gate-2 WIN). Pre-registered: fail-stop at step 50K if bpb > 2.10.

Acceptance gates

  • G1 (Wave A) champion-on-honest-corpus reproduced; LR-knee ≤ 0.003.
  • G2 (Wave B) ≥ 1 run with best_bpb < 2.10 (architectural floor pierced).
  • G3 (Wave C) precision-vs-optimizer Pareto frontier mapped, no precision regresses > +0.05 vs bf16.
  • G4 (Wave D) depth headroom characterised (monotone or knee).
  • G5 (Wave E) Gate-2 hit OR honest fail-stop with arch-update proposal.

Scheduling

Blockers (need merge before each wave)

R5 honesty rules (no exceptions)

  1. Никаких gate_status=new_champion без corpus=fineweb И final_step ≥ 27000 И image_sha != recovery-T-10H.
  2. Plateau объявляется только при ≥5 ticks AND step ≥ 50K (matches ARCHITECTURAL_FLOOR_BPB cull-safety).
  3. Pre-registration BPB-target ОБЯЗАТЕЛЕН перед запуском — иначе run пропускается.

Refs

phi^2 + phi^-2 = 3 · TRINITY · NEVER STOP

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions