feat(pretrain): task #125 — --mode flag + HP defaults per regime#938
Merged
feat(pretrain): task #125 — --mode flag + HP defaults per regime#938
Conversation
Contract training-loop-pretrain-v1 v1.2.0 → v1.3.0:
- New hyperparameter_defaults table:
* finetune: (Finetune, lr=5e-5, warmup=100, target=2.2)
* from_scratch: (FromScratch, lr=3e-4, warmup=1000, target=3.0)
- New INV-TRAIN-009: `apr pretrain --mode={finetune|from-scratch}`
atomically flips the 4-tuple; explicit --lr / --warmup-steps /
--target-val-loss overrides still win.
- New GATE-TRAIN-009 with evidence_discharged_by listing 3 Rust tests.
- `pv validate` green (0 errors).
CLI (apr-cli):
- New `PretrainMode` ValueEnum (finetune | from-scratch), default
finetune. Re-exported from crate root.
- `--lr`, `--warmup-steps`, `--target-val-loss` now `Option`; omit
to inherit the mode default.
- New `--vocab-size` flag (default 50257) plumbs into
TrainingRegime::FromScratch so INV-TRAIN-005 epoch-zero cap =
2·ln(vocab_size) lands correctly.
- `mode_defaults()` resolver is the single source of truth binding
the YAML table row to the PretrainConfig fields — no way to
construct a config where regime says FromScratch but lr/warmup/
target came from the finetune row.
Falsifier tests (GATE-TRAIN-009):
- `mode_finetune_is_default_and_matches_contract` — defaults.
- `mode_from_scratch_applies_all_four_defaults` — cold-start 4-tuple.
- `mode_from_scratch_honors_explicit_lr_override` — override wins,
regime still flips.
Motivation: task #119 smoke run used MODEL-1 finetune HPs (lr=5e-5,
target=2.2) on a from-scratch regime — wrong LR band AND
unreachable target. This closes that drift at the CLI boundary.
Gates: 6/6 apr-cli pretrain tests, 1371 contract tests, clippy+fmt
clean on touched files.
Closes: #125
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
training-loop-pretrain-v1v1.2.0 → v1.3.0: newhyperparameter_defaultstable (finetune vs from_scratch rows) + INV-TRAIN-009 / GATE-TRAIN-009 binding the CLI flag to the table.pv validategreen.apr pretrain --mode {finetune|from-scratch}flag atomically flips(regime, lr_max, warmup_steps, target_val_loss). Explicit--lr/--warmup-steps/--target-val-lossstill win.--vocab-size(default 50257) so INV-TRAIN-005 epoch-zero cap forfrom-scratchlands correctly.Why
Task #119's real-compute smoke run used MODEL-1 finetune HPs (lr=5e-5, target=2.2) on a from-scratch regime — wrong LR band AND unreachable target. This PR closes that drift at the CLI boundary so operators can't silently ship a cold-start run with finetune hyperparameters.
Also unblocks subsequent MODEL-2 real-compute lanes: one flag flip applies the full 4-tuple together.
Test plan
pv validate contracts/training-loop-pretrain-v1.yaml→ 0 errorscargo test -p apr-cli --features training --lib pretrain→ 6/6cargo test -p aprender-contracts --lib→ 1371 passcargo clippy -p apr-cli --features training --lib -- -D warnings→ cleancargo fmtclean on touched filesapr pretrain --helpsurfaces--mode,--vocab-sizeflagsci / gate+workspace-testgreenCloses #125
🤖 Generated with Claude Code