Repository navigation
Releases
v1.5.0
This release changes the example tooling and the tests. The installed aspire package has no code changes.
Added
- Verifier-plan smoke test S1 (
examples/verifier-distill/s1_nli_critic.py): an NLI critic run on the Intel NPU and iGPU,
with its pre-registration and receipts. It failed its pass line on both devices
(see the run note indocs/runs/receipts/2026-10-10-smoke/). - Verifier-plan smoke test S2 (
examples/verifier-distill/s2_grpo_smoke.py): a GRPO memory-and-speed smoke on one GPU,
with its pre-registration, an exclusive-GPU guard that works on Windows,
memory flags, per-phase allocator logging, a probe mode, a hash deny-list and CPU tests.- Prompt draws are recorded (
s2_draw.py,--draw), and--rollout-onlycounts correct items per prompt. - Logging: save-all output (written crash-safe), logging and post-pass modes,
--stop-after-new, and S2 sign diagnostics
with a pre-pass baseline and seed. - GRPO loss masks, truncation handling,
--stress-pool {selected,tune}with width telemetry, and the offrig quote rule
(quote_rule.py), including its edge cases. - A CT interview pipeline (
s2_interview.py,s2_interview_tag.py).
- Prompt draws are recorded (
examples/verifier-distill/s2_grpo_smoke.py: the S2 GRPO training run, launchable in chunks.--arm mech|critic,--train-seed,--trunc-window-updatesand--trunc-halt-ratename a training run.
All four are required, with no defaults, once any training flag is given.--stop-after-updates Nends a chunk cleanly. The train state (LoRA tensors, AdamW state, RNG states,
global step, CT guard, truncation window, and the length and sha256 of every archive) is saved every
--state-every-updatesupdates (default 10) and at the chunk's end; the last 2 saves are kept.
--resume-from <chunk dir>continues into a fresh--outfrom the latest save. After a kill it cuts a
copy of each archive back to the save point and records the cut.- A resume marks the chunk it consumed (
CONSUMED_BY.json). Resuming a consumed chunk, or any state that is
not the run's newest, is refused (RESUME_ALREADY_CONSUMED,RESUME_NOT_LATEST). The one escape,
--resume-override-reason, is recorded, and every later chunk of that run carriesforked_or_rewound: true
and a verdict line endingFORKED_OR_REWOUND. The claim (CONSUMED_BY.json) is created exclusively
just before the resuming chunk's first save: of two resumes racing for one chunk, the later halts with
S2-TRAIN-HALT-RESUME_RACE_LOSTand saves no state. A cut re-checks the saved prefix, hashes its copy, and leaves
CUT.jsonin the killed chunk naming the canonical copies. - Resume refuses any difference in the run header: settings, seeds, holdout and draw shas, aspire commit,
and the sha256 of the script,quote_rule.pyands2_draw.py, model revision, frozen stack. - Resume is state-exact on the CPU. On CUDA it restores all state, but the kernels are not
bitwise-deterministic; nothing switches deterministic mode on, and the header records
cuda_deterministic: falsewith the cuDNN and cuBLAS settings. - Training is seeded: the fresh LoRA's init, and each rollout by (seed, step, group). Save-all records the
seed that was applied, and an unseeded smoke run records none. - A rolling truncation halt beside the CT collapse guard, sharing its halt path. Every guard window (S1, S2,
U1, U2 and the truncation window) is appended tos2-hf-guard-history.jsonl.gz. - The critic arm: each all-wrong group gets one retry with a mechanical tool hint (offrig's quote rule and
lexical retrieval; NLI stays off because S1 failed). Retries are scored on gold and archived in
s2-hf-critic-retries.jsonl.gz. They never enter the loss, the guards or the training save-all, and
they draw from a forked RNG. The registered runs are mechanical only; this arm is a diagnostic. - Training verdicts:
S2-TRAIN-CHUNK-OKandS2-TRAIN-COMPLETE(exit 0),S2-TRAIN-HALT-<rule>(exit 2),
S2-TRAIN-FAIL(exit 1).
Changed
- Coverage now also counts
examples/verifier-distill, and CI's floor of 90% applies to it. - Public files no longer name personal desktop apps: the canned desktop is OS-only, and a test keeps the names out.
- The site's lockfile moves
undicito 8.12.3 andzodto 4.6.7 (4 Dependabot alerts).
Fixed
- Two S1 label counts in the docs were counted on a truncated key and are corrected.
aspire-si 1.4.0
Added
examples/sft-experiment/sft.py: training options for distillation ladders (#168):--chat-template-kwargs,--overlong dropwith--max-dropped-fraction,--gradient-checkpointing, and--ledger/--record-package.- Handbook: a page on the fine-tuning experiment and the distillation options.
Changed
- LICENSE and the forge integration's metadata name mcp-tool-shop as the copyright holder and author (#169).
- The release workflow pins
pypa/gh-action-pypi-publishto v1.14.3 (#166). - README translations refreshed (#171).
Fixed
- The 1.3.0 changelog entry listed options that weren't in 1.3.0; they're under 1.4.0.
Full notes: CHANGELOG.md
v1.3.0
Added
-
examples/sft-experiment/: the fine-tune-then-ASPIRE experiment (#11) and its
run report. It holds:- a dataset builder that keeps evaluation prompts out by word overlap and embeddings;
- a judge set of one-sentence planted errors;
- a cleaning step that records every drop;
- QLoRA fine-tuning;
- probes with schema 1.1 drift exports;
- the held-out and judge measures, with intervals that resample prompts;
- pod scripts that check the rented host's CUDA driver, GPU memory and download speed before
installing anything.
Result, over three training seeds (run 1 report):
fine-tuning the student before ASPIRE makes no reliable difference to how well the critic spots
planted errors, with either teacher. A first single-seed result said it hurt the composite
critic (0.65 against 0.87); two more seeds did not bear that out. The critic's accuracy varies
much more between training runs than between conditions (the composite control ranged from
0.43 to 0.87). The hidden-state trajectory replicates: the fine-tune moves the student about
30 times as far as ASPIRE does, always in the same direction, and ASPIRE's own drift opposes
that direction in every seed, with or without the fine-tune. -
Judges read order-averaged (report):
fresh_pairs.py
plants errors on new prompts,judge_kev.order_averagedaverages a judge's probabilities over
both answer orders, andjudge_logprob.pyreads the teacher's A/B log-probabilities. By single
choices every judge tried is position-biased and below 0.75. Order-averaged, on 149 fresh pairs,
Kev-4B reaches 0.973 (CI 0.946 to 0.993), Kev-9B 0.859 and the 32B teacher 0.886. Kev-4B,
read this way, is the reference judge for later comparisons. -
Step 2 (report): critics trained on 128 prompts instead of 32
score higher on planted errors (mean 0.745 against 0.635, every seed up by 0.10 to 0.13), but
differ between seeds exactly as much (a range of 0.181 at both). The fine-tune comparison waits
for a critic setup that varies less. -
Critic-init test (report): one draw of the critic's
initial weights, with nothing else changed, produced a critic that prefers the flawed answer
(0.315, interval entirely below 0.5). Under the committed rule both the critic's initial weights
and the run's seed carry the spread between runs; the ranges rest on three runs each and one
noise-floor pair. -
critic.init_seed(config, default unset): seeds the critic's initial weights apart from the
run's seed, leaving the run's own random stream (student adapter, data order, sampling) as if no
critic had been built. Unset, nothing changes. Used by the critic-init test
(plan). -
Critics with chosen attributes (report): the critic found by
chance that prefers flawed answers reads as an auditor on fresh pairs (0.678 [0.582, 0.763]).
Fostered Auditor and Advocate heads on three feature sources trained, but the shuffled-label
control failed by design (a learnable edit takes a random sign), so no role reading is made. The
descriptive numbers show heads that separate a pair (up to 0.97) but barely flag a lone flawed
answer (AUC about 0.55).
Changed
aspire.teachers.local: the scoring request and its parser are module functions
(evaluation_request,parse_evaluation,extract_score,extract_improved), so other code
scores exactly as the local teacher does. The request text and the parsed scores are unchanged.
Security
- Workflow jobs run with a read-only token by default.
ci.ymlhad jobs with nopermissionsblock, so they inherited the repository default token. A top-levelpermissions: contents: readnow applies to every job that does not set its own (the jobs that publish or write keep theirs). Closes theactions/missing-workflow-permissionscode-scanning alerts.
aspire-si 1.2.0
The first release tested end to end on real models. A 1.5B student was trained against 32B teachers on a rented GPU; the run report is in docs/runs/2026-10-06-pod-run.md.
Fixed
- Twelve bugs that the real runs exposed, each with a test:
- Checkpoints could not be loaded, and
aspire evaluateignored the trained LoRA. - The critic read the prompt without the response it was judging.
- The student's input was cut from the wrong end.
- Teacher JSON broke on LaTeX escapes, or was cut off mid-answer.
--configlost its settings.
- Checkpoints could not be loaded, and
- Security: checkpoints load with
weights_only=True, so a crafted checkpoint can no longer run code.
Added
aspire judgeandaspire.judgescore responses with a trained critic. No teacher is needed.- Local and composite teachers work from the config and the CLI. The Claude teacher defaults to
claude-sonnet-5-5, and the device defaults to cuda only when CUDA is available. - Geometry export schema 1.1:
step_axisandscalar_sourcetell ScalarScope how to read the steps. - The export is written after every epoch.
examples/local-runholds a run that needs no API key.examples/pod-runholds the configs, probes and drift scripts behind the ScalarScope fixtures.
Honest docs: the critic learns the teacher's score, but the student is not yet trained toward better answers. That work is tracked in #11. The README, the handbook and all seven translations say so.
Full details are in CHANGELOG.md.
aspire-si 1.1.0
What's Changed
- Training-dynamics export for ScalarScope; trainer tests run again by @mcp-tool-shop in #6
- Full treatment: 99% coverage held at 90%, structured CLI errors, ~40 bug fixes by @mcp-tool-shop in #8
- ci: bump the actions group across 1 directory with 5 updates by @dependabot[bot] in #9
- Release 1.1.0 prep: new logo, release.yml for PyPI Trusted Publishing by @mcp-tool-shop in #10
New Contributors
- @mcp-tool-shop made their first contribution in #6
- @dependabot[bot] made their first contribution in #9
Full Changelog: https://github.com/mcp-tool-shop-org/aspire-si/commits/v1.1.0