Add offline submission bundle validator for miners#4
Merged
Conversation
tmimmanuel
requested changes
Jul 8, 2026
tmimmanuel
left a comment
Collaborator
There was a problem hiding this comment.
-
Add the offline validator step to PR automation before submission upload. Right now the workflow still only checks for
best_theta.npy, so invalid bundles can still get queued and fail later. Runpython scripts/validate_submission.py --dir pr/submissions/final_modelin the workflow and fail early if it exits non-zero. -
Move
validate_submission.pyout ofscripts/into a utility path, likeutility/validate_submission.py. The currentscripts/classifier will keep tagging that file astrain/eval, which creates label noise for a tooling-only change.
glorydavid03023
added a commit
to glorydavid03023/minirouter
that referenced
this pull request
Jul 8, 2026
Per maintainer review guidance on PR mini-router#4: the pr-automation classifier tags any scripts/ file as train+eval, which is label noise for a tooling-only file. The LRA ablation is an offline validation harness, not a train/eval entry point, so it belongs under utility/ (which the classifier does not label). This drops the spurious `eval` label from this PR — the optimizer change earns `train` from src/ on its own. Updated all path references (docstrings, docs, config, tests).
tmimmanuel
requested changes
Jul 8, 2026
tmimmanuel
left a comment
Collaborator
There was a problem hiding this comment.
CONTRIBUTOIN.mdandsrc/trinity/validate_submission.pystill presentpython -m trinity.validate_submissionas a supported command. On a fresh clone, that fails unlessPYTHONPATH=srcis set or the package is installed. The zero-config entrypoint is the new wrapper inutility/validate_submission.py, so the docs should either drop the module form or make thePYTHONPATHrequirement explicit.
Catch missing or malformed final_model artifacts before PR automation uploads them (closes mini-router#3). Co-authored-by: Cursor <cursoragent@cursor.com>
Do not treat README under submissions/final_model as a miner submission; only queue when best_theta.npy changes. Co-authored-by: Cursor <cursoragent@cursor.com>
Move the CLI wrapper to utility/ so scripts/ path labels stay clean; fail early on invalid bundles. Co-authored-by: Cursor <cursoragent@cursor.com>
…oint. Drop python -m trinity.validate_submission from miner docs; it requires install or PYTHONPATH. Co-authored-by: Cursor <cursoragent@cursor.com>
claytonlin1110
force-pushed
the
sn74-claytonlin1110
branch
from
July 8, 2026 20:17
3e6f93e to
54d52ff
Compare
tmimmanuel
approved these changes
Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
python scripts/validate_submission.py/python -m trinity.validate_submission) that checkssubmissions/final_model/before miners open a PR.best_theta.npyshape againstParamSpec.n_total(13312), finite floats, and coherentsummary.json; reports optionalhistory.json/eval.json.CONTRIBUTOIN.md, README submit workflow, andsubmissions/final_model/README.md; unit tests cover pass/fail cases.Test plan
python -m pytest tests/test_validate_submission.py -q(8 passed)python scripts/validate_submission.pyagainst a real experimentbest_theta.npy+summary.json