Skip to content

Add offline submission bundle validator for miners#4

Merged
tmimmanuel merged 4 commits into
mini-router:mainfrom
claytonlin1110:sn74-claytonlin1110
Jul 8, 2026
Merged

Add offline submission bundle validator for miners#4
tmimmanuel merged 4 commits into
mini-router:mainfrom
claytonlin1110:sn74-claytonlin1110

Conversation

@claytonlin1110

Copy link
Copy Markdown
Contributor

Summary

  • Closes Add local submission bundle validator for miners #3: adds an offline CLI (python scripts/validate_submission.py / python -m trinity.validate_submission) that checks submissions/final_model/ before miners open a PR.
  • Validates required files, best_theta.npy shape against ParamSpec.n_total (13312), finite floats, and coherent summary.json; reports optional history.json / eval.json.
  • Documents the checker in CONTRIBUTOIN.md, README submit workflow, and submissions/final_model/README.md; unit tests cover pass/fail cases.

Test plan

  • python -m pytest tests/test_validate_submission.py -q (8 passed)
  • Run python scripts/validate_submission.py against a real experiment best_theta.npy + summary.json
  • Confirm a deliberately broken bundle exits non-zero with clear errors

@github-actions github-actions Bot added docs Documentation changes eval Evaluation changes miner Miner contribution PR submission Final submission bundle train Training or routing changes labels Jul 8, 2026
@tmimmanuel tmimmanuel added the feature New feature label Jul 8, 2026

@tmimmanuel tmimmanuel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Add the offline validator step to PR automation before submission upload. Right now the workflow still only checks for best_theta.npy, so invalid bundles can still get queued and fail later. Run python scripts/validate_submission.py --dir pr/submissions/final_model in the workflow and fail early if it exits non-zero.

  2. Move validate_submission.py out of scripts/ into a utility path, like utility/validate_submission.py. The current scripts/ classifier will keep tagging that file as train/eval, which creates label noise for a tooling-only change.

@claytonlin1110
claytonlin1110 requested a review from tmimmanuel July 8, 2026 18:52
glorydavid03023 added a commit to glorydavid03023/minirouter that referenced this pull request Jul 8, 2026
Per maintainer review guidance on PR mini-router#4: the pr-automation classifier tags any
scripts/ file as train+eval, which is label noise for a tooling-only file. The
LRA ablation is an offline validation harness, not a train/eval entry point, so
it belongs under utility/ (which the classifier does not label). This drops the
spurious `eval` label from this PR — the optimizer change earns `train` from
src/ on its own. Updated all path references (docstrings, docs, config, tests).

@tmimmanuel tmimmanuel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • CONTRIBUTOIN.md and src/trinity/validate_submission.py still present python -m trinity.validate_submission as a supported command. On a fresh clone, that fails unless PYTHONPATH=src is set or the package is installed. The zero-config entrypoint is the new wrapper in utility/validate_submission.py, so the docs should either drop the module form or make the PYTHONPATH requirement explicit.

claytonlin1110 and others added 4 commits July 8, 2026 15:14
Catch missing or malformed final_model artifacts before PR automation uploads them (closes mini-router#3).

Co-authored-by: Cursor <cursoragent@cursor.com>
Do not treat README under submissions/final_model as a miner submission; only queue when best_theta.npy changes.

Co-authored-by: Cursor <cursoragent@cursor.com>
Move the CLI wrapper to utility/ so scripts/ path labels stay clean; fail early on invalid bundles.

Co-authored-by: Cursor <cursoragent@cursor.com>
…oint.

Drop python -m trinity.validate_submission from miner docs; it requires install or PYTHONPATH.

Co-authored-by: Cursor <cursoragent@cursor.com>
@tmimmanuel
tmimmanuel merged commit 6fef335 into mini-router:main Jul 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation changes eval Evaluation changes feature New feature miner Miner contribution PR submission Final submission bundle train Training or routing changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add local submission bundle validator for miners

2 participants