Skip to content

Ask the evaluator during an epoch, not only at its boundary - #98

Merged
rasros merged 1 commit into
mainfrom
occasional-panel
Aug 18, 2026
Merged

Ask the evaluator during an epoch, not only at its boundary#98
rasros merged 1 commit into
mainfrom
occasional-panel

Conversation

@rasros

@rasros rasros commented Aug 17, 2026

Copy link
Copy Markdown
Owner

The cheap measures can be driven a long way without the drawing getting better — on one run they improved 64% while the evaluator scored the result within 0.000004 of a run it should have beaten. Asking the evaluator only at the epoch boundary means noticing that after the fact.

What runs

--epoch-eval-interval (default 2000 tasks) puts the front to the evaluator mid-epoch. The field is epoch_parents(pool, FRONT_EVAL_CAP) — the best-ranked distinct candidates, capped at 24, since the top tier can be most of the pool and evaluating near-clones spends the expensive part of the run learning nothing.

--epoch-eval-patience (unset) ends the epoch once the evaluator has gone that many rounds without seeing anything better. Rounds rather than checks, so the number means the same thing whatever cadence checks run at.

Both are affordable only because of #97: the evaluator's score is absolute and cached per node, so a check re-prices what is new and recalls the rest. A fully cached check does not load a model.

Why patience is unset

Too low and epochs end on the evaluator's noise; too high and it does not stop the unsupervised drift it exists for. Nothing measured so far pins it, and defaulting it on would be guessing — the mistake that put collapse floors in and ended every epoch of a run on arrival.

best_node now updates at every check rather than only at boundaries, so the run's best is verified far more often, and the final fallback is a candidate the evaluator has actually seen.

Also here

epoch_parents still sorted its Pareto front by n.score, which #96 made a constant — a dead sort. Removed: non_dominated_sort already yields the best tier first, so the head is the best-ranked distinct candidates and there is nothing left to sort by.

Not here

The GPU variant. The plan was back-to-back checks on GPU and interval-based on CPU; this is interval-based on both. Running the evaluator on a background thread alongside search needs its own change for shutdown and for the races around a pool that mutates while it is being read — it should not ride along with the criterion itself.

@rasros
rasros merged commit ef483b0 into main Aug 18, 2026
1 check passed
@rasros
rasros deleted the occasional-panel branch August 18, 2026 05:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant