Make the panel score absolute so two evaluations can be compared

[rasros]

Groundwork for evaluating occasionally during an epoch rather than only at its boundary. That is impossible while the panel's scores only mean something inside one call, which is what they did.

The problem

rank() put every pair to the panel and scored each candidate by how many rivals it beat. That ranks a field correctly and says nothing outside it: the same drawing scores differently depending on who it was ranked against. Its own docstring said "nothing compares them against an absolute threshold."

Two consequences. Scores could not be cached — a value from an earlier call was meaningless later. And cross-run comparisons made with these numbers were invalid, including one I reported today: v8 at 0.004392 against v7 at 0.019767 is not a 4.5x improvement, it is two candidates each near the top of a different field.

Calibrated median

Each member's raw cosine distance is divided by that member's distance from the target to a blank canvas, measured once when the reference is prepared. 0 is the target, about 1 is as wrong as an empty drawing — on every member and every target.

Calibration is what makes combining them honest. The members are cosine distances from three different embedding spaces spanning different widths, so an uncalibrated average is decided by whichever member spreads widest: one model steering the run, the thing a panel exists to prevent.

Then the median, which is the panel argument in absolute form. With three members the median is the majority position: for any standard you might hold a candidate to, "the panel says it meets this" is true exactly when the median says so, and a member that is idiosyncratic about this particular drawing cannot move it. The pairwise vote said that about pairs; this says it about candidates, which is what lets two scores be compared at all.

The cache this unlocks

rank_front now skips any node that already carries FRONT_SCORE and recalls the stored value. A run asks about the same pool members repeatedly, and re-rasterising and re-embedding one the evaluator has already seen bought an identical number at full price. A call the cache answers in full no longer loads a model at all.

Not here yet

The periodic evaluation itself. rank_front is a closure inside run_vector_search, so the caching path above has no unit test — extracting it to a module-level function is the next step and makes both testable.