Skip to content

Reward function experiments: audited three-seed mechanism study

Latest

Choose a tag to compare

@arpjw arpjw released this 12 Oct 05:05

Completed nine-worker mechanism study

This release adds the frozen, independently audited A/B/C study across paired seeds 103, 211 and 307. Each fresh Qwen3-4B policy completed 64 groups of four trajectories and evaluated 96 previously untouched final tasks twice. The 34-page paper retains its derivations, earlier development study and all eight figures, and adds the executed training recipe, qualification, recovery and limitations.

Primary full-penalty B minus homogeneous-success-mask C: -29.564236 tokens (-1.804218% relative to C), descriptive 95% crossed seed/task interval [-76.662413, +15.480469]. Accuracy: +3.993056 percentage points, interval [-0.520833, +8.506944]. The registered promising token-cost response criterion was not met. Only three seeds and restricted synthetic tasks limit inference; a smaller effect is not ruled out. No adaptive-controller advantage is established.

Public scientific reconstruction

Includes all 2,304 training trajectories, 1,728 trained-policy final records, the one 192-record initial reference, frozen scientific source/configuration/tasks, raw numerical gates and independent audit receipts. CPU reconstruction checks complete original member hashes and reproduces recorded scoring, rewards/advantages/masks, update support, pairing and the registered bootstrap. The downloadable archive is the complete reviewed public package; verify its file manifest before analysis.

Actual model/optimizer/RNG binaries, encrypted operational archives, credentials, provider account/support records and the original private Git history are excluded. Recorded tensor hashes are audit receipts, not independently rehashed absent tensor bytes. No model or cloud connection is required for public reconstruction. Training-source availability does not authorize another paid run.

The paper, website visualizer and Work entry reference this same immutable version. Earlier releases and historical evidence remain preserved. Original code: Apache 2.0; original text/tasks/measurements: CC BY 4.0, subject to third-party notices.