Skip to content

QUALMZ v0.1.0

Latest

Choose a tag to compare

@PNX89 PNX89 released this 29 Aug 02:29
· 5 commits to main since this release

A look budget enforced by a constraint, and a promotion that stops for a named human.

The budget. A researcher gets a fixed number of looks at a held out window. The count is a row with a unique constraint on the strategy, the configuration hash and the window, so re-running the same configuration does not buy a fresh look and a crash does not cost one. The rule lives in the schema rather than in application code: an insert that goes around the API entirely still fails, which is the case every honour system loses.

Four gates before a promotion. Data validation before the model sees the frame, a pinned golden set that fails on any change including an improvement, a wall-clock budget taken as the best of several runs, and a data-provenance check. The pipeline runs twice and the second run differs in exactly one value: the data version the challenger was scored on. It is refused, and every number in it is correct.

This repository consumes a provenance hash and cannot mint one. A test parses the source and fails if anything computes a SHA-256, naming the two local hashes as allowed, with a second test asserting neither ever meets a provenance value.

The halt is watched. Skipping downstream tasks, tasks_skipped=1, and the run finishes successful rather than failed. A promotion waiting on a named human is not an incident.

Bad luck or a bug, with a number in it. Two paired series of 500 days. On the second, with no drift between them at all, the naive standard error reports t = -3.26 and calls it a bug; across 25 non-overlapping blocks it is -0.95. Both series are generated and no capital was ever allocated.

Adopted rather than rebuilt. The holdout schema and role-scoped grant come from QUIZZ, which owns them, and the first screenful says so.

Verification. 8 required status checks, four of which re-run a measurement and fail if a byte changed. The grant is watched refusing against a real PostgreSQL.