Valence

Toward a cleaner, more rigorous standard for distributed chess engine testing.
A coordinator hands out games, workers play them and sign the results, and nothing counts until the signature checks out.

It's built against OpenBench, which have been run for years by many. It works, and that's the point: the goal is to beat it, not to differ from it. Same job, then, minus the parts I got tired of living with. Results trusted because a worker said so. A tuner that steps on whatever games trickle in. A worker you stand up by dropping Python and a repo on the box. SQLite, the day fifty machines report at once.

So Valence signs every result with Ed25519 and re-tallies from the games rather than the reported count. The tuner moves once a generation, after every vector has filled its budget, so a slow box can't bend the run. The worker is a single static binary that mints its own key and prints it; you paste that to authorize it, and nothing secret ever leaves the machine. Postgres from the start.

Where it is

Early, and alive end to end. A worker provisions its own pinned fastchess, fetches engines by commit from any git host, proves every build against the bench its commit declares (or the override an author types when a commit declares none), plays the games, and signs what it saw. The server checks the signature, re-tallies from the PGNs rather than the claimed counts, reclaims what a vanished worker left behind, and moves the SPRT live on the dashboard.

Signups land pending, and a run starts only once its project's owner or a team member approves it, so a shared fleet burns compute on runs somebody vouched for. Tests are created from per-engine profiles that prefill the whole form. Datagen is a first-class run kind: the worker mints openings through the engine's own genfens, every game is replayed before it counts, and the stored games export for trainers as Texel triples or viriformat. Networks upload once and pin by sha; books pin the same way. The artifact store keeps every report's games under zstd, retention-swept with datagen exempt, and the outlier pass flags the worker whose perfectly signed games came from somewhere else.

Still to come: the tuners, generational from the first one.

Running it

Postgres lives in a container and the server points at it by default, so there's nothing to install and nothing to configure:

make db     # Postgres, up and healthy
make build  # the dashboard, then both binaries, landed at the repo root
./valence   # the coordinator on :7878, dashboard included

The same binary runs the operator jobs. Point the importer at an OpenBench database for something to look at:

./valence import-openbench path/to/db.sqlite3

To let a worker in, run it first: it mints an identity, prints the public key, and waits. Authorize that key and the worker starts pulling games on its own, no restart:

./valence-worker --server http://localhost:7878   # prints the key to authorize
./valence user add testarossa --admin
./valence worker authorize <key> --owner testarossa --name laptop

A test you create starts at once when you may approve it, and lands pending otherwise: the project's owner, its team, or an admin clears it from the test page, and the workers start pulling.

Hacking on the dashboard itself is the one time you want Vite between you and the server: make web runs it with hot reload, proxying /api through.

Deploying it

The server is an OCI image with a two-line contract: DATABASE_URL in, plain HTTP on 7878 out. An operator with their own Postgres and reverse proxy runs exactly one container, and deploy/valence.container is a ready Podman Quadlet for that shape. On a bare host, one line brings up the database, the coordinator, and auto-provisioned HTTPS:

DOMAIN=tests.example.org docker compose --profile server --profile caddy up -d

valence dump writes the whole database to a pg_dump archive, and valence restore loads one into an empty database, refusing a non-empty one. The archive holds schema and data both, so nothing under Postgres's data directory needs to travel, and a newer binary migrates a restored instance forward on boot. From a container, stream it out:

docker compose exec -T server valence dump - > valence.dump

# docker compose exec -T server valence restore - < valence.dump

The same two commands move a whole instance into a fresh local install: restore must be the first thing that touches the new database, since it refuses one that already has tables and the server's boot-time migration would create them.

The worker never deploys. It's a binary you run wherever the cores are, pointed at the server's URL.

The pieces

valence is the coordinator: the API, the dashboard, the database, and the operator commands (user, worker, test, network, book, pgn, dump, restore, import-openbench). No subcommand means serve.

valence-worker is the binary a contributor downloads and runs. One file, no interpreter, no database. It provisions a pinned fastchess, builds the engines a parcel names, refuses any build whose bench disagrees with its test's declared number, and signs every result before it leaves the machine.

The rest: crates/core holds the shared protocol, the signed exchanges and the keys both sides speak, and web/ is the Svelte dashboard, compiled and embedded into the coordinator binary.

Working on it

make all is the gate before a commit: format, clippy, tests, and the dashboard typecheck.
make reset wipes and re-migrates the dev database when you want a clean one.
Migrations only append from here: every dump records the migrations that built it, and editing a shipped migration strands the dumps written before the edit.
Rust is stable, edition 2024, pinned, with the floor at 1.97.0.

Licensed AGPL-3.0-or-later.