Project Settings

The project folder

A project folder (parityrun/ after init) holds:

parityrun_project.py    PROJECT = parityrun.Project(...)
impls.json              the implementations: how to start each one
scenarios/<name>.yaml   what to do
fixtures/               uploads, long LLM replies, files copied into the data folder
golden/<name>/          the recordings (written by `record`, committed), with coverage.json when measured
golden/endpoints.json   every route, for `coverage` and `capture`

Commit it with the code; out/ (what check and port leave) is git-ignored.

impls.json

Each implementation has a working directory (cwd, relative to repo) and a command with {port}, {repo}, {python} and {shims} (parityrun's shims, see Libraries) filled in. A program named without a path is looked up on PATH. Optionally:

Key	Meaning
cli	a command for command-line steps
env	environment values for this implementation only ({port}, {repo}, {python}, {shims} filled in), on top of the project's env()
build	a command run in its folder before record or check; a failed build is 0 % parity, with the compiler's output
skip	names not copied into an isolated run's copy (target for Rust)

{"php":  {"cmd": ["php", "-S", "127.0.0.1:{port}", "-t", "public"]},
 "node": {"cwd": "../port-js", "cmd": ["node", "server.js"], "env": {"PORT": "{port}"}},
 "rust": {"cwd": "rust", "cmd": ["{repo}/rust/target/release/notes", "--port", "{port}"],
          "build": ["cargo", "build", "--release"], "skip": ["target"], "cli": ["{repo}/rust/target/release/notes"]}}

A new version gets its port in the PORT environment variable (init gives every implementation one) and is checked with parityrun check --impl NAME.

parityrun_project.py

import pathlib

from parityrun import Project, RunInfo

def env(run: RunInfo) -> dict:
    """What every implementation is started with: the contract."""
    return {"PORT": str(run.port), "DATA_DIR": str(run.data_dir), "LLM_BASE_URL": f"{run.stub_url}/v1",
            "PAYMENTS_API": run.stub_url}

def outside(stub, method, host, rest, query, body, headers):
    """Answers for {stub}/<host>/<path>; None for anything else (a 404)."""
    if host == "api.payments.example" and rest == "/charges":
        return 200, {"id": f"ch_{stub.next_id()}", "status": "paid"}, {}
    return None

PROJECT = Project(
    root=pathlib.Path(__file__).parent,
    reference="php",
    quick=["smoke", "billing"],
    env=env,
    ready="/health",
    html=True,
    redirects=True,
    isolate=True,
    databases=["database/app.sqlite"],
    stub_handler=outside,
)

Settings

Setting	Meaning
root, repo	the project folder; what {repo} means (default: the folder above root)
reference	the implementation record runs
quick	the scenarios check --quick runs; --slow runs the rest
env(run)	the environment on top of the caller's: run.data_dir, run.work, run.port, run.stub_url, run.repo
ready, ready_timeout	GET this path until it answers 200 (default /, 90 s)
shutdown	e.g. "POST /shutdown"; without it, or if it doesn't stop in 20 s, the process is killed
headers	sent with every request (e.g. a CSRF header)
events	a server-sent events path a step's sse: n listens to
html	record HTML answers as normalised lines: attribute order, entity spelling, indentation and comments don't count
redirects	record a 3xx answer's Location
isolate	run each implementation in a fresh copy of its folder; what it creates, changes or deletes there is recorded as app/<path> (isolate_skip: names not copied, default .git)
ignore_files	path patterns never recorded, e.g. ["storage/logs/*", "*.log"]
coverage	measure which lines of a PHP implementation the scenarios run (PCOV or Xdebug)
library	the reference is a library its scenarios call through a shim (see Libraries)
databases	SQLite files in the data folder (or, isolated, in the app's folder); the first is recorded as db
snapshot(run), setup(run)	any other database (see Databases); setup runs before each start
random_files	file names random by design: only their size is compared
interleaved_files	JSON Lines files written by parallel work: line order is free
measured_keys, measured_companions	wall-clock numbers (e.g. seconds), and numbers computed from them beside them (e.g. cost_usd): <measured>
extra_roots	{"site": "site"}: files under <work>/site are recorded as site/...
stub_handler, stub_queues	answers for outside services; step keys whose lists feed stub.queues[key] (e.g. chat updates)
post_process(recording)	a last change to the normalised recording (e.g. ids of rows parallel work inserts in any order)
labels	the placeholders for the run's data, work, stub and service addresses
image_ssim	how alike pictures must be (default 0.98)
work_prefix, prog	the temporary folders' name start; the command name in help text

A project with its own entry point can skip --project: parityrun.cli.main(project=PROJECT).