Skip to content

Concepts and Vocabulary

Tom Barber edited this page Nov 6, 2026 · 3 revisions

Concepts and vocabulary

Hairspring uses a small set of words consistently. This page defines each one and gives one concrete example. The examples use the built-in demo targets and the C language fixture from the repository, so you can try them yourself once you have followed Installation and Your-First-Run.

If you only want one-line definitions, see Glossary.

The picture in one diagram

flowchart LR
  T[Target<br/>your system + a property] --> R[Run]
  R --> T1[Trial, seed 1]
  R --> T2[Trial, seed 2]
  R --> T3[Trial, seed 3 ...]
  T2 -->|violation| B[Bundle<br/>seed + recorded inputs]
  B --> RP[Replay<br/>same seed, same schedule]
  B --> C[Confirmation<br/>recorded checks]
Loading

A target is tested by a run, which is made of many trials. Each trial uses a seed, which decides the schedule. A trial ends with a verdict. A failing trial gets a bundle, which you can replay or put through confirmation. A sweep is a run that keeps going after failures so it can count them.


Target

What it is. The thing you are testing, plus the rule it must obey, registered under a name. A target has a kind (sim, wasm or process, matching the three backends) and a budget that limits how much exploring a run may do.

Example. The built-in target pubsub-buggy is described as "At-least-once consumer; check, apply and record are separate steps (duplicate effect)". Its fixed twin, pubsub-atomic, does the three steps atomically and should pass. hairspring targets lists them.

A process target for your own program is a few lines in hairspring.toml, as in the quick start:

[target.at-c-buggy]
kind = "process"
command = ["/path/to/hairspring/examples/language-fixtures/c/build/gccO2/at-c", "buggy"]
trial_timeout_secs = 120

[target.at-c-buggy.budget]
max_trials = 20

Property

What it is. The rule that must hold. A trial that breaks it is a failure. Properties are how you tell Hairspring what "wrong" means.

Example. The C fixture at-c reports a property named c.counter_exact through the C SDK, which says the final counter equals 4,000. In buggy mode, a lost-update race between threads can make the counter come out lower, and the property is then false.

Properties come in kinds (from the guest protocol): always (must hold every time it is checked), sometimes (must be true at least once across the run), reachable (must be reached at least once), unreachable (must never be reached) and always_or_unreachable.

Trial

What it is. One attempt to make the target fail. On the process and wasm backends a trial is one seed: the program runs once from start to finish. On a sim target a trial is one exploration round.

Example. hairspring test at-c-buggy --seed 1 runs the C fixture under Hermit once per trial, each trial with a different scheduler seed, up to the max_trials = 20 budget.

Run

What it is. One invocation of testing a target: a job that runs many trials. A normal run (a search) stops at its first confirmed failure. A sweep runs its whole budget.

Example. hairspring test lock-lease --seed 7 starts one run of the lock-lease demo. hairspring results lists stored runs and hairspring show <run id> shows one.

Run inputs

What it is. For each run, Hairspring can keep a fingerprint (a SHA-256 hash) of each input the run executed with: the program under test, the target name and, where the backend reports them, its settings. Only fingerprints are kept, never the values, so environment values and paths are not stored. A fingerprint can tell you that an input is the same or different between two runs; it cannot tell you what it was, and Hairspring never shows one. It also records whether the worker that reported them was trusted.

What it is not. It is not a comparison. Nothing compares runs by these fingerprints yet.

Which runs have it. Runs stored before this was added have none, and Hairspring treats them as "not recorded" rather than guessing. Only built-in simulated targets record them, whether run by the server, hairspring test or a remote worker of this release. Hermit and WASM runs, and runs by workers from older releases, record none.

If a job is run again. When a job is given to another worker after the first one was lost, the new worker's trials replace the first worker's, and its fingerprints replace the first worker's too.

Seed

What it is. A number that drives every choice Hairspring makes during a trial. The same seed on the same setup makes the same choices. Different seeds make different choices, which is how Hairspring explores.

Example. On the process backend the seed is passed to Hermit as --sched-seed=<seed>, which decides which thread runs next and when it is interrupted. In the repository's determinism measurements, 20 Hermit seeds on a 4-thread racy C program gave 8 distinct interleavings, and each seed gave the same result in 5 of 5 runs.

Schedule

What it is. The order in which things actually happened in a trial: which thread ran when, which message was delivered first. On the process backend Hairspring compares schedules by a schedule fingerprint, which the showcase pages describe as a hash of five fields of Hermit's run summary.

Example. In the README's pubsub-buggy failure, the schedule is the list of steps: Deliver(1, 0), Tick, Tick, Deliver(0, 0), Step(1), Step(0), Step(1), Step(0). Change one step and you may get a different outcome.

Verdict

What it is. How a trial ended. In a sweep's stored rows the verdicts are:

Verdict Meaning
pass the trial ran and no property failed
violation a property failed, the program printed HAIRSPRING_VIOLATION:, Hermit reported a deadlock, or the program exited non-zero
inconclusive it timed out, or was cut short by the wall-clock budget; not a pass
unusable its reported events could not be used (lost, cut or contradictory); not a pass
infra the host or Hermit failed, not the program ("failed in the host")

Example. A trial of the C fixture that runs past trial_timeout_secs is inconclusive. The quick start is firm about this: such a trial is "never a pass". Only pass and violation count as rated trials when a rate is stated.

Bundle

What it is. A small text file that records what is needed to replay one failing trial: the seed, the build, the target, the backend, backend-specific parameters (param.* lines) and the path (the recorded steps or choices). Bundles from the process backend also record the execution environment (kernel, CPU, loader file hashes, Hermit executable hash and so on) so a replay can tell you if the machine has changed.

Example. The README's run wrote .hairspring/bundles/pubsub-buggy-d3858727.hairspring. The layout looks like this (values shortened for illustration):

hairspring-bundle v2
seed=5
build=...
target=pubsub-buggy
backend=sim
path=...,...,...

A process bundle carries param.sched_seed=... and, when it records the environment, starts with hairspring-bundle v3.

Replay

What it is. Running a bundle again to see whether the failure comes back. Replay refuses a bundle recorded against different code where it can tell.

Example. hairspring replay .hairspring/bundles/<bundle>.hairspring. For a process bundle the exit code says what happened:

Exit Meaning
0 not reproduced, on the environment it was recorded in
1 reproduced: the property is still violated
2 error, including a replay refused because the dynamic loader changed
3 not reproduced on a changed environment: inconclusive, not evidence the bug is fixed

So after you fix a bug, replaying its bundle and getting exit 0 on the same machine is good evidence the fix works for that schedule. Exit 3 tells you nothing either way.

Sweep

What it is. A run that does not stop at the first failure. It runs every trial in its budget, stores every trial, and reports how many were violations out of the rated trials, with a 95% interval.

Example. hairspring test checkout --sweep --trials 1000 --seed 1 runs trials at seeds 1 to 1000. The report gives a line such as sweep: <n> trials: <a> pass, <b> violation, <c> inconclusive, <d> unusable, <e> infra, names the lowest failing seeds, and states the rate. See Sweeps-and-Confirmation.

Confirmation

What it is. A set of explicit, recorded checks that a failure is real and repeatable, run by hairspring confirm <run-id | failure-id | bundle> on the process backend. The checks are: rerunning the same seed, replaying the bundle with the environment check, optionally running a fixed build at the same seed (a control), and optionally Hermit's strict determinism verify. The failure then gets a state such as candidate, rerun_repeated, confirmed or verified.

Example. hairspring confirm <failure-id> --control at-c-fixed would rerun the failing seed, replay the bundle, and run a second target that uses the fixed build at the same seed and flags (the target name here is your own; the --control flag is documented in the README). See Sweeps-and-Confirmation.

Guest SDK

What it is. A small library you add to a program run on the process backend, so the program can declare named properties and report them to Hairspring. "Guest" means the program under test; Hairspring is the host. Without an SDK, Hairspring still works: a non-zero exit is a failure. With one, you get named properties tracked across trials.

Example. With the C SDK:

#include "hairspring.h"

HS_ALWAYS(nbackfill <= mxframe, "wal.ckpt.nbackfill_le_mxframe", "nBackfill <= live mxFrame", nbackfill, mxframe);
HS_SOMETIMES(waited, "wal.writer.waited", "a writer waited for the lock");

SDKs exist for C (also used for C++), Go, Rust, Python, Java and .NET. See Guest-SDKs-by-Language, and Limits-and-Honesty for which have passed the acceptance test under Hermit.

Events

What it is. The messages a guest SDK writes so Hairspring can read the program's properties. Each one is a line on standard output or standard error that starts with HAIRSPRING_EVENT (one space) followed by a JSON object. Other lines are ordinary program output and are ignored. The format is the frozen guest event protocol v1.

Example. A process says hello first, sends a catalog line for each property id at startup, an assert line with its counts at exit, and an end line last. A line such as

HAIRSPRING_EVENT {"v":1,"kind":"work","proc":"main","ops":4000}

reports work done (the field names are from the protocol's message table; the values here are illustrative). You do not write these by hand: the SDK does. If the events are lost or contradictory, the trial is marked degraded and is not counted as a pass.

The server also streams live events about runs (progress, finished) to the web UI and API clients; that is a separate thing from guest events.

Backend

What it is. The part of Hairspring that actually runs a trial. There are three: sim (a Rust state machine), wasm (a WebAssembly module) and process (an unmodified Linux program under Hermit). See What-is-Hairspring.

Example. The C fixture target above has kind = "process", so it uses the process backend and needs Linux on x86-64.

Finding

What it is. A failure Hairspring recorded, together with the evidence about it. Its state is derived from the stored confirmation checks, never stored by hand, so it cannot drift from them.

Example. A failure whose confirmation reruns all repeated it, but whose bundle replay has not (yet) repeated it, is rerun_repeated. Hairspring's own wording is careful: that state "is not 'reproduced'", and only confirmed and verified reach the generated claim text.

In the web UI. A failure's page shows its state as a ladder: Candidate (no check has repeated the failure), Rerun repeated, Confirmed and Verified, with two ends that are not rungs, Not reproducible and Control failed. A failure found on a simulation target has no checks, so it stays a candidate.

The Confirmation section of a failure page: Candidate is the current step; Rerun repeated, Confirmed and Verified are marked Not reached; the page says no checks are recorded.

The ladder for a simulation failure. None of the states says the failure is fixed.

Worker

What it is. A process that takes queued jobs and runs them. hairspring serve runs one inside the server unless you give it --workers 0, and hairspring worker registers a remote one. The Workers page lists each worker's state (Busy, Idle or Not seen recently), whether it is Trusted or Untrusted (an administrator sets this), and what it said about itself when it registered.

Example. The Workers page of a fresh hairspring serve lists one worker, embedded-0, Idle and Trusted. Data from an untrusted worker is shown, labelled as unverified, and claims need trusted workers.

Clone this wiki locally