Skip to content

Testing a Service

Tom Barber edited this page Nov 6, 2026 · 2 revisions

Testing a service

A process target is one command that starts, does its work and exits with a verdict. A server only starts and waits, so to test one you make the command a small driver. This is the short version of docs/guides/testing-a-service.md, which has the full Go source of a worked example.

The driver:

  1. starts your service as a child process;
  2. waits for it to listen on a loopback port;
  3. sends a fixed workload;
  4. checks the result, or relies on properties the service declares through an SDK (Guest-SDKs-by-Language);
  5. stops the service;
  6. exits with a verdict.

Everything runs inside one Hermit trial, so the service's threads and the driver's are scheduled together from the seed. Each trial is one start, one workload, one stop.

What was run. The example driver and service (Go) were built and run natively on a Mac, which proves the driver's logic. The Linux x86-64 builds were made. Nothing was run under Hermit, so how often and at which seeds the bug shows up under Hermit, and how Hermit handles loopback networking, were not measured.

Rules for the driver

  • Verdict. Exit non-zero, or print a line starting HAIRSPRING_VIOLATION:, when the service misbehaved (see Running-Your-Own-Software, "How a failure is detected"). Use a different exit code for "the driver could not do its job" (the port never opened), so a setup problem does not look like a bug in the service.
  • Fixed port and working directory. Put the port in the target's env and set cwd. If something else already holds a fixed port, the driver can talk to the wrong process: in the native run with another listener on the port, the service failed to start and the driver's connections were reset by the other listener.
  • Configuration comes from the target's env, not your shell. Hairspring starts the program from a minimal pinned environment plus what you put in env, with an empty standard input. This also works with --base-env=minimal in hermit_args.
  • Wait for readiness by connecting, for a bounded time. Do not sleep for a guess.
  • Stop the service on every exit path. An error path that calls os.Exit without stopping the child leaves the service running.
  • Keep output free of the clock, pids and ephemeral ports if you want hairspring doctor to say the target is deterministic.
  • Timeouts and process groups. Set trial_timeout_secs for the whole start, workload and stop, and raise wall_secs (the default of 60 seconds allows few Hermit trials). On a timeout or at the end of a trial Hairspring kills the program's whole process group; a child that calls setsid leaves the group and is not killed, so do not daemonise the service.
  • Parallelism. Leave parallelism at 1 while the port is fixed. Whether concurrent trials see each other's loopback ports under Hermit was not tested.

The target

[target.counter-service]
kind = "process"
command = ["/opt/counter-test/driver-linux", "/opt/counter-test/service-linux"]
cwd = "/opt/counter-test"
env = { SERVICE_PORT = "7391" }
trial_timeout_secs = 120

[target.counter-service.budget]
max_trials = 50
wall_secs = 600

hairspring targets lists it (run on the Mac). Natively, the driver against the buggy service printed HAIRSPRING_VIOLATION: lost updates: counter is 86, expected 200 and exited 1 (147 and 146 in two more runs); against the corrected service it printed ok: counter is 200 and exited 0 in each of three runs; with no SERVICE_PORT it exited 2. Cross-built for Linux with CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build, both programs are statically linked.

On a Linux machine with Hermit the next steps are the usual ones, not run here: hairspring doctor counter-service, hairspring test counter-service --seed 1, hairspring replay <bundle>.

Next: Build-a-Linux-Executable, Test-Logic-with-Wasm, Is-My-Machine-Suitable.

Clone this wiki locally