v0.4.0

• Add multi-process scenarios, whose roles, declared in [[package.metadata.base-benchmark.scenario.role]], are each one supervised process in its own process group for every sample: services, such as a server, and one driver, such as a load client. Exactly one role runs the subject, with the scenario's arguments and variants; the others run the same program with the same arguments for every subject. Each sample is one session: every executable is verified, services start in declared order and are each waited on until they print a line matching their ready pattern, whose {NAME} captures, such as the address a server bound, later roles receive as {ROLE.NAME}; then the driver runs, and the services are stopped in reverse order, by a declared signal with a grace period after which the group is killed, or by waiting for them to exit by themselves. Ready patterns match in time linear in the line's length, earlier captures taking the longest text that lets the rest of the line match. In role arguments, {{ and }} are literal braces, and any other brace, or a reference to a value no earlier service captures, is an error when the scenario is planned. Every role runs under the same launch controls as a single-process workload, one deadline covers the session, and a failure, the deadline, cancellation, or a panic kills every role's group and reaps every role, in reverse start order, before the host lock is released. A service that exits before it is ready or stopped, does not stop successfully, or a driver that fails or writes an invalid record stops the run with an error naming the role. The driver's record is the sample's.
• Add external programs, such as a load client installed with cargo install, which roles name with external = "NAME" and runs give by their path on the measuring host with --program NAME=PATH (Workspace::programs, Workspace::with_program). The plan records each program's resolved path and SHA-256 (Plan::programs, ExternalProgram), and the hash is verified before every sample, so a program changed during a campaign fails it. --host passes program paths to the host unchanged.
• Add --role-cpus ROLE=CPUS (RunRequest::role_cpus) to pin each role to its own CPUs. Recorded campaigns need a CPU set for every role, and --cpus applies only to single-process scenarios.
• Start every workload and role with the stop signals, SIGTERM, SIGINT, SIGHUP, SIGQUIT, SIGUSR1, and SIGUSR2, at their default dispositions and unblocked, even when base-benchmark started with some of them ignored, as & in a non-interactive shell leaves SIGINT and SIGQUIT and nohup leaves SIGHUP, or blocked, as a launcher's signal mask passes to what it starts. A child keeps the signals its parent ignores and the signal mask of the thread that started it, so a service stopped by one of them would outlast it and be killed after its grace period. Other signals' dispositions and the rest of the mask are left as inherited.
• Cancel campaigns and preparations on the first SIGINT, SIGTERM, or SIGHUP even when base-benchmark started with them blocked: the handler's thread unblocks them, so they reach it. They stayed pending, so neither Ctrl-C nor SIGTERM stopped base-benchmark.
• Kill and reap a single-process workload before the host lock is released when a panic, or a helper thread that fails to spawn, interrupts running it, as sessions already did. It was left running and unreaped after the lock was released.
• Add the driver's measured window. Under the runner, a multi-process scenario's driver marks when its counted operations start and stop with the lines base-benchmark window start and base-benchmark window stop on stdout, as whole, flushed lines before any output of its record; it is told to by BASE_BENCHMARK_CONTROL=stdio, which the plan records as the driver's PlannedRole::environment, and the runner acknowledges the start on its stdin once it has snapshotted every service, then closes its stdin. Window, available without the runner feature, sends them: Window::from_environment returns None outside the runner, Window::start_acknowledged waits for the acknowledgement, so that the snapshots precede the first counted operation, and Window::start does not. A driver that sends no markers, sends one twice, stops before it starts, sends an invalid marker, a marker after output of its record, or one without a newline at the end of its output fails its sample, as does one whose output a descendant holds open after it exits. Window implements Debug.
• Measure each service's CPU time over the driver's window, from its CPU-time clock, and the kernel's user and system split, from /proc/PID/stat, at the window's markers, as the cpu_time, user_cpu_time, and system_cpu_time totals of the new Phase::Window, journalled with each service's RoleEntry::window. Summaries report them as ROLE/METRIC totals, and each divided by the driver's operations as ROLE/METRIC_per_operation, CPU per request, with the new Statistic::ApproximatePerOperation, reported as ns/op, approximate: the window's edges, failed requests, and draining after the stop marker are not exactly the counted operations. No other service value is divided. The time counted is what the kernel accounts to the service's own threads: its child processes, such as the workers of a forking server, and softirq time spent on its behalf, such as network processing, are not included, so CPU per request understates a network server's cost, and summaries warn that window CPU time covers only each service's own process.
• Treat invalid responses as making comparisons invalid. A driver role names the total in its record that counts responses which failed its validation with invalid-responses, such as base-benchmark-load's responses_invalid (Role::invalid_responses, PlannedRole::invalid_responses). In screening, a sample with any keeps its numbers, and the summary reports every comparison involving its subject as invalid, without metrics (PairSummary::invalid), with a warning listed first; a recorded campaign fails on any. A driver record without the declared total fails its sample. Summaries of multi-process scenarios state that operations are the driver's successful operations, so every per-operation value, the driver's own included, and every request error count, is per successful operation.
• Add helper roles, which run a workspace binary, such as a load client or a wrapper of a separately published one, with binary = "NAME" and optionally package and features. Preparation builds each helper the prepared scenarios need once, from the baseline tree, unsalted, and records it as a fixed artifact in the manifest's helpers (HelperArtifact, ArtifactRole::Helper, PrepareOptions::helpers, Helper); every subject runs the same executable, staged under a name of the same length as the artifacts'. Workspace::smoke_executable skips scenarios whose roles run helpers, since none was built.
• Record each external program's version, the first line it prints for --version within 10 seconds, on stdout or else the start of stderr (ExternalProgram::version), or none when that fails; an unknown version is recorded, not refused. Campaigns ask once they hold the host lock, on the CPUs of the role that runs the program and under the campaign's cancellation, and record it in the plan in metadata.json; plans themselves leave it unknown.
• Breaking: artifact manifests are version 3, adding helpers; version 2 manifests, which have none, are still read and run. ArtifactManifest and PrepareOptions gain public fields, and ArtifactRole gains Helper, which breaks struct literals and exhaustive matches.
• Check whether roles share physical cores: two roles share one when their CPU sets include the same logical CPU or SMT siblings, as the measuring host reports them in /sys/devices/system/cpu/cpuN/topology/thread_siblings_list (Topology, PlanOptions::topology). Sharing that the scenario does not declare is refused by recorded campaigns and warned about by screening campaigns, when they start and in their summaries, and so are placed CPUs whose siblings the host does not report. A scenario that sets shared-placement = true, for example to model contention on a cloud host, may share cores in both modes, through reported siblings or not, and its summaries and reports label its placement as deliberately shared (Summary::placement). A virtual machine that hides SMT cannot be checked. Roles placed on CPUs that do not exist or are offline, according to /sys/devices/system/cpu/present and online, fail when the campaign is planned. Plans record the sharing they found (Plan::shared_cores, SharedCore, Plan::unknown_topology).
• Breaking: Phase gains Window, which records cannot contain, so records remain version 2; matches on Phase need the new arm. Summaries are version 3, since they can contain window metrics and the approximate statistic.
• Breaking: plans are version 3 and campaigns version 2. Plans of scenarios with roles add roles (PlannedRole) and programs, and their samples add roles (RoleEntry), journalling each role's command, CPUs, stack padding, executable hash, host, ready line and captured values, exit status, whether it was killed after its stop grace period, and the end of its output; plans and samples of single-process scenarios omit them and are otherwise unchanged. Scenario, PlanOptions, RunRequest, Workspace, and SampleEntry gain public fields, which breaks struct literals of them. Workspace::smoke needs the programs of every scenario's roles.