Skip to content

v0.2.0

Choose a tag to compare

@github-actions github-actions released this 08 Nov 01:27
· 16 commits to main since this release
993d7ec
  • Add --host HOST to prepare, run, and smoke (remote::Remote) to run them on a measuring host over SSH. Local paths, made canonical, are mirrored on the host with rsync, including the repositories of linked worktrees, the release of this version is installed there when it is missing, and outputs are copied back, also after a failure. Interrupting the local command stops the host's. --remote-directory and --remote-program choose where and what runs. Add version, which prints the command's version.
  • run makes its output directory absolute, so a relative --output works when the command runs outside the workspace root.
  • Build layout replicas with a distinct -C metadata salt for every crate, recorded as BuildIdentity::layout_salt. Rebuilding at another path alone gave an executable byte-identical to the baseline, since Cargo identifies workspace members by relative path, so 0.1's replicas measured run-to-run variation of one executable rather than build layout. The salt is added by a RUSTC_WRAPPER script in the replica's target directory, which chains RUSTC_WRAPPER or CARGO_BUILD_RUSTC_WRAPPER from the environment, such as sccache. A wrapper set in Cargo configuration cannot be chained, so preparing replicas then fails. RUSTFLAGS and Cargo configuration are untouched. Summaries warn about replicas identical to their baseline. Salting needs Unix.
  • Add --overlay true (PrepareRequest::overlay, Overlay) to build every tree with the current workspace's workload package, so that revisions from before the workload existed, or with an older version of it, run the same harness. Each tree is copied to a sibling directory, its workload package replaced, its lockfile seeded with the current workspace's entries for packages it does not lock, and the copy built without --locked; trees are never modified. Artifacts record the overlaid package in SourceIdentity::overlay, and in SourceIdentity::lockfile_changes every locked version the build changed and every added package that resolved differently from the current workspace. Recorded campaigns require the overlaid package to be committed and refuse lockfile changes. PrepareOptions and SourceIdentity gain these public fields.
  • Let scenarios set their own Cargo features, so that a diagnostic or allocation-counting build can be a declared scenario without reaching the timed builds. Features are canonical: entries are split on commas and whitespace, sorted, and deduplicated (canonical_features). Preparation builds each tree once for every distinct feature set of the scenarios it prepares, or of those named with --scenario, and a run uses only the builds with exactly its scenario's features; Workspace::smoke builds every feature set. Executables are frozen under artifacts/LABEL/N/, numbered by feature set.
  • Breaking: artifact manifests are version 2, with features on each Artifact instead of the manifest, and plans version 2. A manifest's artifacts all have known features or all unknown ones, and each feature set builds the same trees with the same roles. Campaigns from 0.1 still parse and summarize; reading a version 1 artifacts.json asks to prepare again. Scenario and Artifact gain a public features field. Workspace::prepare takes a PrepareRequest, PrepareOptions has feature_sets, and Plan::new narrows the plan's artifacts to its scenario's builds (ArtifactManifest::for_features). Workspace::smoke_executable takes the features the executable was built with, skips scenarios that need others, and fails if none can run.
  • Add Metric::quantile and Measurement::Quantile for order statistics such as a 99th percentile latency, summarized as Statistic::Quantile with the quantile, and reported as, for example, ns p99. A metric reported at different quantiles is a different definition, flagged rather than compared.
  • Records are now version 2 and summaries version 2, since both can contain quantiles. The runner still accepts version 1 records, so workloads built against 0.1 keep working. A 0.1 runner rejects 0.2 records: with a version mismatch, or for records with quantiles, with an unknown variant quantile.
  • Breaking: Measurement and Statistic gain variants and are now #[non_exhaustive], so matches on them need a wildcard arm. Statistic is no longer Eq, since it now carries the quantile.
  • Verify executables without reading them again while their identity is unchanged: device, inode, size, and modification time on Unix, taken from the opened file. Copies made when staging cannot hard-link inherit their source's verified hash. Every sample and every campaign verifies its executable, so smoke runs of unoptimized test builds spent most of their time hashing; a 19-scenario smoke test of a 50 MB debug executable took 27 s. A same-size rewrite that keeps or restores the modification time, or happens within the file system's timestamp granularity, is no longer detected within one process. sha256_identified is public.
  • Keep the collector's LLVM_PROFILE_FILE in the workload environment outside recorded campaigns, so that coverage runs collect workload profiles instead of leaving default_*.profraw files in the workspace.