v1.0.16 — OpenSSL build diagnostics + HW acceleration verification, refreshed dashboards

Overview

This release focuses on making benchmark results more explainable and trustworthy by capturing detailed OpenSSL build diagnostics (including assembly/CPU acceleration signals) and surfacing that information in the generated dashboards. It also improves the robustness of memory sampling and refreshes published results/artifacts.

What changed

Benchmarking: capture build diagnostics and verify hardware acceleration

The benchmark runner now records additional environment/build metadata and performs a runtime verification to confirm whether OpenSSL is actually using hardware acceleration.

New metadata captured in summary.json:

• metadata.build_diagnostics
  • version_full: output from openssl version -a
  • openssl_options: output from openssl version -o
  • asm_flags_detected: boolean flags for common ASM indicators (AES/SHA/Poly1305/BN/ECP)
  • hw_accel_verification: a runtime “proof” test that compares AES-256-GCM throughput:
    • normal run (“with HW accel”)
    • run with CPU capabilities masked (OPENSSL_ia32cap on x86_64, OPENSSL_armcap on aarch64)
    • outputs include verified, speedup_ratio, and raw *_kbs values
  • arm (aarch64 only): availability of ARM crypto extensions and whether OpenSSL build appears to enable ARM ASM

Why it matters:
Performance regressions/improvements between OpenSSL versions are often dominated by build configuration and whether assembly/hardware paths are actually active. This release makes those factors explicit so benchmark output is easier to interpret and compare across environments.

Impact:

• Slightly longer benchmark runtime due to additional diagnostics and quick speed checks.
• More verbose stderr logs (JSON output is still on stdout).

Memory sampling: more robust and less brittle

src/measure_memory.sh was hardened to avoid failing the benchmark due to missing tools or platform differences:

• No longer uses set -e (handles errors gracefully).
• Works without bc (falls back to bash arithmetic for sampling interval and averaging).
• Checks for /proc availability and process existence; exits cleanly if unavailable.
• Handles process termination during sampling.

Why it matters:
Memory metrics collection is now less likely to fail in minimal or constrained environments.

Docker image: include required utilities + add build/ASM verification steps

The benchmark Docker image now:

• Installs bc and procps (needed for timing/math and process inspection).
• Adds build-time and post-install checks intended to catch misconfigured builds:
  • greps configdata.pm and Makefile for ASM-related indicators
  • runs openssl version -a after install
  • attempts to locate common ASM symbols in libcrypto via nm

Why it matters:
Helps prevent “silent” benchmark runs where OpenSSL was built without expected optimizations.

Dashboards/visualizations: new build-info page and formatting improvements

Visualization generators were updated (single-page and multipage):

• Adds a “Build Configuration” page that reads from the new metadata and summarizes build/acceleration status.
• Standardizes chart styling:
  • larger axis label font sizes
  • consistent tick formatting using k suffix (lowercase)
• Updates the shared color scale to include the 3.6 series.
• Pages now include a versioned header/footer and additional run context (e.g., last run date from summary.json mtime and per-version iteration counts when present).

Results/artifacts refreshed

• results/*.html and results/summary.json were regenerated to reflect the latest benchmark run and updated templates.
• Release housekeeping: package.json and package-lock.json updated to 1.0.16; generated tooling artifacts under node_modules/ refreshed.

Breaking changes

No breaking API changes were detected.

Compatibility note (for downstream consumers):

• The summary.json schema is extended with a new metadata.build_diagnostics object. This is additive, but any consumers that strictly validate/whitelist fields should allow unknown keys.

Practical guidance

• If you’re investigating unexpected performance differences, start with the new Build Configuration page (or inspect metadata.build_diagnostics in summary.json) to confirm:
  • OpenSSL build options and full version info
  • ASM flags detected
  • whether hardware acceleration was verified via the throughput delta test
• For consistent memory sampling, bc is preferred, but the scripts now degrade gracefully without it.