Skip to content

Benchmark v0.8.0 - Beta

Pre-release
Pre-release

Choose a tag to compare

View all tags
@jamesgober jamesgober released this 04 Sep 22:58
v0.8.0
bfb4507

Version 0.8.0-beta RC1 - 2025-09-04

Feature-complete release candidate introducing advanced histogram strategies, production sampling controls, and enhanced observability integrations. This beta represents the next evolution of rust-benchmark as a comprehensive performance toolkit.


Features

  • Configurable histogram strategies for improved percentile accuracy across diverse workloads.
  • Production sampling controls to manage overhead in high-throughput systems.
  • Export formats supporting OpenTelemetry, Prometheus, and JSON pipelines.
  • Warm-up detection automatically excludes JIT compilation from measurements.
  • Memory tracking is an optional measurement of allocation patterns alongside timing.

Highlights

  • HDR-inspired histogram: Log-scale buckets for 99.99th percentile accuracy without the overhead.
  • Adaptive sampling: Automatically adjusts collection rate based on system load.
  • Zero-allocation exports: Streaming formatters for metrics without memory pressure.
  • Coordinated omission handling: Detects and corrects for missed measurements under load.
  • Framework integrations: First-class support for Axum, Actix, and Rocket middleware.

Changes

Added

  • Histogram strategies: Linear, Logarithmic, and Adaptive bucketing modes.
  • Sampling API: Watch::with_sampler() for production rate control.
  • Export traits: MetricExporter for pluggable output formats.
  • Warm-up detection: benchmark_with_warmup! macro for accurate cold-start handling.
  • Memory profiling: measure_with_allocs() tracks both time and allocations.
  • Percentile presets: Common queries (p50, p90, p95, p99) with optimized paths.

Enhanced

  • Histogram performance: 3x faster percentile queries via binary search optimization.
  • Watch efficiency: Lock-free fast path for high-frequency recordings.
  • Timer precision: Monotonic clock adjustments for sub-microsecond accuracy.
  • Collector stats: Streaming algorithms for variance and standard deviation.
  • Documentation: Real-world examples from production deployments.

𖢥 Bug Fixes

  • Histogram overflow: Fixed saturation handling for nanosecond values exceeding 2^53.
  • Timer edge case: Corrected double-drop scenario in nested panic handlers.
  • Export precision: JSON formatter now preserves full nanosecond precision.
  • Sampling bias: Fixed systematic undersampling of short operations.

Minimum Supported Rust Version (MSRV)

1.75.0 (unchanged)


Migration Notes

  • From v0.7.x: Mostly compatible. Histogram API has new options but defaults unchanged.
  • Breaking: Watch::new() now returns Watch<LinearHistogram>. Use Watch::with_histogram() for custom strategies.
  • Deprecation: Collector::stats() deprecated in favor of streaming Collector::summarize().

Quick Migration

// Old (still works with linear histogram)
let watch = Watch::new();

// New (explicit histogram choice)
let watch = Watch::with_histogram(LogarithmicHistogram::new());

// Production setup with sampling
let watch = Watch::builder()
    .histogram(AdaptiveHistogram::new())
    .sampler(AdaptiveSampler::new(0.1)) // 10% sampling
    .build();

Export Examples

// Prometheus format
let mut buffer = String::new();
watch.export(&mut PrometheusExporter::new(&mut buffer))?;

// OpenTelemetry
let exporter = OtelExporter::new(endpoint);
watch.export(&mut exporter)?;

// Custom JSON streaming
let mut writer = BufWriter::new(file);
watch.export(&mut JsonExporter::new(&mut writer))?;

Performance Characteristics

Benchmarked on M1 Pro (10-core):

  • Linear histogram: 85ns record, 12ns percentile query (unchanged)
  • Logarithmic histogram: 92ns record, 8ns percentile query (new)
  • Adaptive histogram: 88-95ns record, 6-15ns query (new)
  • With 10% sampling: ~10ns average for skipped measurements
  • Export overhead: <1µs per metric for all formats

Memory usage:

  • Linear: 8KB fixed (1024 buckets)
  • Logarithmic: 2KB fixed (256 buckets)
  • Adaptive: 1-16KB dynamic

Beta Testing Focus

We need feedback on:

  • Histogram accuracy: Do the new strategies match your workload patterns?
  • Sampling strategies: Is adaptive sampling hitting the right balance?
  • Export formats: Missing fields or formatting issues?
  • API ergonomics: Is the builder pattern intuitive?
  • Performance impact: Any regression in your hot paths?

Testing Checklist

# Test new histogram strategies
cargo test --features "metrics,histogram-strategies"

# Benchmark your specific workload
cargo bench --features "all"

# Verify export formats
cargo run --example export_formats --features "exporters"

# Check memory usage
valgrind --tool=massif cargo run --example long_running

CI Enhancements

New validation for v0.8.0:

  • ✅ Histogram accuracy tests: Statistical validation across value distributions
  • ✅ Export format validation: Round-trip tests for all exporters
  • ✅ Sampling correctness: Chi-squared tests for sampling uniformity
  • ✅ Memory leak detection: Valgrind on long-running examples
  • ✅ Framework integration tests: Middleware tests for web frameworks
  • ✅ Performance regression suite: Automated alerts for >5% slowdowns

Release Candidate Status

Beta Quality - Not Yet Production Ready

✅ What's Ready:

  • Core histogram strategies fully tested
  • Export formats validated against spec
  • API surface stable (pending feedback)
  • Performance meets or exceeds v0.7.x

⚠️ What's In Progress:

  • Documentation for new features (70% complete)
  • Integration examples being collected
  • Community feedback incorporation
  • Final performance tuning

🚧 Known Issues:

  • Adaptive histogram may over-allocate in pathological cases
  • JSON exporter doesn't yet support custom field names
  • Some framework integrations need middleware updates

Roadmap to 0.8.0 Stable

  1. 2 week beta period for community testing
  2. Address feedback on API and performance
  3. Complete documentation with production examples
  4. Final benchmark validation across platforms
  5. Stable release targeted for late September

Breaking Changes Summary

  • Watch::new() type signature changed (but source-compatible)
  • Collector::stats() → Collector::summarize() (deprecated, not removed)
  • Minimum bucket count reduced from 1024 to 256 for logarithmic histogram
  • Export traits in new exporters module (feature-gated)

Full Changelog: v0.7.2...v0.8.0-beta

Status: 🚧 BETA - Feature complete, seeking feedback

Assets 2
Loading