Skip to content

feat(bench): add poll measurements and comparison tooling - #4122

Draft
diegomrsantos wants to merge 3 commits into
apache:masterfrom
diegomrsantos:feat/poll-benchmark-campaign
Draft

feat(bench): add poll measurements and comparison tooling#4122
diegomrsantos wants to merge 3 commits into
apache:masterfrom
diegomrsantos:feat/poll-benchmark-campaign

Conversation

@diegomrsantos

Copy link
Copy Markdown

Which issue does this PR address?

Relates to #4117. The benchmark tooling was extracted from #4119 so it can be reviewed independently of the poll completion fix.

Rationale

Comparing changes to polling needs explicit control over polling behavior and latency measurement, together with raw observations and a repeatable workload setup.

What changed?

The benchmark client adds optional polling and latency selectors, raw poll observations, run manifests, and producer samples. Failed and interrupted runs retain their artifacts, and reports handle short runs with empty time series. Existing selector defaults and the report JSON schema are preserved.

A Python runner prepares workloads, records binary hashes and process samples, schedules paired runs, and compares results. The tooling builds against current master and does not require the server changes in #4119. The documentation describes measurement limits and treats server diagnostics as optional.

Local Execution

  • Passed cargo test --locked -p iggy-bench -p bench-report: 42 tests.
  • Passed all nine Python campaign tests.
  • Passed Clippy for both benchmark crates with all features, all targets, and warnings denied.
  • Passed a local macOS smoke run against a server without the poll fix: 100 completed polls, with artifacts retained and no empty responses or errors. This checks execution, not performance.
  • Passed workspace formatting and dependency sort checks, changed TOML formatting, Markdown lint, spelling, license headers, whitespace, and trailing newline checks.
  • Pre-commit hooks are not installed in this checkout; the checks above were run directly.

No new performance comparison is claimed by this extraction. The previously recorded measurements remain in #4119.

AI Usage

Codex was used for the extraction, dependency review, standalone documentation adjustments, and local verification listed above. This PR is a draft for further review.

Describe comparisons between arbitrary server builds and require an explicit seccomp profile path. Treat poll diagnostics as an optional capability of the server revision being measured.
@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.01%. Comparing base (df4c75d) to head (7b397b4).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #4122      +/-   ##
============================================
- Coverage     86.02%   86.01%   -0.01%     
+ Complexity     1451     1449       -2     
============================================
  Files          1248     1248              
  Lines        194879   195623     +744     
  Branches     160177   160921     +744     
============================================
+ Hits         167637   168273     +636     
- Misses        23141    23198      +57     
- Partials       4101     4152      +51     
Components Coverage Δ
Rust Core 86.92% <ø> (-0.01%) ⬇️
Java SDK 67.54% <ø> (-0.02%) ⬇️
C# SDK 76.98% <ø> (-0.09%) ⬇️
Python SDK 91.33% <ø> (ø)
PHP SDK 85.65% <ø> (ø)
Node SDK 96.26% <ø> (ø)
Go SDK 69.43% <ø> (+0.03%) ⬆️
see 45 files with indirect coverage changes
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant