Skip to content

Conversation

@sjmonson
Copy link
Collaborator

Summary

Fixes a bug when an HTML benchmark reports is generated. The HTML report crashes in the browsers with e.g. Error: xs[2] == xs[3] when rendering distributions with consecutive duplicate percentile values.

This occurs when benchmarks have limited data points (e.g., < 10 requests), causing multiple percentiles to collapse to the same value. The visualization library expects strictly increasing x-values and fails when encountering duplicates.

Root Cause

When a benchmark has very few successful requests (e.g., 3 requests), percentile calculations result in many duplicate values:

# Before fix - causes visualization error
{
  "p001": 15.288,
  "p01": 15.288,   # duplicate
  "p05": 15.288,   # duplicate
  "p10": 15.288,   # duplicate
  "p25": 15.288,   # duplicate
  "p50": 16.413,   # unique
  "p75": 16.413,   # duplicate
  "p90": 17.035,   # unique
  ...
}

# After fix - renders successfully
{
  "p001": 15.288,
  "p50": 16.413,
  "p90": 17.035
}

Details

  • Added _filter_duplicate_percentiles() helper function to remove consecutive duplicate percentile values
  • Override model_dump() in _TabularDistributionSummary to automatically apply filtering during HTML generation
  • Added comprehensive unit tests (8 test cases) covering all edge cases

Test Plan

Unit Tests

uv run pytest tests/unit/benchmark/test_html_output.py -v
# 8/8 tests passing

Related


  • "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • Includes AI-assisted code completion
  • Includes code generated by an AI application
  • Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

…value

Signed-off-by: Joseph Berry <joberry@redhat.com>
Signed-off-by: Joseph Berry <joberry@redhat.com>
…or accuracy

Signed-off-by: Joseph Berry <joberry@redhat.com>
Signed-off-by: Joseph Berry <joberry@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants