FEAT: sort scenario per-group breakdown by success rate#1809
Merged
hannahwestra25 merged 5 commits intoMay 26, 2026
Merged
Conversation
Add an opt-in 'sort_groups_by_success_rate' parameter to PrettyScenarioResultPrinter (and its memory subclass). When enabled, the Per-Group Breakdown section renders groups in descending success-rate order so the most successful groups appear at the top. Sort is stable, so ties keep their original ordering. Default behavior is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Surface the new sort flag on output_scenario_async and add a section to doc/code/scenarios/1_common_scenario_parameters that shows how to render the Per-Group Breakdown with the most successful groups at the top. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the sort_groups_by_success_rate example to follow the baseline run, where the default strategy set produces enough rows to make sorting visibly useful. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Inject the sorted rendering of baseline_result into the sort_groups_by_success_rate cell so the notebook visibly demonstrates the feature — base64 (100%) and the three 50% strategies float to the top, 0% groups fall through in their original order. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Address review feedback on the per-group sort: store (name, count, rate) tuples with parameterized types, drop the redundant len(group_results) call in the render loop, and add a test that verifies output_scenario_async forwards sort_groups_by_success_rate to the printer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
behnam-o
approved these changes
May 26, 2026
hannahwestra25
deleted the
hannahwestra25/sort-scenario-results-by-success
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an opt-in
sort_groups_by_success_rateparameter toPrettyScenarioResultPrinter(and thePrettyScenarioResultMemoryPrintersubclass). When enabled, the Per-Group Breakdown section is rendered with the highest-success-rate group at the top, making it easier to scan results at a glance.Behavior
False— current insertion-order behavior is unchanged.Usage
Tests
Added 3 new tests covering:
All 167 existing tests in
tests/unit/output/continue to pass;pre-commit(ruff, ty) is clean.