Skip to content

Add combined results and plot generation #219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
avolkov-intel wants to merge 8 commits into
base: main
Choose a base branch
from
+588 −1
Open

Add combined results and plot generation #219

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ca76b25
Add an option --combined-results to create list with all use cases
avolkov-intel Feb 19, 2026
0f007d3
Add options to draw comparison plots
avolkov-intel Feb 19, 2026
b0f520d
Handle n_jobs param correctly and add Summary 2 and All cases 2 pages
avolkov-intel Jun 2, 2026
c77e14c
Add footnote for plot and add metrics
avolkov-intel Aug 12, 2026
3946b64
Update footnote
avolkov-intel Aug 17, 2026
d1dbe1e
Update docs
avolkov-intel Aug 17, 2026
2341e70
Code format
avolkov-intel Aug 17, 2026
2078312
Address comments
avolkov-intel Sep 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions sklbench/report/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ Raw results are converted into a pandas dataframe and the final report is made b
| `--report-file` | str | report.xlsx | | Report file path. |
| `--report-type` | str | separate-tables | ('separate-tables',) | Report type ("separate-tables" is the only supported now). |
| `--compatibility-mode` | | False | | [EXPERIMENTAL] Compatibility mode drops and modifies results to make them comparable (for example, sklearn and cuML parameters). |
| `--performance-stability-metrics`</br>`-psm` | | False | | Adds performance stability metrics (`1st run time[ms]`, `1st-mean run ratio`, `median time[ms]`, `time CV`) to the report. |
| `--combined-results` | | False | | [EXPERIMENTAL] Creates `All cases` and `Summary (for plots)` sheets combining time and speedup of all algorithms with per-dtype/total GEOMEAN rows (use with `--compatibility-mode`). |
| `--draw-plots` | | False | | [EXPERIMENTAL] Draws Training/Inference speedup bar charts from the combined results (requires `--combined-results`). Footnote hardware parameters are hardcoded; edit them in the code for a production-ready plot. |
| `--plot-output` | str | `plot.png` | | [EXPERIMENTAL] Output file path for plots (e.g. `plots.png`); if unset, plots are saved to `plot.png`. Only takes effect together with `--draw-plots`. |
| `--drop-columns`</br>`--drop-cols` | str | [] | | Columns to drop from report. |
| `--diff-columns`</br>`--diff-cols` | str | ['environment_name', 'library', 'format', 'device'] | | Columns to show difference between. |
| `--split-columns` | str | ['estimator', 'method', 'function'] | | Splitting columns for subreports/sheets. |
Expand Down
22 changes: 22 additions & 0 deletions sklbench/report/arguments.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,28 @@ def add_report_generator_arguments(
help="Selects which part of one-vs-one difference to show "
"(all matrix or one of triangles).",
)
parser.add_argument(
"--combined-results",
default=False,
action="store_true",
help="[EXPERIMENTAL] Creates 'All cases' and 'Summary (for plots)' sheets "
"combining time[ms] and speedup of all algorithms into single pages with "
"per-dtype and total GEOMEAN rows. Use together with --compatibility-mode.",
)
parser.add_argument(
"--draw-plots",
default=False,
action="store_true",
help="[EXPERIMENTAL] Draw Training/Inference speedup bar charts from combined "
"results (requires --combined-results).",
)
parser.add_argument(
"--plot-output",
type=str,
default="plot.png",
help="[EXPERIMENTAL] Output file path for plots (e.g., plots.png). "
"If not specified, plots are saved to plot.png.",
)
# color scale settings
parser.add_argument(
"--perf-color-scale",
Expand Down
6 changes: 6 additions & 0 deletions sklbench/report/compatibility.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ def transform_results_to_compatible(results: pd.DataFrame):
"min_bin_size",
],
)
if (
"n_jobs" in results.columns
and results["n_jobs"].isna().any()
and results["n_jobs"].notna().any()
Comment thread
ethanglaser marked this conversation as resolved.
):
results.drop(inplace=True, columns=["n_jobs"])
if results["environment_name"].unique().size > 1:
# DBSCAN `eps` parameter drop for different CPUs
results.drop(
Expand Down
Loading
Loading