Add warning messages when algorithms are skipped in autotune benchmarking #723
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Problem
When algorithms exceed
maxtime
during benchmarking, they are correctly skipped for larger matrix sizes. However, users only saw the initial "exceeded maxtime" warning but not when algorithms were subsequently skipped. This made it unclear why some algorithms weren't being tested on larger matrices.Users would see logs like:
But then would not see any indication that these algorithms were actually being skipped on the larger 15000×15000 matrices, leading to confusion about whether the skipping logic was working.
Solution
Added a warning message in
benchmarking.jl:151
when algorithms are skipped:Now users see both:
Algorithm X exceeded maxtime (Y.Zs > maxtime) for size N
Algorithm X skipped for size M (exceeded maxtime on size N matrix)
Test Plan
The fix is a simple one-line addition that improves user experience without changing any algorithmic behavior.
🤖 Generated with Claude Code