Remove annoying error message #5956
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
cost-model-budgeting-benchexecutable runs benchmarks for the builtins but also runs some no-op benchmarks to give us an idea of the overhead involved in calling a builtin. To get accurate results for the no-op benchmarks we run them with a bigger time limit, which involves calling Criterion twice. If you asked it to benchmark some subset of the builtins (eg, by passingEqualsStringon the command line) then there would be an error with the message "none of the specified names matches a benchmark" when it ran the no-op benchmarks (because none of them do match). This was annoying and also meant that you couldn't run a subset of the benchmarks on the benchmarking machine and get the results back. This PR removes the error mesage. Now it's less annoying, but if you ask it to benchmark something that doesn't exist (maybe because you mistype a name) then it'll complete silently and leave you with a CSV file containing only a header. This might be a bit mystifying, but it's better than a spurious error message. We could maybe do better if we were to bypass the Criterion command-line processing code altogether, but that might be quite a lot of work for a small gain.I don't think this needs to be reviewed, so I'll set it to auto-merge.