New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CLI - Add command sb benchmark [list,list-parameters] #279
Conversation
Add command `sb benchmark list` and `sb benchmark list-parameters` to support listing all optional paramters for benchmarks.
Fix unit test.
Add docs and tests.
Codecov Report
@@ Coverage Diff @@
## main #279 +/- ##
==========================================
+ Coverage 88.53% 88.61% +0.08%
==========================================
Files 75 76 +1
Lines 4368 4401 +33
==========================================
+ Hits 3867 3900 +33
Misses 501 501
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
Update command description.
self.cmd('sb benchmark list-parameters', checks=[NoneCheck()]) | ||
self.cmd('sb benchmark list-parameters -n pytorch-[a-z]+', checks=[NoneCheck()]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to check the exit_code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
by default expect_failure
is False
so exit code is expected to be 0, otherwise the case will fail
Description
Add command
sb benchmark list
andsb benchmark list-parameters
to support listing all optional parameters for benchmarks.Examples
Major Revisions
sb benchmark list
to list benchmarks matching given name.sb benchmark list-parameters
to list parameters for benchmarks which match given name.Minor Revisions