Test run customization documentation #226
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nbhoski
reviewed
Oct 6, 2022
CONFIGDOC.md
Outdated
| To customize your test run, select options in the **Customize Test Run** section: | ||
|
|
||
| * To apply strict checks when running the tests, select **Strict**. When you select this option, the plugin generates a qualification failure whenever a test issues a warning. Selecting **Strict** is the same as specifying the `Strict` name-value argument of [`runtests`](https://www.mathworks.com/help/matlab/ref/runtests.html) as `true`. | ||
| * To run tests in parallel (requires Parallel Computing Toolbox™), select **Use Parallel**. This feature might not be compatible with certain options, in which case, the plugin runs the tests in serial regardless of your selection. Selecting **Use parallel** is the same as specifying the `UseParallel` name-value argument of `runtests` as `true`. |
There was a problem hiding this comment.
Should we Add the disclaimer that this will work only if user has valid PCT licence installed ?
There was a problem hiding this comment.
Strengthened the tone in the new version. Let me know what you think.
nbhoski
reviewed
Oct 6, 2022
CONFIGDOC.md
Outdated
| * To apply strict checks when running the tests, select **Strict**. When you select this option, the plugin generates a qualification failure whenever a test issues a warning. Selecting **Strict** is the same as specifying the `Strict` name-value argument of [`runtests`](https://www.mathworks.com/help/matlab/ref/runtests.html) as `true`. | ||
| * To run tests in parallel (requires Parallel Computing Toolbox™), select **Use Parallel**. This feature might not be compatible with certain options, in which case, the plugin runs the tests in serial regardless of your selection. Selecting **Use parallel** is the same as specifying the `UseParallel` name-value argument of `runtests` as `true`. | ||
| * To control the amount of event detail displayed for your test run, select a value from the **Output detail** list. Selecting a value for this option is the same as specifying the `OutputDetail` name-value argument of `runtests` as that value. | ||
| * To display diagnostics logged by the [`log (TestCase)`](https://www.mathworks.com/help/matlab/ref/matlab.unittest.testcase.log.html) or [`log (Fixture)`](https://www.mathworks.com/help/matlab/ref/matlab.unittest.fixtures.fixture.log.html) methods at a specified verbosity level and below, select a value from the **Logging level** list. Selecting a value for this option is the same as specifying the `LoggingLevel` name-value argument of `runtests` as that value. |
There was a problem hiding this comment.
For both Logging level and Output details should we mention what default selection does ?
There was a problem hiding this comment.
Added information on default behavior.
nbhoski
reviewed
Oct 6, 2022
CONFIGDOC.md
Outdated
|
|
||
| Input | Description | ||
| ------------------------- | --------------- | ||
| `sourceFolder` | (Optional) Location of the folder containing source code, relative to the project root folder. The specified folder and its subfolders are added to the top of the MATLAB search path. If you specify `sourceFolder` and then generate a coverage report, the plugin uses only the source code in the specified folder and its subfolders to generate the report.<br/>**Example:** `['source']`<br/>**Example:** `['source/folderA', 'source/folderB']` | ||
| `selectByFolder` | (Optional) Location of the folder used to select test suite elements, relative to the project root folder. To create a test suite, the plugin uses only the tests in the specified folder and its subfolders.<br/>**Example:** `['test']`<br/>**Example:** `['test/folderA', 'test/folderB']` | ||
| `selectByTag` | (Optional) Test tag used to select test suite elements. To create a test suite, the plugin uses only the test elements with the specified tag.<br/>**Example:** `'FeatureA'` | ||
| `strict` | (Optional) Whether to apply strict checks when running the tests, specified as `false` or `true`. By default, the value is `false`. If you specify a value of `true`, the plugin generates a qualification failure whenever a test issues a warning. | ||
| `useParallel` | (Optional) Whether to run tests in parallel, specified as `false` or `true`. By default, the value is `false` and tests run in serial. If the test runner configuration is suited for parallelization, you can specify a value of `true` to run tests in parallel. This input requires a Parallel Computing Toolbox™ license. | ||
| `outputDetail` | (Optional) Amount of event detail displayed for the test run, specified as `none`, `terse`, `concise`, `detailed`, or `verbose`. Selecting a value for this input is the same as specifying the `OutputDetail` name-value argument of [`runtests`](https://www.mathworks.com/help/matlab/ref/runtests.html) as that value. |
There was a problem hiding this comment.
there is an issue to consider here that when someone specify the OutPutDetail or Logging level in pipeline then only values supported are as described in document. Default as value will throw an warnning and fall backs to the default state of the level which is 4 or detailed. closes issue #224 . per me we can clearly mention this in quick note section. so that we can avoid this as Xbug
There was a problem hiding this comment.
Added information on the default behavior. Not sure whether we need a note because the users are provided with a finite list of options and should choose from that list.
mcafaro
approved these changes
Oct 10, 2022
davidbuzinski
approved these changes
Oct 10, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.