Add indexes to test names so that Specmatic always generates tests with unique names. #1082
Conversation
…d wih an index so that the test name is always unique.
…scenarios. Fixed space format issue in index label. Added test for negative test scenarios.
| @@ -365,19 +365,13 @@ data class Feature( | |||
| val negativeTestScenarios = | |||
| negativeScenario.generateTestScenarios(flagsBased, testVariables, testBaseURLs) | |||
|
|
|||
| negativeTestScenarios.filterNot { negativeTestScenarioR -> | |||
| val filtered = negativeTestScenarios.filterNot { negativeTestScenarioR -> | |||
There was a problem hiding this comment.
Can we please rename filtered to what it actually contains? This will help readability of line where filtered is being used?
There was a problem hiding this comment.
| @@ -365,19 +365,13 @@ data class Feature( | |||
| val negativeTestScenarios = | |||
| negativeScenario.generateTestScenarios(flagsBased, testVariables, testBaseURLs) | |||
|
|
|||
| negativeTestScenarios.filterNot { negativeTestScenarioR -> | |||
| val filtered = negativeTestScenarios.filterNot { negativeTestScenarioR -> | |||
There was a problem hiding this comment.
not sure I understand why we are naming this negativeTestScenarioR? what is the R? Can you please check?
There was a problem hiding this comment.
negativeTestScenarioRis essentially a temporary variable used to hold each individual negative test scenario as it's being evaluated by thefilterNotfunction.- This will be renamed to
currentNegativeTestScenario. So the name indicates that this is the current negative test scenario being evaluated in the filter operation.
There was a problem hiding this comment.
| @@ -400,6 +394,27 @@ data class Feature( | |||
| serverState = emptyMap() | |||
| } | |||
|
|
|||
| private fun addIndexes( | |||
There was a problem hiding this comment.
Technically this method is adding both +v/-ve prefix as well as index, should it be named appropriately?
There was a problem hiding this comment.
Will also be super helpful to unit test this method separately. While the feature level tests are very useful, this will allow is try all difficult combinations.
There was a problem hiding this comment.
Is this method doing too much, should we separate out application of prefix and index as two separate collections.map operations?
There was a problem hiding this comment.
ToDo's
- Technically this method adds both the +v/-ve prefix and the index, should it be named appropriately?
- It will also be super helpful to unit test this method separately. While the feature-level tests are very useful, this will allow us to try all difficult combinations.
- Is this method doing too much, should we separate the application of prefix and index as two separate collections.map operations?
…run only tests filtered out by name in test, due to incorrect position of index in test names.
…n-string non-spec-conformant examples
…rio' for clarity"
What:
At present, Specmatic adds an index suffix to all negative tests with the same name, however this behaviour is not implemented for positive tests. This results in Specmatic generating positive tests with the same name duplicated.
This PR enforces a consistent behaviour so that, whenever a group of tests [ Positive or Negative ] with the same name are generated, an index is suffixed to the test name so that all tests have unique names.
Why:
Duplicate test names result in python wrapper reporting an incorrect test count.
Checklist: