diff --git a/docs/data_driven_testing.adoc b/docs/data_driven_testing.adoc index 7be116de12..50df634b5a 100644 --- a/docs/data_driven_testing.adoc +++ b/docs/data_driven_testing.adoc @@ -402,7 +402,7 @@ such a method. == Unrolled Iteration Names -By default the names of unrolled iterations are the name of the feature, plus the data variables and the iteration +By default, the names of unrolled iterations are the name of the feature, plus the data variables and the iteration index. This will always produce unique names and should enable you to identify easily the failing data variable combination. @@ -468,35 +468,74 @@ include::{sourcedir}/datadriven/DataSpec.groovy[tag=unrolled-3a] include::{sourcedir}/datadriven/DataSpec.groovy[tag=unrolled-3b] ---- -Additionally to the data variables the tokens `#featureName` and `#iterationIndex` are supported. +Additionally, to the data variables the tokens `#featureName` and `#iterationIndex` are supported. The former does not make much sense inside an actual feature name, but there are two other places -where an unroll pattern can be defined, where it is more useful. +where an unroll-pattern can be defined, where it is more useful. [source,groovy,indent=0] ---- -def "#person is #person.age years old [iterationIndex: #iterationIndex]"() { +def "#person is #person.age years old [#iterationIndex]"() { ---- -Alternatively to specifying the unroll pattern as method name, it can be given as parameter +will be reported as + +---- +╷ +└─ Spock ✔ + └─ PersonSpec ✔ + └─ #person.name is #person.age years old [#iterationIndex] ✔ + ├─ Fred is 38 years old [0] ✔ + ├─ Wilma is 36 years old [1] ✔ + └─ Pebbles is 5 years old [2] ✔ +---- + +Alternatively, to specifying the unroll-pattern as method name, it can be given as parameter to the `@Unroll` annotation which takes precedence over the method name: [source,groovy,indent=0] ---- @Unroll("#featureName[#iterationIndex] (#person.name is #person.age years old)") def "person age should be calculated properly"() { +// ... +---- + +will be reported as + ---- +╷ +└─ Spock ✔ + └─ PersonSpec ✔ + └─ person age should be calculated properly ✔ + ├─ person age should be calculated properly[0] (Fred is 38 years old) ✔ + ├─ person age should be calculated properly[1] (Wilma is 36 years old) ✔ + └─ person age should be calculated properly[2] (Pebbles is 5 years old) ✔ +---- + +The advantage is, that you can have a descriptive method name for the whole feature, while having a separate template for each iteration. +Furthermore, the feature method name is not filled with placeholders and thus better readable. If neither a parameter to the annotation is given, nor the method name contains a `#`, the <> setting `defaultPattern` in the `unroll` section is inspected. If it is set to a non-`null` -string, this value is used as unroll pattern. This could for example be set to +string, this value is used as unroll-pattern. This could for example be set to - `#featureName` to have all iterations reported with the same name, or - `#featureName[#iterationIndex]` to have a simply indexed iteration name, or - `#iterationName` if you make sure that in each data-driven feature you also set a data variable called `iterationName` that is then used for reporting -.Set Default Unroll Pattern +=== Special Tokens + +This is the complete list of special tokens: + +- `#featureName` is the name of the feature (mostly useful for the `defaultPattern` setting) +- `#iterationIndex` is the current iteration index +- `#dataVariables` lists all data variables for this iteration, e.g. `x: 1, y: 2, z: 3` +- `#dataVariablesWithIndex` the same as `#dataVariables` but with an index at the end, e.g. `x: 1, y: 2, z: 3, #0` + +=== Configuration + +.Set Default Unroll-Pattern [source,groovy] ---- unroll { @@ -504,14 +543,14 @@ unroll { } ---- -If none of the three described ways is used to set a custom unroll pattern, by default +If none of the three described ways is used to set a custom unroll-pattern, by default the feature name is used, suffixed with all data variable names and their values and finally the iteration index, so the result will be for example `my feature [x: 1, y: 2, z: 3, #0]`. If there is an error in an unroll expression, for example typo in variable name, exception during evaluation of a property or method in the expression and so on, the test will fail. This is not -true for the automatic fall back rendering of the data variables if there is no unroll pattern +true for the automatic fall back rendering of the data variables if there is no unroll-pattern set in any way, this will never fail the test, no matter what happens. The failing of test with errors in the unroll expression can be disabled by setting the @@ -519,10 +558,45 @@ The failing of test with errors in the unroll expression can be disabled by sett in the `unroll` section to `false`. If this is done and an error happens, the erroneous expression `#foo.bar` will be substituted by `#Error:foo.bar`. -.Disable Unroll Pattern Expression Asserting +.Disable Unroll-pattern Expression Asserting [source,groovy] ---- unroll { validateExpressions false } ---- + +Some reporting frameworks, or IDEs support proper tree based reporting. +For these cases it might be desirable to omit the feature name from the iteration reporting. + +.Disable repetition of feature name in iterations +[source,groovy] +---- +unroll { + includeFeatureNameForIterations false +} +---- + +With `includeFeatureNameForIterations true` +---- +╷ +└─ Spock ✔ + └─ ASpec ✔ + └─ really long and informative test name that doesn't have to be repeated ✔ + ├─ really long and informative test name that doesn't have to be repeated [x: 1, y: a, #0] ✔ + ├─ really long and informative test name that doesn't have to be repeated [x: 2, y: b, #1] ✔ + └─ really long and informative test name that doesn't have to be repeated [x: 3, y: c, #2] ✔ +---- + +.With `includeFeatureNameForIterations false` +---- +╷ +└─ Spock ✔ + └─ ASpec ✔ + └─ really long and informative test name that doesn't have to be repeated ✔ + ├─ x: 1, y: a, #0 ✔ + ├─ x: 2, y: b, #1 ✔ + └─ x: 3, y: c, #2 ✔ +---- + +NOTE: The same can be achieved for individual features by using `@Unroll('#dataVariablesWithIndex')`. diff --git a/spock-core/src/main/java/org/spockframework/runtime/DataVariablesIterationNameProvider.java b/spock-core/src/main/java/org/spockframework/runtime/DataVariablesIterationNameProvider.java index af765a44e3..68aed2ff78 100644 --- a/spock-core/src/main/java/org/spockframework/runtime/DataVariablesIterationNameProvider.java +++ b/spock-core/src/main/java/org/spockframework/runtime/DataVariablesIterationNameProvider.java @@ -14,17 +14,30 @@ package org.spockframework.runtime; -import org.spockframework.runtime.model.FeatureInfo; -import org.spockframework.runtime.model.IterationInfo; -import org.spockframework.runtime.model.NameProvider; - -import java.util.Map; -import java.util.StringJoiner; - import static java.lang.String.format; import static org.spockframework.util.RenderUtil.toStringOrDump; +import org.spockframework.runtime.model.*; + +import java.util.*; + public class DataVariablesIterationNameProvider implements NameProvider { + private final boolean includeFeatureNameForIterations; + private final boolean includeIterationIndex; + + public DataVariablesIterationNameProvider() { + this(true, true); + } + + public DataVariablesIterationNameProvider(boolean includeFeatureNameForIterations) { + this(includeFeatureNameForIterations, true); + } + + public DataVariablesIterationNameProvider(boolean includeFeatureNameForIterations, boolean includeIterationIndex) { + this.includeFeatureNameForIterations = includeFeatureNameForIterations; + this.includeIterationIndex = includeIterationIndex; + } + @Override public String getName(IterationInfo iteration) { FeatureInfo feature = iteration.getFeature(); @@ -32,11 +45,10 @@ public String getName(IterationInfo iteration) { return feature.getName(); } - StringJoiner nameJoiner = new StringJoiner(", ", "[", "]"); + StringJoiner nameJoiner = new StringJoiner(", "); Map dataVariables = iteration.getDataVariables(); if (dataVariables != null) { - dataVariables.forEach((name, value) -> - { + dataVariables.forEach((name, value) -> { String valueString; try { valueString = toStringOrDump(value); @@ -46,7 +58,9 @@ public String getName(IterationInfo iteration) { nameJoiner.add(format("%s: %s", name, valueString)); }); } - nameJoiner.add(format("#%d", iteration.getIterationIndex())); - return format("%s %s", feature.getName(), nameJoiner.toString()); + if (includeIterationIndex) { + nameJoiner.add(format("#%d", iteration.getIterationIndex())); + } + return includeFeatureNameForIterations ? format("%s [%s]", feature.getName(), nameJoiner) : nameJoiner.toString(); } } diff --git a/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollConfiguration.java b/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollConfiguration.java index 40ea19029a..aac463e369 100644 --- a/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollConfiguration.java +++ b/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollConfiguration.java @@ -26,6 +26,7 @@ * unrollByDefault false // default true * defaultPattern '#featureName[#iterationIndex]' // default null * validateExpressions false // default true + * includeFeatureNameForIterations false // default true * } * * @@ -37,4 +38,5 @@ public class UnrollConfiguration { public boolean unrollByDefault = true; public String defaultPattern = null; public boolean validateExpressions = true; + public boolean includeFeatureNameForIterations = true; } diff --git a/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollExtension.java b/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollExtension.java index eb95fda328..655dc75822 100644 --- a/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollExtension.java +++ b/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollExtension.java @@ -107,6 +107,6 @@ private NameProvider chooseNameProvider(String specUnrollPattern, if (unrollConfiguration.defaultPattern != null) { return new UnrollIterationNameProvider(feature, unrollConfiguration.defaultPattern, unrollConfiguration.validateExpressions); } - return new DataVariablesIterationNameProvider(); + return new DataVariablesIterationNameProvider(unrollConfiguration.includeFeatureNameForIterations); } } diff --git a/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollIterationNameProvider.java b/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollIterationNameProvider.java index af5f1c936f..40b6ccd0da 100644 --- a/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollIterationNameProvider.java +++ b/spock-core/src/main/java/org/spockframework/runtime/extension/builtin/UnrollIterationNameProvider.java @@ -29,6 +29,10 @@ */ public class UnrollIterationNameProvider implements NameProvider { private static final Pattern EXPRESSION_PATTERN = Pattern.compile("#([a-zA-Z_$]([\\w$.]|\\(\\))*)"); + private static final DataVariablesIterationNameProvider DATA_VARIABLES = + new DataVariablesIterationNameProvider(false, false); + private static final DataVariablesIterationNameProvider DATA_VARIABLES_WITH_INDEX = + new DataVariablesIterationNameProvider(false, true); private final boolean validateExpressions; private final FeatureInfo feature; @@ -43,16 +47,16 @@ public UnrollIterationNameProvider(FeatureInfo feature, String namePattern, bool // always returns a name @Override public String getName(IterationInfo iterationInfo) { - return nameFor(iterationInfo.getIterationIndex(), iterationInfo.getDataVariables()); + return nameFor(iterationInfo.getDataVariables(), iterationInfo); } - private String nameFor(int iterationIndex, Map dataVariables) { + private String nameFor(Map dataVariables, IterationInfo iterationInfo) { StringBuffer result = new StringBuffer(); expressionMatcher.reset(); while (expressionMatcher.find()) { String expr = expressionMatcher.group(1); - String value = evaluateExpression(expr, iterationIndex, dataVariables); + String value = evaluateExpression(expr, dataVariables, iterationInfo); expressionMatcher.appendReplacement(result, Matcher.quoteReplacement(value)); } @@ -60,7 +64,7 @@ private String nameFor(int iterationIndex, Map dataVariables) { return result.toString(); } - private String evaluateExpression(String expr, int iterationIndex, Map dataVariables) { + private String evaluateExpression(String expr, Map dataVariables, IterationInfo iterationInfo) { String[] exprParts = expr.split("\\."); String firstPart = exprParts[0]; Object result; @@ -71,7 +75,15 @@ private String evaluateExpression(String expr, int iterationIndex, Map> feature getIterationIndex() >> 99 @@ -77,4 +78,40 @@ class DataVariablesIterationNameProviderSpec extends Specification { expect: testee.getName(iteration) == "$feature.name [x: #Error:RuntimeException during rendering, y: 2, z: 3, #$iteration.iterationIndex]" } + + def 'returns data variables and iteration index when reporting iterations and includeFeatureNameForIterations=false'() { + given: + testee = new DataVariablesIterationNameProvider(false) + iteration.getDataVariables() >> [x: 1, y: 2, z: 3] + + expect: + testee.getName(iteration) == "x: 1, y: 2, z: 3, #$iteration.iterationIndex" + } + + def 'returns data variables only when reporting iterations and includeFeatureNameForIterations=false, includeIterationIndex=false'() { + given: + testee = new DataVariablesIterationNameProvider(false, false) + iteration.getDataVariables() >> [x: 1, y: 2, z: 3] + + expect: + testee.getName(iteration) == "x: 1, y: 2, z: 3" + } + + def 'renders data variables in Groovy style and includeFeatureNameForIterations=false'() { + given: + testee = new DataVariablesIterationNameProvider(false) + iteration.getDataVariables() >> [x: [1], y: [a: 2], z: [3] as int[]] + + expect: + testee.getName(iteration) == "x: [1], y: [a:2], z: [3], #$iteration.iterationIndex" + } + + def 'returns iteration index when reporting iterations but data variables are null and includeFeatureNameForIterations=false'() { + given: + testee = new DataVariablesIterationNameProvider(false) + iteration.getDataVariables() >> null + + expect: + testee.getName(iteration) == "#$iteration.iterationIndex" + } } diff --git a/spock-specs/src/test/groovy/org/spockframework/smoke/parameterization/UnrolledFeatureMethods.groovy b/spock-specs/src/test/groovy/org/spockframework/smoke/parameterization/UnrolledFeatureMethods.groovy index ca02a10eae..8e11a13168 100644 --- a/spock-specs/src/test/groovy/org/spockframework/smoke/parameterization/UnrolledFeatureMethods.groovy +++ b/spock-specs/src/test/groovy/org/spockframework/smoke/parameterization/UnrolledFeatureMethods.groovy @@ -126,6 +126,46 @@ def foo() { "one foo two 2 three"] } + def "naming pattern may refer to dataVariables"() { + when: + def result = runner.runSpecBody(""" +@Unroll("one #dataVariables two") +def foo() { + expect: true + + where: + x << [1, 2, 3] + y << ["a", "b", "c"] +} + """) + + then: + result.testEvents().started().list().testDescriptor.displayName == ["foo", + "one x: 1, y: a two", + "one x: 2, y: b two", + "one x: 3, y: c two"] + } + + def "naming pattern may refer to dataVariablesWithIndex"() { + when: + def result = runner.runSpecBody(""" +@Unroll("one #dataVariablesWithIndex two") +def foo() { + expect: true + + where: + x << [1, 2, 3] + y << ["a", "b", "c"] +} + """) + + then: + result.testEvents().started().list().testDescriptor.displayName == ["foo", + "one x: 1, y: a, #0 two", + "one x: 2, y: b, #1 two", + "one x: 3, y: c, #2 two"] + } + def "old iteration naming is restorable using configuration script"() { given: runner.configurationScript { @@ -136,7 +176,6 @@ def foo() { when: def result = runner.runSpecBody(""" -@Unroll def foo() { expect: true @@ -163,7 +202,6 @@ def foo() { when: def result = runner.runSpecBody(""" -@Unroll def foo() { expect: true @@ -180,6 +218,54 @@ def foo() { "foo"] } + def "includeFeatureNameForIterations is configurable using configuration script"() { + given: + runner.configurationScript { + unroll { + includeFeatureNameForIterations false + } + } + + when: + def result = runner.runSpecBody(""" +def foo() { + expect: true + + where: + x << [1, 2, 3] + y << ["a", "b", "c"] +} + """) + + then: + result.testEvents().started().list().testDescriptor.displayName == ["foo", + "x: 1, y: a, #0", + "x: 2, y: b, #1", + "x: 3, y: c, #2" + ] + } + + def "dataVariablesWithIndex can be used in @Unroll"() { + when: + def result = runner.runSpecBody(""" +@Unroll("#dataVariablesWithIndex") +def foo() { + expect: true + + where: + x << [1, 2, 3] + y << ["a", "b", "c"] +} + """) + + then: + result.testEvents().started().list().testDescriptor.displayName == ["foo", + "x: 1, y: a, #0", + "x: 2, y: b, #1", + "x: 3, y: c, #2" + ] + } + @Issue("https://github.com/spockframework/spock/issues/187") def "variables in naming pattern whose value is null are replaced correctly"() { when: