Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to omit feature name from iterations #1386

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 84 additions & 10 deletions docs/data_driven_testing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -402,7 +402,7 @@ such a method.

== Unrolled Iteration Names

By default the names of unrolled iterations are the name of the feature, plus the data variables and the iteration
By default, the names of unrolled iterations are the name of the feature, plus the data variables and the iteration
index. This will always produce unique names and should enable you to identify easily the failing data variable
combination.

Expand Down Expand Up @@ -468,61 +468,135 @@ include::{sourcedir}/datadriven/DataSpec.groovy[tag=unrolled-3a]
include::{sourcedir}/datadriven/DataSpec.groovy[tag=unrolled-3b]
----

Additionally to the data variables the tokens `#featureName` and `#iterationIndex` are supported.
Additionally, to the data variables the tokens `#featureName` and `#iterationIndex` are supported.
The former does not make much sense inside an actual feature name, but there are two other places
where an unroll pattern can be defined, where it is more useful.
where an unroll-pattern can be defined, where it is more useful.

[source,groovy,indent=0]
----
def "#person is #person.age years old [iterationIndex: #iterationIndex]"() {
def "#person is #person.age years old [#iterationIndex]"() {
----

Alternatively to specifying the unroll pattern as method name, it can be given as parameter
will be reported as

----
└─ Spock ✔
└─ PersonSpec ✔
└─ #person.name is #person.age years old [#iterationIndex] ✔
├─ Fred is 38 years old [0] ✔
├─ Wilma is 36 years old [1] ✔
└─ Pebbles is 5 years old [2] ✔
----

Alternatively, to specifying the unroll-pattern as method name, it can be given as parameter
to the `@Unroll` annotation which takes precedence over the method name:

[source,groovy,indent=0]
----
@Unroll("#featureName[#iterationIndex] (#person.name is #person.age years old)")
def "person age should be calculated properly"() {
// ...
----

will be reported as

----
└─ Spock ✔
└─ PersonSpec ✔
└─ person age should be calculated properly ✔
├─ person age should be calculated properly[0] (Fred is 38 years old) ✔
├─ person age should be calculated properly[1] (Wilma is 36 years old) ✔
└─ person age should be calculated properly[2] (Pebbles is 5 years old) ✔
----

The advantage is, that you can have a descriptive method name for the whole feature, while having a separate template for each iteration.
Furthermore, the feature method name is not filled with placeholders and thus better readable.

If neither a parameter to the annotation is given, nor the method name contains a `#`,
the <<extensions.adoc#spock-configuration-file,configuration file>> setting `defaultPattern`
in the `unroll` section is inspected. If it is set to a non-`null`
string, this value is used as unroll pattern. This could for example be set to
string, this value is used as unroll-pattern. This could for example be set to

- `#featureName` to have all iterations reported with the same name, or
- `#featureName[#iterationIndex]` to have a simply indexed iteration name, or
- `#iterationName` if you make sure that in each data-driven feature you also set
a data variable called `iterationName` that is then used for reporting

.Set Default Unroll Pattern
=== Special Tokens

This is the complete list of special tokens:

- `#featureName` is the name of the feature (mostly useful for the `defaultPattern` setting)
- `#iterationIndex` is the current iteration index
- `#dataVariables` lists all data variables for this iteration, e.g. `x: 1, y: 2, z: 3`
- `#dataVariablesWithIndex` the same as `#dataVariables` but with an index at the end, e.g. `x: 1, y: 2, z: 3, #0`

=== Configuration

.Set Default Unroll-Pattern
[source,groovy]
----
unroll {
defaultPattern '#featureName[#iterationIndex]'
}
----

If none of the three described ways is used to set a custom unroll pattern, by default
If none of the three described ways is used to set a custom unroll-pattern, by default
the feature name is used, suffixed with all data variable names and their values and
finally the iteration index, so the result will be for example
`my feature [x: 1, y: 2, z: 3, #0]`.

If there is an error in an unroll expression, for example typo in variable name, exception during
evaluation of a property or method in the expression and so on, the test will fail. This is not
true for the automatic fall back rendering of the data variables if there is no unroll pattern
true for the automatic fall back rendering of the data variables if there is no unroll-pattern
set in any way, this will never fail the test, no matter what happens.

The failing of test with errors in the unroll expression can be disabled by setting the
<<extensions.adoc#spock-configuration-file,configuration file>> setting `validateExpressions`
in the `unroll` section to `false`. If this is done and an error happens, the erroneous expression
`#foo.bar` will be substituted by `#Error:foo.bar`.

.Disable Unroll Pattern Expression Asserting
.Disable Unroll-pattern Expression Asserting
[source,groovy]
----
unroll {
validateExpressions false
}
----

Some reporting frameworks, or IDEs support proper tree based reporting.
For these cases it might be desirable to omit the feature name from the iteration reporting.

.Disable repetition of feature name in iterations
[source,groovy]
----
unroll {
includeFeatureNameForIterations false
}
----

With `includeFeatureNameForIterations true`
----
└─ Spock ✔
└─ ASpec ✔
└─ really long and informative test name that doesn't have to be repeated ✔
├─ really long and informative test name that doesn't have to be repeated [x: 1, y: a, #0] ✔
├─ really long and informative test name that doesn't have to be repeated [x: 2, y: b, #1] ✔
└─ really long and informative test name that doesn't have to be repeated [x: 3, y: c, #2] ✔
----

.With `includeFeatureNameForIterations false`
----
└─ Spock ✔
└─ ASpec ✔
└─ really long and informative test name that doesn't have to be repeated ✔
├─ x: 1, y: a, #0 ✔
├─ x: 2, y: b, #1 ✔
└─ x: 3, y: c, #2 ✔
----

NOTE: The same can be achieved for individual features by using `@Unroll('#dataVariablesWithIndex')`.
Original file line number Diff line number Diff line change
Expand Up @@ -14,29 +14,41 @@

package org.spockframework.runtime;

import org.spockframework.runtime.model.FeatureInfo;
import org.spockframework.runtime.model.IterationInfo;
import org.spockframework.runtime.model.NameProvider;

import java.util.Map;
import java.util.StringJoiner;

import static java.lang.String.format;
import static org.spockframework.util.RenderUtil.toStringOrDump;

import org.spockframework.runtime.model.*;

import java.util.*;

public class DataVariablesIterationNameProvider implements NameProvider<IterationInfo> {
private final boolean includeFeatureNameForIterations;
private final boolean includeIterationIndex;

public DataVariablesIterationNameProvider() {
this(true, true);
}

public DataVariablesIterationNameProvider(boolean includeFeatureNameForIterations) {
this(includeFeatureNameForIterations, true);
}

public DataVariablesIterationNameProvider(boolean includeFeatureNameForIterations, boolean includeIterationIndex) {
leonard84 marked this conversation as resolved.
Show resolved Hide resolved
this.includeFeatureNameForIterations = includeFeatureNameForIterations;
this.includeIterationIndex = includeIterationIndex;
}

@Override
public String getName(IterationInfo iteration) {
FeatureInfo feature = iteration.getFeature();
if (!feature.isReportIterations()) {
return feature.getName();
}

StringJoiner nameJoiner = new StringJoiner(", ", "[", "]");
StringJoiner nameJoiner = new StringJoiner(", ");
Map<String, Object> dataVariables = iteration.getDataVariables();
if (dataVariables != null) {
dataVariables.forEach((name, value) ->
{
dataVariables.forEach((name, value) -> {
String valueString;
try {
valueString = toStringOrDump(value);
Expand All @@ -46,7 +58,9 @@ public String getName(IterationInfo iteration) {
nameJoiner.add(format("%s: %s", name, valueString));
});
}
nameJoiner.add(format("#%d", iteration.getIterationIndex()));
return format("%s %s", feature.getName(), nameJoiner.toString());
if (includeIterationIndex) {
nameJoiner.add(format("#%d", iteration.getIterationIndex()));
}
return includeFeatureNameForIterations ? format("%s [%s]", feature.getName(), nameJoiner) : nameJoiner.toString();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
* unrollByDefault false // default true
* defaultPattern '#featureName[#iterationIndex]' // default null
* validateExpressions false // default true
* includeFeatureNameForIterations false // default true
* }
* </pre>
*
Expand All @@ -37,4 +38,5 @@ public class UnrollConfiguration {
public boolean unrollByDefault = true;
public String defaultPattern = null;
public boolean validateExpressions = true;
public boolean includeFeatureNameForIterations = true;
}
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,6 @@ private NameProvider<IterationInfo> chooseNameProvider(String specUnrollPattern,
if (unrollConfiguration.defaultPattern != null) {
return new UnrollIterationNameProvider(feature, unrollConfiguration.defaultPattern, unrollConfiguration.validateExpressions);
}
return new DataVariablesIterationNameProvider();
return new DataVariablesIterationNameProvider(unrollConfiguration.includeFeatureNameForIterations);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@
*/
public class UnrollIterationNameProvider implements NameProvider<IterationInfo> {
private static final Pattern EXPRESSION_PATTERN = Pattern.compile("#([a-zA-Z_$]([\\w$.]|\\(\\))*)");
private static final DataVariablesIterationNameProvider DATA_VARIABLES =
new DataVariablesIterationNameProvider(false, false);
private static final DataVariablesIterationNameProvider DATA_VARIABLES_WITH_INDEX =
new DataVariablesIterationNameProvider(false, true);

private final boolean validateExpressions;
private final FeatureInfo feature;
Expand All @@ -43,24 +47,24 @@ public UnrollIterationNameProvider(FeatureInfo feature, String namePattern, bool
// always returns a name
@Override
public String getName(IterationInfo iterationInfo) {
return nameFor(iterationInfo.getIterationIndex(), iterationInfo.getDataVariables());
return nameFor(iterationInfo.getDataVariables(), iterationInfo);
}

private String nameFor(int iterationIndex, Map<String, Object> dataVariables) {
private String nameFor(Map<String, Object> dataVariables, IterationInfo iterationInfo) {
StringBuffer result = new StringBuffer();
expressionMatcher.reset();

while (expressionMatcher.find()) {
String expr = expressionMatcher.group(1);
String value = evaluateExpression(expr, iterationIndex, dataVariables);
String value = evaluateExpression(expr, dataVariables, iterationInfo);
expressionMatcher.appendReplacement(result, Matcher.quoteReplacement(value));
}

expressionMatcher.appendTail(result);
return result.toString();
}

private String evaluateExpression(String expr, int iterationIndex, Map<String, Object> dataVariables) {
private String evaluateExpression(String expr, Map<String, Object> dataVariables, IterationInfo iterationInfo) {
String[] exprParts = expr.split("\\.");
String firstPart = exprParts[0];
Object result;
Expand All @@ -71,7 +75,15 @@ private String evaluateExpression(String expr, int iterationIndex, Map<String, O
break;

case "iterationIndex":
result = String.valueOf(iterationIndex);
result = String.valueOf(iterationInfo.getIterationIndex());
break;

case "dataVariables":
result = DATA_VARIABLES.getName(iterationInfo);
break;

case "dataVariablesWithIndex":
result = DATA_VARIABLES_WITH_INDEX.getName(iterationInfo);
break;

default:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ class DataVariablesIterationNameProviderSpec extends Specification {
name = 'the feature'
reportIterations = true
}

IterationInfo iteration = Stub {
getFeature() >> feature
getIterationIndex() >> 99
Expand Down Expand Up @@ -77,4 +78,40 @@ class DataVariablesIterationNameProviderSpec extends Specification {
expect:
testee.getName(iteration) == "$feature.name [x: #Error:RuntimeException during rendering, y: 2, z: 3, #$iteration.iterationIndex]"
}

def 'returns data variables and iteration index when reporting iterations and includeFeatureNameForIterations=false'() {
given:
testee = new DataVariablesIterationNameProvider(false)
iteration.getDataVariables() >> [x: 1, y: 2, z: 3]

expect:
testee.getName(iteration) == "x: 1, y: 2, z: 3, #$iteration.iterationIndex"
}

def 'returns data variables only when reporting iterations and includeFeatureNameForIterations=false, includeIterationIndex=false'() {
given:
testee = new DataVariablesIterationNameProvider(false, false)
iteration.getDataVariables() >> [x: 1, y: 2, z: 3]

expect:
testee.getName(iteration) == "x: 1, y: 2, z: 3"
}

def 'renders data variables in Groovy style and includeFeatureNameForIterations=false'() {
given:
testee = new DataVariablesIterationNameProvider(false)
iteration.getDataVariables() >> [x: [1], y: [a: 2], z: [3] as int[]]

expect:
testee.getName(iteration) == "x: [1], y: [a:2], z: [3], #$iteration.iterationIndex"
}

def 'returns iteration index when reporting iterations but data variables are null and includeFeatureNameForIterations=false'() {
given:
testee = new DataVariablesIterationNameProvider(false)
iteration.getDataVariables() >> null

expect:
testee.getName(iteration) == "#$iteration.iterationIndex"
}
}
Loading