Skip to content

Commit

Permalink
Support CSV headers in display names in parameterized tests
Browse files Browse the repository at this point in the history
Given the following parameterized test that sets
useHeadersInDisplayName to true and uses {arguments} instead of
{argumentsWithNames} for its display name pattern...

@ParameterizedTest(name = "[{index}] {arguments}")
@CsvSource(useHeadersInDisplayName = true, textBlock = """
	FRUIT,  RANK
	apple,  1
	banana, 2
	cherry, 3
	""")
void test(String fruit, int rank) {}

The generated display names are:

[1] FRUIT = apple, RANK = 1
[2] FRUIT = banana, RANK = 2
[3] FRUIT = cherry, RANK = 3

See #2759
  • Loading branch information
sbrannen committed Nov 19, 2021
1 parent 69aed70 commit 4ef6e70
Show file tree
Hide file tree
Showing 14 changed files with 345 additions and 84 deletions.
Expand Up @@ -6,6 +6,7 @@
*Scope:*

* Text blocks in `@CsvSource` are treated like CSV files
* CSV headers in display names for `@CsvSource` and `@CsvFileSource`
* Custom quote character support in `@CsvSource` and `@CsvFileSource`

For a complete list of all _closed_ issues and pull requests for this release, consult the
Expand All @@ -29,8 +30,13 @@ No changes.
quoted strings. See the
<<../user-guide/index.adoc#writing-tests-parameterized-tests-sources-CsvSource, User
Guide>> for details and examples.
* CSV headers can now be used in display names in parameterized tests. See
<<../user-guide/index.adoc#writing-tests-parameterized-tests-sources-CsvSource,
`@CsvSource`>> and
<<../user-guide/index.adoc#writing-tests-parameterized-tests-sources-CsvFileSource,
`@CsvFileSource`>> in the User Guide for details and examples.
* The quote character for _quoted strings_ in `@CsvSource` and `@CsvFileSource` is now
configurable via new `quoteCharacter` attributes in each annotation.
configurable via a new `quoteCharacter` attribute in each annotation.


[[release-notes-5.8.2-junit-vintage]]
Expand Down
49 changes: 44 additions & 5 deletions documentation/src/docs/asciidoc/user-guide/writing-tests.adoc
Expand Up @@ -1332,7 +1332,9 @@ include::{testDir}/example/ExternalMethodSourceDemo.java[tags=external_MethodSou

`@CsvSource` allows you to express argument lists as comma-separated values (i.e., CSV
`String` literals). Each string provided via the `value` attribute in `@CsvSource`
represents a CSV record and results in one invocation of the parameterized test.
represents a CSV record and results in one invocation of the parameterized test. The first
record may optionally be used to supply CSV headers (see the Javadoc for the
`useHeadersInDisplayName` attribute for details and an example).

[source,java,indent=0]
----
Expand Down Expand Up @@ -1375,12 +1377,16 @@ by default. This behavior can be changed by setting the
If the programming language you are using supports _text blocks_ -- for example, Java SE
15 or higher -- you can alternatively use the `textBlock` attribute of `@CsvSource`. Each
record within a text block represents a CSV record and results in one invocation of the
parameterized test. Using a text block, the previous example can be implemented as follows.
parameterized test. The first record may optionally be used to supply CSV headers by
setting the `useHeadersInDisplayName` attribute to `true` as in the example below.

Using a text block, the previous example can be implemented as follows.

[source,java,indent=0]
----
@ParameterizedTest
@CsvSource(textBlock = """
@ParameterizedTest(name = "[{index}] {arguments}")
@CsvSource(useHeadersInDisplayName = true, textBlock = """
FRUIT, RANK
apple, 1
banana, 2
'lemon, lime', 0xF1
Expand All @@ -1391,6 +1397,15 @@ void testWithCsvSource(String fruit, int rank) {
}
----

The generated display names for the previous example include the CSV header names.

----
[1] FRUIT = apple, RANK = 1
[2] FRUIT = banana, RANK = 2
[3] FRUIT = lemon, lime, RANK = 0xF1
[4] FRUIT = strawberry, RANK = 700_000
----

In contrast to CSV records supplied via the `value` attribute, a text block can contain
comments. Any line beginning with a `+++#+++` symbol will be treated as a comment and
ignored. Note, however, that the `+++#+++` symbol must be the first character on the line
Expand Down Expand Up @@ -1435,7 +1450,11 @@ your text block.

`@CsvFileSource` lets you use comma-separated value (CSV) files from the classpath or the
local file system. Each record from a CSV file results in one invocation of the
parameterized test.
parameterized test. The first record may optionally be used to supply CSV headers. You can
instruct JUnit to ignore the headers via the `numLinesToSkip` attribute. If you would like
for the headers to be used in the display names, you can set the `useHeadersInDisplayName`
attribute to `true`. The examples below demonstrate the use of `numLinesToSkip` and
`useHeadersInDisplayName`.

The default delimiter is a comma (`,`), but you can use another character by setting the
`delimiter` attribute. Alternatively, the `delimiterString` attribute allows you to use a
Expand All @@ -1457,6 +1476,26 @@ include::{testDir}/example/ParameterizedTestDemo.java[tags=CsvFileSource_example
include::{testResourcesDir}/two-column.csv[]
----

The following listing shows the generated display names for the first two parameterized
test methods above.

----
[1] country=Sweden, reference=1
[2] country=Poland, reference=2
[3] country=United States of America, reference=3
[4] country=France, reference=700_000
----

The following listing shows the generated display names for the last parameterized test
method above that uses CSV header names.

----
[1] COUNTRY = Sweden, REFERENCE = 1
[2] COUNTRY = Poland, REFERENCE = 2
[3] COUNTRY = United States of America, REFERENCE = 3
[4] COUNTRY = France, REFERENCE = 700_000
----

In contrast to the default syntax used in `@CsvSource`, `@CsvFileSource` uses a double
quote (`+++"+++`) as the quote character by default, but this can be changed via the
`quoteCharacter` attribute. See the `"United States of America"` value in the example
Expand Down
Expand Up @@ -236,6 +236,13 @@ void testWithCsvFileSourceFromFile(String country, int reference) {
assertNotNull(country);
assertNotEquals(0, reference);
}

@ParameterizedTest(name = "[{index}] {arguments}")
@CsvFileSource(resources = "/two-column.csv", useHeadersInDisplayName = true)
void testWithCsvFileSourceAndHeaders(String country, int reference) {
assertNotNull(country);
assertNotEquals(0, reference);
}
// end::CsvFileSource_example[]

// tag::ArgumentsSource_example[]
Expand Down
2 changes: 1 addition & 1 deletion documentation/src/test/resources/two-column.csv
@@ -1,4 +1,4 @@
Country, Reference
COUNTRY, REFERENCE
Sweden, 1
Poland, 2
"United States of America", 3
Expand Down
Expand Up @@ -15,6 +15,7 @@

import java.io.StringReader;
import java.lang.annotation.Annotation;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Set;
Expand All @@ -23,6 +24,7 @@

import com.univocity.parsers.csv.CsvParser;

import org.junit.jupiter.api.Named;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.params.support.AnnotationConsumer;
import org.junit.platform.commons.PreconditionViolationException;
Expand Down Expand Up @@ -53,56 +55,96 @@ public Stream<? extends Arguments> provideArguments(ExtensionContext context) {
Preconditions.condition(this.annotation.value().length > 0 ^ textBlockDeclared,
() -> "@CsvSource must be declared with either `value` or `textBlock` but not both");

if (textBlockDeclared) {
return parseTextBlock(this.annotation.textBlock()).stream().map(Arguments::of);
}

AtomicInteger index = new AtomicInteger(0);
// @formatter:off
return Arrays.stream(this.annotation.value())
.map(line -> parseLine(line, index.incrementAndGet()))
.map(Arguments::of);
// @formatter:on
return textBlockDeclared ? parseTextBlock() : parseValueArray();
}

private List<String[]> parseTextBlock(String textBlock) {
private Stream<Arguments> parseTextBlock() {
String textBlock = this.annotation.textBlock();
boolean useHeadersInDisplayName = this.annotation.useHeadersInDisplayName();
List<Arguments> argumentsList = new ArrayList<>();

try {
AtomicInteger index = new AtomicInteger(0);
List<String[]> csvRecords = this.csvParser.parseAll(new StringReader(textBlock));
String[] headers = useHeadersInDisplayName ? getHeaders(this.csvParser) : null;

AtomicInteger index = new AtomicInteger(0);
for (String[] csvRecord : csvRecords) {
index.incrementAndGet();
Preconditions.notNull(csvRecord,
() -> "Line at index " + index.get() + " contains invalid CSV: \"\"\"\n" + textBlock + "\n\"\"\"");
processNullValues(csvRecord, this.nullValues);
() -> "Record at index " + index + " contains invalid CSV: \"\"\"\n" + textBlock + "\n\"\"\"");
argumentsList.add(processCsvRecord(csvRecord, this.nullValues, useHeadersInDisplayName, headers));
}
return csvRecords;
}
catch (Throwable throwable) {
throw handleCsvException(throwable, this.annotation);
}

return argumentsList.stream();
}

private String[] parseLine(String line, int index) {
private Stream<Arguments> parseValueArray() {
boolean useHeadersInDisplayName = this.annotation.useHeadersInDisplayName();
List<Arguments> argumentsList = new ArrayList<>();

try {
String[] csvRecord = this.csvParser.parseLine(line + LINE_SEPARATOR);
Preconditions.notNull(csvRecord,
() -> "Line at index " + index + " contains invalid CSV: \"" + line + "\"");
processNullValues(csvRecord, this.nullValues);
return csvRecord;
String[] headers = null;
AtomicInteger index = new AtomicInteger(0);
for (String input : this.annotation.value()) {
index.incrementAndGet();
String[] csvRecord = this.csvParser.parseLine(input + LINE_SEPARATOR);
// Lazily retrieve headers if necessary.
if (useHeadersInDisplayName && headers == null) {
headers = getHeaders(this.csvParser);
}
Preconditions.notNull(csvRecord,
() -> "Record at index " + index + " contains invalid CSV: \"" + input + "\"");
argumentsList.add(processCsvRecord(csvRecord, this.nullValues, useHeadersInDisplayName, headers));
}
}
catch (Throwable throwable) {
throw handleCsvException(throwable, this.annotation);
}

return argumentsList.stream();
}

static void processNullValues(String[] csvRecord, Set<String> nullValues) {
if (!nullValues.isEmpty()) {
for (int i = 0; i < csvRecord.length; i++) {
if (nullValues.contains(csvRecord[i])) {
csvRecord[i] = null;
}
// Cannot get parsed headers until after parsing has started.
static String[] getHeaders(CsvParser csvParser) {
return Arrays.stream(csvParser.getContext().parsedHeaders())//
.map(String::trim)//
.toArray(String[]::new);
}

/**
* Processes custom null values, supports wrapping of column values in
* {@link Named} if necessary (for CSV header support), and returns the
* CSV record wrapped in an {@link Arguments} instance.
*/
static Arguments processCsvRecord(Object[] csvRecord, Set<String> nullValues, boolean useHeadersInDisplayName,
String[] headers) {

// Nothing to process?
if (nullValues.isEmpty() && !useHeadersInDisplayName) {
return Arguments.of(csvRecord);
}

Preconditions.condition(!useHeadersInDisplayName || (csvRecord.length <= headers.length),
() -> String.format(
"The number of columns (%d) exceeds the number of supplied headers (%d) in CSV record: %s",
csvRecord.length, headers.length, Arrays.toString(csvRecord)));

Object[] arguments = new Object[csvRecord.length];
for (int i = 0; i < csvRecord.length; i++) {
Object column = csvRecord[i];
if (nullValues.contains(column)) {
column = null;
}
if (useHeadersInDisplayName) {
column = Named.of(headers[i] + " = " + column, column);
}
arguments[i] = column;
}
return Arguments.of(arguments);
}

/**
Expand Down
Expand Up @@ -13,9 +13,9 @@
import static java.util.Spliterators.spliteratorUnknownSize;
import static java.util.stream.Collectors.toList;
import static java.util.stream.StreamSupport.stream;
import static org.junit.jupiter.params.provider.Arguments.arguments;
import static org.junit.jupiter.params.provider.CsvArgumentsProvider.getHeaders;
import static org.junit.jupiter.params.provider.CsvArgumentsProvider.handleCsvException;
import static org.junit.jupiter.params.provider.CsvArgumentsProvider.processNullValues;
import static org.junit.jupiter.params.provider.CsvArgumentsProvider.processCsvRecord;
import static org.junit.jupiter.params.provider.CsvParserFactory.createParserFor;
import static org.junit.platform.commons.util.CollectionUtils.toSet;

Expand Down Expand Up @@ -119,41 +119,49 @@ private static class CsvParserIterator implements Iterator<Arguments> {

private final CsvParser csvParser;
private final CsvFileSource annotation;
private final boolean useHeadersInDisplayName;
private final Set<String> nullValues;
private Object[] nextCsvRecord;
private Arguments nextArguments;
private String[] headers;

CsvParserIterator(CsvParser csvParser, CsvFileSource annotation) {
this.csvParser = csvParser;
this.annotation = annotation;
this.useHeadersInDisplayName = annotation.useHeadersInDisplayName();
this.nullValues = toSet(annotation.nullValues());
advance();
}

@Override
public boolean hasNext() {
return this.nextCsvRecord != null;
return this.nextArguments != null;
}

@Override
public Arguments next() {
Arguments result = arguments(this.nextCsvRecord);
Arguments result = this.nextArguments;
advance();
return result;
}

private void advance() {
String[] csvRecord = null;
try {
csvRecord = this.csvParser.parseNext();
String[] csvRecord = this.csvParser.parseNext();
if (csvRecord != null) {
processNullValues(csvRecord, this.nullValues);
// Lazily retrieve headers if necessary.
if (this.useHeadersInDisplayName && this.headers == null) {
this.headers = getHeaders(this.csvParser);
}
this.nextArguments = processCsvRecord(csvRecord, this.nullValues, this.useHeadersInDisplayName,
this.headers);
}
else {
this.nextArguments = null;
}
}
catch (Throwable throwable) {
handleCsvException(throwable, this.annotation);
}

this.nextCsvRecord = csvRecord;
}

}
Expand Down
Expand Up @@ -27,7 +27,9 @@
* or {@link #files}.
*
* <p>The CSV records parsed from these resources and files will be provided as
* arguments to the annotated {@code @ParameterizedTest} method.
* arguments to the annotated {@code @ParameterizedTest} method. Note that the
* first record may optionally be used to supply CSV headers (see
* {@link #useHeadersInDisplayName}).
*
* <p>Any line beginning with a {@code #} symbol will be interpreted as a comment
* and will be ignored.
Expand Down Expand Up @@ -95,6 +97,34 @@
*/
String lineSeparator() default "\n";

/**
* Configures whether the first CSV record should be treated as header names
* for columns.
*
* <p>When set to {@code true}, the header names will be used in the
* generated display name for each {@code @ParameterizedTest} method
* invocation. When using this feature, you must ensure that the display name
* pattern for {@code @ParameterizedTest} includes
* {@value org.junit.jupiter.params.ParameterizedTest#ARGUMENTS_PLACEHOLDER} instead of
* {@value org.junit.jupiter.params.ParameterizedTest#ARGUMENTS_WITH_NAMES_PLACEHOLDER}
* as demonstrated in the example below.
*
* <p>Defaults to {@code false}.
*
*
* <h4>Example</h4>
* <pre class="code">
* {@literal @}ParameterizedTest(name = "[{index}] {arguments}")
* {@literal @}CsvFileSource(resources = "fruits.csv", useHeadersInDisplayName = true)
* void test(String fruit, int rank) {
* // ...
* }</pre>
*
* @since 5.8.2
*/
@API(status = EXPERIMENTAL, since = "5.8.2")
boolean useHeadersInDisplayName() default false;

/**
* The quote character to use for <em>quoted strings</em>.
*
Expand Down

0 comments on commit 4ef6e70

Please sign in to comment.