Introduce named argument set support for @ParameterizedTest

[aws-gibbskt]

Summary

I find myself writing the following.

@ParameterizedTest(name = "{0}")

#2301 introduced a way to name an argument, but it feels very unnatural to be used for multiple argument test cases.

The recommendation in that issue is to do the below with the name as defined above.

return Stream.of(
    arguments(named("Test Case Name", firstArg), secondArg, thirdArg),
    arguments(named...)
);

I would much rather have something like:

return Stream.of(
    named("Test Case Name", arguments(firstArg, secondArg, thirdArg)),
    named("Other Case Name", arguments...)
);

Related Issues

• A way to provide the arguments "name" without actually having it as an argument. #2301

Proposal

Provide the ability to name an individual parameterized test invocation by wrapping the Arguments in Named<Arguments> rather than wrapping a specific argument in Named<T>.

[marcphilipp]

Team decision: Spike the following:

return Stream.of(
    namedArguments("Test Case Name", firstArg, secondArg, thirdArg),
    namedArguments("Other Case Name", arguments...)
);

where Arguments.namedArguments would return an instance of NamedArguments which implements Named<Arguments> and Arguments and the implementation of Named.getPayload would return this.

[scordio]

@marcphilipp @sbrannen if I understood it correctly, the usage of namedArguments won't require setting any additional attribute for @ParameterizedTest, right?

[sbrannen]

@scordio, I've already spiked this locally and will push to a feature branch soon.

if I understood it correctly, the usage of namedArguments won't require setting any additional attribute for @ParameterizedTest, right?

That's correct: the goal is that it will work seamlessly, and it does in my current spike.

However, having said that, it does appear that we'll need to introduce a new display name holder and include that in the default display name pattern. Otherwise, I don't yet see how to reliably include this additional information.

Though maybe we can work around that by introducing an internal placeholder for this purpose.

Time will tell...

It's a work in progress. 😉

[scordio]

Very cool, looking forward to it!

[sbrannen]

Current work on this issue can be viewed in the following feature branch.

main...sbrannen:junit5:issues/3818-NamedArguments

I implemented two approaches that we can consider and use for further brainstorming.

The first commit prepends the value returned NamedArguments#getName plus :: to the display name that would otherwise be generated.

This results in display names for two invocations like this (where "VIP" is from the use of named() for a single argument; the first invocation uses namedArguments(); and the second invocation users arguments()):

Important Files :: [1] file1=VIP, file2=path2
[2] file1=path3, file2=path4

Though, in the description of this issue @aws-gibbskt suggested that the "group name" replace what would otherwise be the display name.

The second commit takes that in account and results in display names for two invocations like this (where both invocations use namedArguments()):

Important Files
Other Files

Feedback and further brainstorming is welcome!

[sbrannen]

In summary, the aforementioned feature branch currently supports a test class like this:

// imports ...
import static org.junit.jupiter.params.provider.Arguments.namedArguments;

class NamedArgumentsSetDemo {

	@ParameterizedTest
	@FieldSource
	void namedArgumentsSet(File file1, File file2, TestInfo testInfo) {
		System.out.println(testInfo.getDisplayName());
	}

	static List<Arguments> namedArgumentsSet = Arrays.asList(
		namedArguments("Important Files", new File("path1"), new File("path2")),
		namedArguments("Other Files", new File("path3"), new File("path4"))
	);

}

Which outputs the following for the display names:

Important Files
Other Files

[marcphilipp]

I think including the index (by default) would be helpful:

• [1] Important Files
• [2] Other Files

[sbrannen]

I think including the index (by default) would be helpful:

I agree.

I'll make that change.

Though we need to decide if we want this to actually be configurable or hard-coded to do exactly that and only that.

Thoughts?

---

One compelling reason not to make it configurable is that the use of Named for a single argument completely overrides what would otherwise be included in the display name for that single argument.

Along that line of thinking, I imagined that NamedArguments would also completely override what would otherwise be generated as the display name for the entire invocation.

[sbrannen]

The third commit always includes the invocation index and results in display names for two invocations like this (where both invocations use namedArguments()):

[1] Important Files
[2] Other Files

[aws-gibbskt]

This looks like a great addition to junit and addresses the intent of my request.

[1] Important Files
[2] Other Files

or

Important Files
Other Files

as the default behavior are both fine.

[sbrannen]

With the latest changes on my feature branch, the following parameterized tests...

@BeforeEach
void printDisplayName(TestInfo testInfo) {
	System.out.println(testInfo.getDisplayName());
}

@ParameterizedTest
@FieldSource("namedArgumentsList")
void defaultNamedArgumentsDisplayName(File file1, File file2) {
}

@ParameterizedTest(name = "{namedArguments} :: {argumentsWithNames}")
@FieldSource("namedArgumentsList")
void customNamedArgumentsDisplayName(File file1, File file2) {
}

static List<Arguments> namedArgumentsList = Arrays.asList(
	namedArguments("Important Files", new File("path1"), new File("path2")),
	namedArguments("Other Files", new File("path3"), new File("path4"))
);

... now output this:

[1] Important Files
[2] Other Files

Important Files :: file1=path1, file2=path2
Other Files :: file1=path3, file2=path4

The default pattern for NamedArguments is:

String DEFAULT_NAMED_ARGUMENTS_DISPLAY_NAME = "[" + INDEX_PLACEHOLDER + "] " + NAMED_ARGUMENTS_PLACEHOLDER;

And that new placeholder is:

String NAMED_ARGUMENTS_PLACEHOLDER = "{namedArguments}";

[sbrannen]

The reason I posted the above is to discuss naming, because... as we all know...

Naming is hard! ™️ 😉

The new type is called NamedArguments and the new method is Arguments.namedArguments().

I think those names are probably fine as-is, but I'm not happy with the names/values of the constants/placeholders.

When you see @ParameterizedTest(name = "{namedArguments} :: {argumentsWithNames}"), that's very confusing.

What's the difference between "named arguments" and "arguments with names"?

If we go with that "technical" naming strategy, I fear it will be too confusing for users (even with good documentation).

So I'd like to brainstorm about better names for those constants and the placeholder.

---

Technically, we're talking about a "set of arguments with a name for that entire set."

We are not actually talking about "named arguments".

So, even if we stick with NamedArguments / Arguments.namedArguments(), I don't think we should use that terminology in the constants and placeholder.

We could go very technical with something like "NamedArguments name" (because of NamedArguments#getName()), which could lead to the following.

String NAMED_ARGUMENTS_NAME_PLACEHOLDER = "{namedArgumentsName}";

String DEFAULT_NAMED_ARGUMENTS_DISPLAY_NAME = "[" + INDEX_PLACEHOLDER + "] " + NAMED_ARGUMENTS_NAME_PLACEHOLDER;

But I think that's extremely ugly, and I'm only including it for the sake of argument (pun intended).

Since we're talking about a "set of arguments" or an "argument set", perhaps something like the following would be better.

String ARGUMENT_SET_NAME_PLACEHOLDER = "{argumentSetName}";

String DEFAULT_ARGUMENT_SET_DISPLAY_NAME = "[" + INDEX_PLACEHOLDER + "] " + ARGUMENT_SET_NAME_PLACEHOLDER;

Or perhaps we should refer to this thing as an "argument group".

Any suggestions or brainstorming ideas are welcome! 👍

---

p.s. Another idea I had was to refer to the name of this new thing as the "invocation name" since you're effectively assigning a name to the current parameterized test invocation when you use namedArguments(). So the placeholder could be {invocationName} or similar if we go that route, but I'm not sure what we'd call the "default XXX display name" pattern in that case, since it would not be a default pattern for all invocations but rather only the default for invocations that make use of namedArguments(). 🤷

[marcphilipp]

I'm in favor of introducing a term like "argument set", "invocation name", or "scenario" to refer to. Depending on which we choose, this might also mean renaming namedArguments to sth. else.

Why do we need DEFAULT_NAMED_ARGUMENTS_DISPLAY_NAME/DEFAULT_ARGUMENT_SET_DISPLAY_NAME? Wouldn't DEFAULT_DISPLAY_NAME be changed to [{index}] {argumentSetNameOrArgumentsWithNames}?

[sbrannen]

Thanks for the feedback, @marcphilipp.

I'm in favor of introducing a term like "argument set", "invocation name", or "scenario" to refer to.

I think we should shy away from "scenario" since that might get confused with "scenario tests" which we don't support, and I basically convinced myself above that "invocation name" is probably too generic.

So, I'm currently leaning toward "argument set" for the terminology and {argumentSetName} for the placeholder.

Depending on which we choose, this might also mean renaming namedArguments to sth. else.

Indeed, that would help to avoid confusion.

Perhaps the static factory method should be named argumentSet(...) instead of namedArguments(...), and the new type would then be ArgumentSet instead of NamedArguments.

[sbrannen]

Why do we need DEFAULT_NAMED_ARGUMENTS_DISPLAY_NAME/DEFAULT_ARGUMENT_SET_DISPLAY_NAME Wouldn't DEFAULT_DISPLAY_NAME be changed to [{index}] {argumentSetNameOrArgumentsWithNames}?

I hadn't considered that option.

My goal was to make it configurable and clear within the available options/constants in ParameterizedTest (and to use predefined patterns instead of manipulating patterns programmatically), but I suppose we could introduce a new {argumentSetNameOrArgumentsWithNames} placeholder as you've proposed to achieve the same goal.

I'll experiment with that.

[sbrannen]

I have finalized my proposal and submitted PR #3825, which is ready for review. 😎

[sbrannen]

argumentSet() support has been introduced in f95d4e8 and is available for testing in 5.11.0 snapshots.

Enjoy! 😎