-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test methods cannot combine RepeatedTest and ParameterizedTest annotations #1224
Comments
Well, that's actually the expected behavior. You cannot have two In addition, it was never foreseen that one would nest So I'm a bit torn as to whether or not we should (or could) consider making a change like this. @junit-team/junit-lambda, thoughts? |
Combining different types of tests (dyanmic, repeated, parameterized) should be expected behavior, so 👍 Here's example: fun ClosedRange<Int>.random() =
ThreadLocalRandom.current().nextInt((endInclusive + 1) - start) + start Ideally: @ParameterizedTest
@CsvSource(value = ["0, 10", "15, 25", "-3, 4"])
@RepeatedTest(100)
@Test fun picksNumberInClosedRange(min: Int, max: Int) {
val value = (min..max).random()
assertThat(value, greaterThanOrEqualTo(min))
assertThat(value, lessThanOrEqualTo(max))
} an Ugly workaround: @RepeatedTest(100)
@TestFactory fun picksNumberInClosedRangeParametersInline() =
arrayOf(
arrayOf(0, 10),
arrayOf(15, 25),
arrayOf(-3, 4)
)
.map { (min, max) -> dynamicTest("Between $min and $max") {
val value = (min..max).random()
assertThat(value, greaterThanOrEqualTo(min))
assertThat(value, lessThanOrEqualTo(max))
} } which doesn't work because the @TestFactory fun picksNumberInClosedRangeJustDynamic() =
arrayOf(
arrayOf(0, 10),
arrayOf(15, 25),
arrayOf(-3, 4)
)
.map { (min, max) -> dynamicTest("Between $min and $max") {
val value = (min..max).random()
assertThat(value, greaterThanOrEqualTo(min))
assertThat(value, lessThanOrEqualTo(max))
} }
.repeat(100)
}
fun Iterable<DynamicTest>.repeat(count: Int) =
(1..count).asSequence().flatMap { this.asSequence() }.asIterable() |
I'm writing code that could make use of this functionality right now |
This will be addressed by #871 which would allow to use |
There is a Stack Overflow issue How to repeat JUnit5 tests on class level? which would be solved by fixing this problem... |
I'm not sure that #871 quite captures the same request as the one being requested here. The main use I want to use this capability for is combining different extensions together. I will give an example: I have a When writing my tests I often want to verify that a bunch of different 'bad' cases result in the correct exceptions so I want to use the @BeforeEach
public void setupService(Datasource datasource) {
// service init code here
}
@DatabseTest
@ParameterizedTest
@ValueSource(strings={"", null})
public void testServiceFailsIfInputIsNotValid(String invalidState) {
// verification code here
} In this example I want the cartesion product of all the test execution contexts. Or I guess in different speek: for each nested context I want to run the whole setup again. I don't think the suite idea described in #871 will help that much since it really only seems to add one level, and that would help with one set. But what if I want to combine three, or more, of these extensions together. It would be nice if there was a way of changing the behavior of the |
Do you need |
I think maybe we could re-write to use that. The actual test harness implements BeforeAll, and AfterAll to manage some JVM global configuration, and AfterEach to clean up the databases created for each test case. It might be possible to create a similar thing with the ArgumentProvicer but the nice thing about the TestTemplateInvocationContextProvider is that each test can, itself, have it's set of Extensions. |
This adds support for combining multiple TestTemplate providers together, in a product style, so that an implementer can use the benefits of multiple extensions together. Issue: junit-team#1224
It turns out this isn't that hard to do in principal so I've thrown together a PR that shows one way of solving the issue. It's open for feedback as much as anything. |
This adds support for combining multiple TestTemplate providers together, in a product style, so that an implementer can use the benefits of multiple extensions together. Issue: junit-team#1224
This adds support for combining multiple TestTemplate providers together, in a product style, so that an implementer can use the benefits of multiple extensions together. Issue: junit-team#1224
This adds support for combining multiple TestTemplate providers together, in a product style, so that an implementer can use the benefits of multiple extensions together. Issue: junit-team#1224
This adds support for combining multiple TestTemplate providers together, in a product style, so that an implementer can use the benefits of multiple extensions together. Issue: junit-team#1224
This adds support for combining multiple TestTemplate providers together, in a product style, so that an implementer can use the benefits of multiple extensions together. Issue: junit-team#1224
Hello, can someone re-evaluate this issue? I see that this issue is closed, because it's superseded by another issue. However, there is a PR addressed to fix this issue, and no one has reviewed the PR as of 2021. Is the current status, "open", because a PR is submitted, or is it "closed" and we need to invalidate the PR? |
I see a branch with the proposal (https://github.com/danielhodder/junit5/tree/feature/1224_repeatable_and_parameterized_tests), but is there an actual PR associated with that branch? |
#2409 is the PR for that branch. There is a description of the change there, but it amounts to changing how combinations of TestTemplates are handled. |
The status of this issue is still closed. PR #2409 will be assessed by the team in order to determine its fate. |
Thanks for the clarification. |
This adds support for combining multiple TestTemplate providers together, in a product style, so that an implementer can use the benefits of multiple extensions together. Issue: junit-team#1224
I think this entire thing has gone the wrong way.
What if
|
Overview
The title pretty much explains it. Test methods annotated with
@RepeatedTest(n)
and@ParameterizedTest
will fail n times with aParameterResolutionException
, then run the parameterized test as normalTest class
Expected result
The parameterized test should run 5 times with the parameters generated by the
source()
method egActual result
There are 5 repetitions of the test, each one failing with this exception:
After this, the test is run with the parameters generated by
source()
This occurs on 5.0.0 and 5.1.0-M1
The text was updated successfully, but these errors were encountered: