TestFactory doesn't seem to work with Kotlin (lambdas)

[maio]

Hello there! First, thanks a lot for ApprovalTests.Java :)

I'm having a problem with TestFactory style tests written in Kotlin. I would like to use Approvals.verify in them but I get exception.

Following test

class KotlinApprovalTest {
    @TestFactory
    fun `test factory`(): List<DynamicTest> {
        return listOf(1, 2).map {
            dynamicTest("test $it") {
                Approvals.verify(it)
            }
        }
    }
}

throws this exception:

	at org.approvaltests.namer.AttributeStackSelector.selectElement(AttributeStackSelector.java:67)
	at com.spun.util.tests.TestUtils.getCurrentFileForMethod(TestUtils.java:206)
	at com.spun.util.tests.TestUtils.getCurrentFileForMethod(TestUtils.java:199)
	at org.approvaltests.namer.StackTraceNamer.<init>(StackTraceNamer.java:18)
	at org.approvaltests.Approvals$1.load(Approvals.java:43)
	at org.approvaltests.Approvals$1.load(Approvals.java:40)
	at org.approvaltests.Approvals.createApprovalNamer(Approvals.java:250)
	at com.spun.util.ArrayUtils.getOrElse(ArrayUtils.java:297)
	at org.approvaltests.core.Options$FileOptions.getNamer(Options.java:89)
	at org.approvaltests.Approvals.verify(Approvals.java:176)
	at org.approvaltests.Approvals.verify(Approvals.java:52)
	at org.approvaltests.Approvals.verify(Approvals.java:61)
	at org.approvaltests.Approvals.verify(Approvals.java:56)
	at cz.maio.KotlinApprovalTest.test_factory$lambda-1$lambda-0(KotlinApprovalTest.kt:97)
	at org.approvaltests.integrations.junit5.JUnit5Approvals.lambda$dynamicTest$0(JUnit5Approvals.java:17)
	at org.junit.jupiter.engine.descriptor.DynamicTestTestDescriptor.lambda$execute$0(DynamicTestTestDescriptor.java:52)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.api.extension.InvocationInterceptor.interceptDynamicTest(InvocationInterceptor.java:161)
	at org.junit.jupiter.engine.descriptor.DynamicTestTestDescriptor.lambda$execute$1(DynamicTestTestDescriptor.java:58)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptorCall.lambda$ofVoid$0(InvocationInterceptorChain.java:78)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.descriptor.DynamicTestTestDescriptor.execute(DynamicTestTestDescriptor.java:57)
	at org.junit.jupiter.engine.descriptor.DynamicTestTestDescriptor.execute(DynamicTestTestDescriptor.java:31)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
	at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:194)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)


java.lang.RuntimeException: Could not find Junit/TestNg TestCase you are running, supported frameworks: Junit3, Junit4, Junit5, TestNg

	at org.approvaltests.namer.AttributeStackSelector.selectElement(AttributeStackSelector.java:67)

[LarsEckart]

hey!

can you clarify, where does TestFactory come from? It's not part of junit5 but some kotlin testing library?

[maio]

hey!

can you clarify, where does TestFactory come from? It's not part of junit5 but some kotlin testing library?

It's from junit5 - org.junit.jupiter.api.TestFactory

dynamicTest = org.approvaltests.integrations.junit5.JUnit5Approvals#dynamicTest

[LarsEckart]

hmm, that rings some bells then. I wonder, I think we once looked into that. I'll check.

[LarsEckart]

There we go, I found something: https://github.com/approvals/ApprovalTests.Java/blob/master/approvaltests/src/test/java/org/approvaltests/namer/JUnit5StackTraceNamerTest.java#L92

We even have some advice when things go wrong, but apparently this exception message didn't reach you :/
https://github.com/approvals/ApprovalTests.Java/blob/master/approvaltests/src/test/java/org/approvaltests/namer/JUnit5StackTraceNamerTest.java#L117

[maio]

I'm using org.approvaltests.integrations.junit5.JUnit5Approvals.dynamicTest so I didn't get this error, but the other one from https://github.com/approvals/ApprovalTests.Java/blob/master/approvaltests/src/main/java/org/approvaltests/namer/AttributeStackSelector.java#L68 which examines stack and looks for test method.

For some reason this code doesn't seem to work with Kotlin.

I'll create small project with example, and I'll try to fix it.

[maio]

https://github.com/maio/approvals-kotlin-problem

https://github.com/maio/approvals-kotlin-problem/blob/main/src/test/java/JTest.java - test written in Java works
https://github.com/maio/approvals-kotlin-problem/blob/main/src/test/kotlin/KTest.kt - same test but in Kotlin doesn't work

https://github.com/maio/approvals-kotlin-problem/runs/5187000476#step:4:45

[maio]

Ok I found it. I'll submit PR with fix. Kotlin has different style of encoding lambda(s) in method names so https://github.com/approvals/ApprovalTests.Java/blob/master/approvaltests-util/src/main/java/com/spun/util/tests/TestUtils.java#L224 needs to be updated.