Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kotest plugin doesn't run test when build & running with Gradle #78

Closed
nomisRev opened this issue Jul 3, 2020 · 11 comments
Closed

Kotest plugin doesn't run test when build & running with Gradle #78

nomisRev opened this issue Jul 3, 2020 · 11 comments
Labels
wontfix This will not be worked on

Comments

@nomisRev
Copy link

nomisRev commented Jul 3, 2020

Which version of Kotest are you using

Kotest: 4.1.0, Plugin: 1.0.4

IntelliJ Version:

=> Freshly installed, new imported project with first synced Gradle & full rebuild from IntelliJ.

IntelliJ IDEA 2020.1.2 (Ultimate Edition)
Build #IU-201.7846.76, built on June 1, 2020
Licensed to IntelliJ IDEA Evaluator
Expiration date: August 2, 2020
Runtime version: 11.0.7+10-b765.53 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 10.15.5
GC: ParNew, ConcurrentMarkSweep
Memory: 1979M
Cores: 12
Registry: ide.balloon.shadow.size=0
Non-Bundled Plugins: org.jetbrains.kotlin, kotest-plugin-intellij

Configuration created:

Screenshot 2020-07-03 at 10 08 14

Output w/ Env var DEBUG_KOTEST=true:

Testing started at 10:06 ...
/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/bin/java -javaagent:/Users/simonvergauwen/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/201.7846.76/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=58552:/Users/simonvergauwen/Library/Application Support/JetBrains/Toolbox/apps/IDEA-U/ch-0/201.7846.76/IntelliJ IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -classpath /Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/charsets.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/cldrdata.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/dnsns.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/dtfj.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/dtfjview.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/jaccess.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/localedata.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/nashorn.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/sunec.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/sunjce_provider.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/sunpkcs11.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/traceformat.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/ext/zipfs.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/jce.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/jsse.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/management-agent.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/resources.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/jre/lib/rt.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/lib/dt.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/lib/jconsole.jar:/Users/simonvergauwen/.sdkman/candidates/java/8.0.252.j9-adpt/lib/tools.jar:/Users/simonvergauwen/Projects/arrow-fx/arrow-fx-coroutines/build/classes/atomicfu/main:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.arrow-kt/arrow-core/0.11.0-SNAPSHOT/191860bcd344f27ba366ad2dd93c98dbe1e96f3/arrow-core-0.11.0-SNAPSHOT.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.kotest/kotest-runner-junit5-jvm/4.1.0/a1aac2d9ea0cce895ba67a1fac7c2432a64d6a07/kotest-runner-junit5-jvm-4.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.kotest/kotest-runner-console-jvm/4.1.0/41a64549fa81a8064174ebfd95ca7bcb0c0ce8e3/kotest-runner-console-jvm-4.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.3.71/898273189ad22779da6bed88ded39b14cb5fd432/kotlin-stdlib-1.3.71.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.kotest/kotest-assertions-core-jvm/4.1.0/eb616620198f0fcbd348187c8d1800c6133aa404/kotest-assertions-core-jvm-4.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.kotest/kotest-property-jvm/4.1.0/e9a03375bc67974eeb1d3b06efdeb4c17d27f63d/kotest-property-jvm-4.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/atomicfu/0.14.3/2b91c4d5b8784f28c63ebf6a755e78b3971a4838/atomicfu-jvm-0.14.3.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.arrow-kt/arrow-core-data/0.11.0-SNAPSHOT/259368011d8f69c857e2038f7818e5076ebd0cdc/arrow-core-data-0.11.0-SNAPSHOT.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.3.61/70dffc5f8ac5ea7c34f30deac5b9d8b1d48af066/kotlin-stdlib-jdk7-1.3.61.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.arrow-kt/arrow-annotations/0.11.0-SNAPSHOT/29d22b38db42cafdd8fd564bc19da6fb1a5b791e/arrow-annotations-0.11.0-SNAPSHOT.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.3.6/84cc2e5a484555e49e908fd831ff503303401dd0/kotlinx-coroutines-core-1.3.6.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-suite-api/1.6.2/36796f8e82db47f0a09580be1fb68449b3b8ee6f/junit-platform-suite-api-1.6.2.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter-api/5.6.2/c9ba885abfe975cda123bf6f8f0a69a1b46956d0/junit-jupiter-api-5.6.2.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-launcher/1.6.2/d866de2950859ca1c7996351d7b3d97428083cd0/junit-platform-launcher-1.6.2.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-engine/1.6.2/1752cad2579e20c2b224602fe846fc660fb35805/junit-platform-engine-1.6.2.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-reflect/1.3.71/fca74670d8e6906a3241b3cf3732409ca77b2508/kotlin-reflect-1.3.71.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/com.github.ajalt/clikt/2.6.0/6d87f906ef310b5be66db1fdef0cc23c5a66688d/clikt-jvm-2.6.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/com.github.ajalt/mordant/1.2.1/6cbab1a74ab6dafbf81b7706733d4c2fbaff2e0b/mordant-1.2.1.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.8.69/6bd8c9033563e162b5c12de12b139724dbf71f48/classgraph-4.8.69.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.3.71/e71c3fef58e26affeb03d675e91fd8abdd44aa7b/kotlin-stdlib-common-1.3.71.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains/annotations/13.0/919f0dfe192fb4e063e7dacadee7f8bb9a2672a9/annotations-13.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.kindedj/kindedj/1.1.0/462731347602a3f24e3f21feec50928f9a657741/kindedj-1.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.kotest/kotest-common-jvm/4.1.0/6b1013a43e20c53dce2ad67fdb85fdcc01f594c7/kotest-common-jvm-4.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.kotest/kotest-core-jvm/4.1.0/e448d4d494f1fbbe0ac6c92ddb7f7dc21201b82d/kotest-core-jvm-4.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.kotest/kotest-extensions-jvm/4.1.0/f17a3679b3eec6d7ca0a04fb0c3589c902aa62ef/kotest-extensions-jvm-4.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.apiguardian/apiguardian-api/1.1.0/fc9dff4bb36d627bdc553de77e1f17efd790876c/apiguardian-api-1.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.junit.platform/junit-platform-commons/1.6.2/7644a14b329e76b5fe487628b50fb5eab6ba7d26/junit-platform-commons-1.6.2.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.opentest4j/opentest4j/1.2.0/28c11eb91f9b6d8e200631d46e20a7f407f2a046/opentest4j-1.2.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/com.github.ajalt/colormath/1.2.0/c62f49b31f34588dbbfb477c08fd56bc3026d202/colormath-1.2.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.kotest/kotest-assertions-shared-jvm/4.1.0/b506be009e1e365b5eeb093b257f59fecfacdfc8/kotest-assertions-shared-jvm-4.1.0.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.3.71/9180d3aec3f0b2ea6ef0dcf01b464a6e2219e381/kotlin-stdlib-jdk7-1.3.71.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.3.71/4defc79915cf4f78b49bbc4a8f1e80e87767a5b/kotlin-stdlib-jdk8-1.3.71.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-jdk8/1.3.6/7e94560a7c3cace3e6230326d2b750ac1f153b2f/kotlinx-coroutines-jdk8-1.3.6.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-core-common/1.3.6/b9032e61b87afa518c24f216c717623b4a525882/kotlinx-coroutines-core-common-1.3.6.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/com.github.wumpz/diffutils/2.2/647f913ef6c350d3b1bc1ffa48a74747a13e89f/diffutils-2.2.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/com.univocity/univocity-parsers/2.8.4/c3fadbbecfadd8b9a9911eb2d06d9cf9a873edaa/univocity-parsers-2.8.4.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/com.github.mifmif/generex/1.0.2/b378f873b4e8d7616c3d920e2132cb1c87679600/generex-1.0.2.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.eclipse.jgit/org.eclipse.jgit/4.4.1.201607150455-r/63998ced66e425d9e8bcd0c59f710c98f0c021ff/org.eclipse.jgit-4.4.1.201607150455-r.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/dk.brics.automaton/automaton/1.11-8/6ebfa65eb431ff4b715a23be7a750cbc4cc96d0f/automaton-1.11-8.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.mockk/mockk/1.9.3/d26e5a4a7fd4ecb5635b28a0b70960ab0b20556d/mockk-1.9.3.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/commons-io/commons-io/2.6/815893df5f31da2ece4040fe0a12fd44b577afaf/commons-io-2.6.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.mockk/mockk-dsl-jvm/1.9.3/63de994ae1f30b06395325258d2e5683a8bdd525/mockk-dsl-jvm-1.9.3.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.mockk/mockk-common/1.9.3/7f2d6bc3390a76edeba34947e0de6d2314645f25/mockk-common-1.9.3.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.mockk/mockk-agent-jvm/1.9.3/d6f7b4d893caf1cda2bb28ee637b247ced5d5eab/mockk-agent-jvm-1.9.3.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.mockk/mockk-dsl/1.9.3/4f158b4ed161bc4373356733088e09491ecd13da/mockk-dsl-1.9.3.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.mockk/mockk-agent-common/1.9.3/50f32c350cb9018a62e6de2bed710acf675f887b/mockk-agent-common-1.9.3.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/io.mockk/mockk-agent-api/1.9.3/467c1fd4867fc49b98a7de421a717b619767abfe/mockk-agent-api-1.9.3.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/org.objenesis/objenesis/3.0.1/11cfac598df9dc48bb9ed9357ed04212694b7808/objenesis-3.0.1.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy/1.9.10/211a2b4d3df1eeef2a6cacf78d74a1f725e7a840/byte-buddy-1.9.10.jar:/Users/simonvergauwen/.gradle/caches/modules-2/files-2.1/net.bytebuddy/byte-buddy-agent/1.9.10/9674aba5ee793e54b864952b001166848da0f26b/byte-buddy-agent-1.9.10.jar io.kotest.runner.console.LauncherKt --spec arrow.fx.coroutines.stream.CancellationTest
Exception in thread "main" java.lang.ClassNotFoundException: arrow.fx.coroutines.stream.CancellationTest
	at java.lang.Class.forNameImpl(Native Method)
	at java.lang.Class.forName(Class.java:337)
	at io.kotest.runner.console.KotestConsoleRunner.execute(KotestConsoleRunner.kt:25)
	at io.kotest.runner.console.Execute$run$1.invokeSuspend(launcher.kt:67)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:79)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:54)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at io.kotest.runner.console.Execute.run(launcher.kt:51)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:154)
	at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:14)
	at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:252)
	at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:249)
	at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:267)
	at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:290)
	at io.kotest.runner.console.LauncherKt.main(launcher.kt:15)

Process finished with exit code 1

@nomisRev
Copy link
Author

nomisRev commented Jul 3, 2020

I am able to reproduce consistently with the following setting. If it's configured to IntelliJ & IntelliJ then the test runs correctly. If I switch back to Gradle (default), then it doesn't run.
Due to tools, I am in some cases required to use Gradle since they don't support the IntelliJ build system.

Screenshot 2020-07-03 at 15 56 08

@sksamuel
Copy link
Member

sksamuel commented Jul 3, 2020

I've tried this on my projects and it works for gradle and intellij settings.
I'm wondering if there's some extra compiler stuff going on that isn't compatible - kapt or a compiler plugin or something?
Can you try cloning this project https://github.com/sksamuel/hoplite/ and running some tests in there using the plugin. That's what I've been using to validate that the plugin works. If that works for you as well we can debug specifically on your arrow fx project.

@nomisRev
Copy link
Author

nomisRev commented Jul 3, 2020

In the module that uses kotest relies on atomic-fu compiler plugin. No kapt or other plugins.

There are other unrelated modules in the project that have kapt dependencies and atomic-fu. But they all still use KotlinTest.

This behavior has been flaky for the passed week. I will give the repo you linked a try asap.

@sksamuel
Copy link
Member

sksamuel commented Jul 3, 2020 via email

@nomisRev
Copy link
Author

nomisRev commented Jul 4, 2020

@sksamuel Hey Sam,

I tried the hoplite repo and it's not reproducible for me there.

It's reproducible for me in Arrow Fx, when trying to run kotest tests from IntelliJ directly on any of the Kotest tests in the Arrow Fx Coroutines module.

Note: Whilst it's reproducible from IntelliJ using Gradle build/runner, it still runs from CLI with Gradle.

@sksamuel
Copy link
Member

sksamuel commented Jul 4, 2020 via email

@sksamuel
Copy link
Member

sksamuel commented Jul 5, 2020

Is there a branch where I can find these tests.

@nomisRev
Copy link
Author

nomisRev commented Jul 8, 2020

@sksamuel Yes, sorry for the late reply. You can find the test suite in the Arrow Fxin the Arrow Fx Coroutines Module.

This is the test suite I mentioned before on Slack. I also added a bunch of Arb for ranges, array, etc. I hope to find some time in the near future to contribute these back. I also have my own Spec classes, to manipulate iterations & depth since those are not configurable atm from ProjectConfig.

If you see anything out of the ordinary or have any feedback on the Kotest usage I'd love it 🙏

@sksamuel sksamuel transferred this issue from kotest/kotest Jul 12, 2020
@sksamuel
Copy link
Member

kotest-plugin-intellij-1.1.0.zip

I've re-written the plugin to use the latest code that junit itself uses from intellij 2020. Attached is a plugin build for you to try. To be honest I don't know if it will work, but I'm now just doing what intellij itself does for TestNG and JUnit tests.

Note: Requires Intellij 2020.1

@sksamuel
Copy link
Member

Latest plugin is out - can you try ?

@stale
Copy link

stale bot commented Oct 10, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Oct 10, 2020
@stale stale bot closed this as completed Oct 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

2 participants