Create "Fat Jar" launcher module: kotest-framework-engine-launcher that can be executed as a process

[GavinRay97]

Following conversation on Slack:

Not sure how much merit this has on it's own, but it would allow Kotest to be integrated into tooling that works with any test framework that provides a runnable binary-ish process consuming options via CLI args (as opposed to build-tool specific integrations).

[sksamuel]

I've added a standalone module. All that's left is to publish it.
@GavinRay97 how would you pass the tests to the instance ?

[GavinRay97]

Probably through a small programmatic API that wrapped arguments and then spawned the .jar
Feel like the "safe bet" is for me to copy whatever it is that JUnit does

• https://junit.org/junit5/docs/current/user-guide/#running-tests-console-launcher

It's sort of like the saying: "Nobody ever got fired for buying IBM." 😅

$ javac -d out Foo.java 
$ javac -d out -cp out:junit-platform-console-standalone-1.X.X.jar TestClass.java
$ java -jar junit-platform-console-standalone-1.X.X.jar --class-path out --scan-class-path
╷
├─ JUnit Jupiter ✔
│  └─ TestClass ✔
│     └─ test() ✔
└─ JUnit Vintage ✔

Test run finished after 67 ms

Usage: ConsoleLauncher [-h] [--disable-ansi-colors] [--disable-banner]
                       [--fail-if-no-tests] [--scan-modules] [--scan-classpath[=PATH[;|:
                       PATH...]]]... [--details=MODE] [--details-theme=THEME]
                       [--reports-dir=DIR] [-c=CLASS]... [--config=KEY=VALUE]... [-cp=PATH
                       [;|:PATH...]]... [-d=DIR]... [-e=ID]... [-E=ID]...
                       [--exclude-package=PKG]... [-f=FILE]... [--include-package=PKG]...
                       [-m=NAME]... [-n=PATTERN]... [-N=PATTERN]... [-o=NAME]...
                       [-p=PKG]... [-r=RESOURCE]... [-t=TAG]... [-T=TAG]... [-u=URI]...

[sksamuel]

lol no one ever went wrong for using junit. But mocks ... that's a different story.

…

On Mon, 18 Oct 2021 at 10:41, Gavin Ray ***@***.***> wrote: Probably through a small programmatic API that wrapped arguments and then spawned the .jar Feel like the "safe bet" is for me to copy whatever it is that JUnit does - https://junit.org/junit5/docs/current/user-guide/#running-tests-console-launcher It's sort of like the saying: "Nobody ever got fired for buying IBM." 😅 — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#2416 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFVSGQMZAEKFYF3J6WDT6TUHQ52ZANCNFSM5CKVQDHQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[sksamuel]

Published to maven now:
https://oss.sonatype.org/content/repositories/snapshots/io/kotest/kotest-framework-standalone-jvm/5.0.0.775-SNAPSHOT/

[nikunjy]

Hey thanks for writing that @sksamuel
I wrote a test

class SimpleTest: DescribeSpec() {
    init {
        describe("test1") {
            it("testA") {
                val x = 1
                val y = 2
                x + y shouldBe 3
            }
            it("testB") {
                val x = 1
                val y = 2
                x + y shouldNotBe 4
            }
        }

I am able to run this speific package, and class and spec using the launcher. For example I can do

 "--reporter", "teamcity",
                "--package", "com.kotlin.test",
                "--spec", "com.kotlin.test.SimpleTest",
                "--testpath", "test1",

And this works but how can I run the subtest let's say test1/testA. What args should I pass

[sksamuel]

--testpath test1 -- testA

the -- is our delimiter between nested tests.

[nikunjy]

Got it thank you. I am thinking of using this vs the JunitConsoleLauncher
Does it also support pattern matching, I tried something like --testpath test* but it did not work.

[sksamuel]

not, not wildcard, but it automatically prefixes, so --testpath test would find child tests (but not test2)

[nikunjy]

Thanks for the helpful information.
Is it possible to extend this launcher to support patterns like it does with gradle
https://kotest.io/docs/framework/conditional/conditional-tests-with-gradle.html

I am happy to do this PR but does it make sense to do? I think this launcher is better for my purpose because we use bazel plus junit console launcher. We have to write a reporter that bridges the bazel with junit and parse XML etc. However this launcher has a good runner, reporter abstraction and I would rather just use this.

[sksamuel]

I'm happy to expand the launcher but I'd rather do it better than the gradle way (we just do that for compatibility). Perhaps accept a wildcard for --spec and --test or something ?

…

On Wed, 31 Aug 2022 at 14:47, Nikunj Yadav ***@***.***> wrote: Thanks for the helpful information. Is it possible to extend this launcher to support patterns like it does with gradle https://kotest.io/docs/framework/conditional/conditional-tests-with-gradle.html I am happy to do this PR but does it make sense to do? I think this launcher is better for my purpose because we use bazel plus junit console launcher. We have to write a reporter that bridges the bazel with junit and parse XML etc. However this launcher has a good runner, reporter abstraction and I would rather just use this. — Reply to this email directly, view it on GitHub <#2416 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFVSGREL7E4XGC6DKFMK7TV36ZDZANCNFSM5CKVQDHQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[nikunjy]

Amazing ! Yes that would work. I would love to use it in a way where I can target the test and the child test. So let's say if we have

class SimpleTest: DescribeSpec() {
    init {
        describe("db simple test") {
            it("insert foo") {
                val x = 1
                val y = 2
                x + y shouldBe 3
            }
            it("delete") {
                val x = 1
                val y = 2
                x + y shouldNotBe 4
            }
        }

      describe("db unique test") {
            it("insert foobar") {
                val x = 1
                val y = 2
                x + y shouldBe 3
            }
            it("delete") {
                val x = 1
                val y = 2
                x + y shouldNotBe 4
            }
        }

       describe("table mine test") {
            it("insert") {
                val x = 1
                val y = 2
                x + y shouldBe 3
            }
            it("delete") {
                val x = 1
                val y = 2
                x + y shouldNotBe 4
            }
        }
}

And then we can do

"--reporter", "teamcity", 
"--package", "com.kotlin.test",
"--spec", "*",
"--testpath", "db*",

Runs all the describes that have word db in it in all the test specs under com.kotlin.test in this case there is only com.kotlin.test.SimpleTest

It is not important if you don't support --testpath in this case imo but it is a good to have.

"--reporter", "teamcity", 
"--package", "com.kotlin.test",
"--spec", "com.kotlin.test.SimpleTest",
"--testpath", "db*",

Runs describe #1 and #2

"--reporter", "teamcity", 
"--package", "com.kotlin.test",
"--spec", "com.kotlin.test.SimpleTest",
"--testpath", "db* -- insertfoo*",

Runs db simple test -- insert foo and db unique test -- insert foobar

[sksamuel]

I think that is fine.

[nikunjy]

Updated my comment with the prefix for spec.
Thanks @sksamuel let me know if I can help in some way. It would be pretty awesome to use this :)

[sksamuel]

You could make the PR :)

[nikunjy]

@sksamuel Sounds good. I will give it a shot.

[nikunjy]

Created an initial PR #3200 @sksamuel

I anticipate it might require revisions. Looking forward to getting a code review :)