Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should error when Container tests do not contain a nested test #2383

Closed
Tracked by #2226
miguelbaldi opened this issue Jul 23, 2021 · 4 comments · Fixed by #2530
Closed
Tracked by #2226

Should error when Container tests do not contain a nested test #2383

miguelbaldi opened this issue Jul 23, 2021 · 4 comments · Fixed by #2530
Labels
bug 🐛 Issues that report a problem or error in the code. framework 🏗️ Pertains to the core structure and components of the Kotest framework.
Milestone

Comments

@miguelbaldi
Copy link

miguelbaldi commented Jul 23, 2021

Which version of Kotest are you using
4.6.1

I've noticed the following behavior when I've upgrade Gradle from 6.6.1 to 7.1.1.
When using WordSpec style, if we write the tests with When and should, without one more nested level, test class does not show on JUnit HTML report.
I was able to observe that the tests were running when a test fail, this way, the test is shown but without class name and package (default-package name instead). If we add one more level nested inside should, the tests are shown correctely.
Dependencies versions:

    testImplementation 'io.kotest:kotest-runner-junit5:4.6.1'
    testImplementation 'io.kotest:kotest-assertions-core:4.6.1'
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.6.2'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.6.2'

Compile but does not produce correct JUnit HTML report:

class NotWorkingTests : WordSpec({
    "Hello" When {
        "asked for length" should {
            "Hello".length shouldBe 5 // If we set wrong value here, test is shown on HTML report with wrong package(default-package)
        }
        "appended to Bob" should {
            "Hello " + "Bob" shouldBe "Hello Bob"
        }
    }
})

Compile and does show on JUnit HTML report:

class WorkingTests : WordSpec({
    "Hello" When {
        "asked for length" should {
            "working with gradle 7" {
                "Hello".length shouldBe 5
            }
        }
        "appended to Bob" should {
            "working with gradle 7"{
                "Hello " + "Bob" shouldBe "Hello Bob"
            }
        }
    }
})

Is this a bug? Why this works on Gradle 6.6.1?
I've made an example project to show this behaviour:

Sample project

UPDATE

Actually, the way the reports are shown, has fundamentally changed:
Gradle 6
2021-07-26-15:07:17_1920x1080
Gradle 7
2021-07-26-15:08:12_1920x1080

@sksamuel sksamuel added bug 🐛 Issues that report a problem or error in the code. framework 🏗️ Pertains to the core structure and components of the Kotest framework. labels Aug 1, 2021
@sksamuel sksamuel added this to the 4.6.2 milestone Aug 1, 2021
@sksamuel
Copy link
Member

sksamuel commented Aug 1, 2021

It's because you haven't "completed" the test.
In WordSpec the test after "should" is the outer test and should always be included.
I think for 5.0 we can make the test framework let you know if you don't complete the test properly.

@sksamuel sksamuel modified the milestones: 4.6.2, 5.0 Aug 1, 2021
@sksamuel sksamuel changed the title Gradle 7 with WordSpec does not show test on JUnit HTML report Should error when Container tests do not contain a nested test Aug 1, 2021
@miguelbaldi
Copy link
Author

Thanks for the response, @sksamuel !

And what about the way the tests are shown on Gradle 7, they are pretty different, is that the intended behaviour?

@sksamuel
Copy link
Member

sksamuel commented Sep 10, 2021

Gradle doesn't understand nested tests.
Some open bugs at gradle for several years. I wouldn't count on them being fixed anytime soon.

@sksamuel
Copy link
Member

Added in master for 5.0 release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐛 Issues that report a problem or error in the code. framework 🏗️ Pertains to the core structure and components of the Kotest framework.
Projects
None yet
2 participants