Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests results of CheckAll are not returned until all results are checked #2707

Closed
busches opened this issue Dec 3, 2021 · 0 comments · Fixed by #2729
Closed

Tests results of CheckAll are not returned until all results are checked #2707

busches opened this issue Dec 3, 2021 · 0 comments · Fixed by #2729
Labels
bug 🐛 Issues that report a problem or error in the code. framework 🏗️ Pertains to the core structure and components of the Kotest framework.
Milestone

Comments

@busches
Copy link
Contributor

busches commented Dec 3, 2021

Which version of Kotest are you using
5.0.1

In V4, if I had a test using checkAll it would output the results of each condition as it finished, now the results are not outputted until all the tests are finished.

Example:

import io.kotest.core.spec.style.DescribeSpec
import io.kotest.matchers.shouldBe
import io.kotest.property.checkAll
import io.kotest.property.exhaustive.exhaustive
import kotlinx.coroutines.delay

class Kotest : DescribeSpec({
    describe("Results are delayed") {
        val myList = listOf(1, 2, 3)
        checkAll(myList.exhaustive()) {
            it("Should log for $it") {
                delay(5000)
                it shouldBe 1
            }
        }
    }
})
Screen.Recording.2021-12-03.at.9.06.41.AM.mov

Using the above contrived example, I can see that the results wait until all three have ran before they're returned to the test runner, instead of incrementally as they finish. When running very large sets of data, it is confusing if the test suite is hung or if something is happening, I thought it was hung and that something in my test setup was not finishing, when in reality it's the output behavior that has changed.

@sksamuel sksamuel added bug 🐛 Issues that report a problem or error in the code. framework 🏗️ Pertains to the core structure and components of the Kotest framework. labels Dec 10, 2021
@sksamuel sksamuel added this to the 5.0.2 milestone Dec 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐛 Issues that report a problem or error in the code. framework 🏗️ Pertains to the core structure and components of the Kotest framework.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants