Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runBlockingTest fails with "This job has not completed yet" #1204

Closed
PaulWoitaschek opened this issue May 17, 2019 · 58 comments
Closed

runBlockingTest fails with "This job has not completed yet" #1204

PaulWoitaschek opened this issue May 17, 2019 · 58 comments
Assignees
Labels

Comments

@PaulWoitaschek
Copy link
Contributor

@PaulWoitaschek PaulWoitaschek commented May 17, 2019

This simple block:

  runBlockingTest {
   suspendCancellableCoroutine<Unit> { cont ->
     thread {
       Thread.sleep(1000)
       cont.resume(Unit)
     }
   }
 }

Fails with the exception:

Exception in thread "main" java.lang.IllegalStateException: This job has not completed yet
    at kotlinx.coroutines.JobSupport.getCompletionExceptionOrNull(JobSupport.kt:1114)
    at kotlinx.coroutines.test.TestBuildersKt.runBlockingTest(TestBuilders.kt:53)
    at kotlinx.coroutines.test.TestBuildersKt.runBlockingTest$default(TestBuilders.kt:45)
    at SamplKt.main(Sampl.kt:8)
    at SamplKt.main(Sampl.kt)

This looks like a bug to me as it should pass.

Kotlin 1.3.31
Coroutines 1.2.1

@objcode
Copy link
Contributor

@objcode objcode commented May 17, 2019

Thank you for this minimal repro bug report!

Adding some tests for this and taking a look at getting this patched.

@objcode
Copy link
Contributor

@objcode objcode commented May 17, 2019

as an aside if anyone else is seeing this until it gets fixed - it appears the failure is caused by the thread (the blocking call to sleep is not required)

@objcode
Copy link
Contributor

@objcode objcode commented May 18, 2019

Patch #1206 should fix it!

It turns out the behavior of completion in the presence of multiple threads had a non-deterministic outcome that could, in some cases, cause a random test result.

This patch fixes the issue reported here by making it pass (as expected) while providing a detailed error message and test failure if this race condition is detected.

@rajab57
Copy link

@rajab57 rajab57 commented Jul 30, 2019

Is there a workaround for this issue ?

@jimmymorales
Copy link

@jimmymorales jimmymorales commented Aug 4, 2019

I came with this exception for some Room database tests (androidTest) using the AndroidJUnit4 runner.
It stopped throwing the exception when I added the InstantTaskExecutorRule rule.

@get:Rule
val instantExecutorRule = InstantTaskExecutorRule()

I hope this helps!

@Mugurell
Copy link

@Mugurell Mugurell commented Oct 30, 2019

Is there a workaround for this issue ?

I'm using something like

@Test
fun test() = runBlocking {
    coroutineJob.join()
}

@Anton-Spaans-Intrepid
Copy link

@Anton-Spaans-Intrepid Anton-Spaans-Intrepid commented Oct 30, 2019

@objcode I'm not sure if this patch will fix this issue entirely.

E.g. this code, that uses no other threads, fails with the "This job has not completed" message"

    runBlockingTest {
        flow<Unit> {
            suspendCancellableCoroutine<Nothing> {  }
        }.collect {  }
    }

while this code just finishes without any problems:

    runBlockingTest {
        flow<Unit> {
            delay(Long.MAX_VALUE)
        }.collect {  }
    }

@objcode
Copy link
Contributor

@objcode objcode commented Oct 30, 2019

@Anton-Spaans-Intrepid I would expect that to fail as it the collect {} will never complete.

In the patched version, you should see a 30 second wait followed by an error.

What behavior are you expecting?

@streetsofboston
Copy link

@streetsofboston streetsofboston commented Oct 30, 2019

(I just discovered I used my other GitHub account ... ah well 😄)

I expect the test to never end. It will hang.

@objcode
Copy link
Contributor

@objcode objcode commented Oct 30, 2019

@streetsofboston Hm, interesting. Can you elaborate on why?

To me this seems like it "should" fail since you're suspending the test coroutine (the coroutine in rbt) with suspendCancellableCoroutine<Nothing> { } which means during cleanup there is a non-completed coroutine.

There's probably three options:

  1. Keep this error as is
  2. Generate a different error if the only uncompleted job at the end is the test lambda
  3. Actually let the test hang forever (this is probably not optimal imo since it's somewhat easy to trigger and hard to debug)

@streetsofboston
Copy link

@streetsofboston streetsofboston commented Oct 31, 2019

@objcode
If the code were regular non-suspendable code, just plain blocking code, writing this would hang the test forever as well:

fun test() {
    val flowable = Flowable.never<Unit>()
    flowable
        .firstOrError().blockingGet()
}

The above test would never return.

You could make the case that if a non-suspending piece of code never completes, a corresponding suspending piece of code should never resume (and when wrapping that in a runBlockingXXX { ... } block, the code would never return).

Also, a 30 seconds timeout may not be enough for some tests... should the timeout be configurable?

In the end, it is a matter of taste 😄
Hanging tests are not good, but when regular non-suspending code never completes, the test will hang as well. Should that be the same for suspending code inside a runBlockingTest { ... } block or should we use a timeout, keeping in mind that a plain runBlocking { ... } will hang forever?

@premnirmal
Copy link

@premnirmal premnirmal commented Oct 31, 2019

Any update on when this fix will be released?

@nschwermann
Copy link

@nschwermann nschwermann commented Nov 3, 2019

Having this issue as well, Seems it only happens if touching files in android framework or that contain livedata not quite sure...

@premnirmal
Copy link

@premnirmal premnirmal commented Nov 3, 2019

Happens to my tests (not using livedata). Here are the tests: https://github.com/premnirmal/StockTicker/blob/master/app/src/test/kotlin/com/github/premnirmal/ticker/network/StocksApiTest.kt
I have to use runBlocking right now instead of runBlockingTest until this is fixed

@kimble
Copy link

@kimble kimble commented Jan 8, 2020

Here is another minimal test case:

@Test
fun demo() {
    val scope = TestCoroutineScope()

    scope.runBlockingTest {
        val message = suspendCancellableCoroutine<String> { c ->
            thread(start = true) {
                Thread.sleep(10)
                c.resume("Hello..")
            }
        }

        assertEquals("Hello..", message)
    }
}

This fails with

java.lang.IllegalStateException: This job has not completed yet

at kotlinx.coroutines.JobSupport.getCompletionExceptionOrNull(JobSupport.kt:1188)
at kotlinx.coroutines.test.TestBuildersKt.runBlockingTest(TestBuilders.kt:53)
at kotlinx.coroutines.test.TestBuildersKt.runBlockingTest(TestBuilders.kt:72)

@digitalsanctum
Copy link

@digitalsanctum digitalsanctum commented Jan 12, 2020

I just ran into this issue.

Any update on a permanent fix for the issue or suggested workarounds would be appreciated.

@nschwermann
Copy link

@nschwermann nschwermann commented Jan 12, 2020

@digitalsanctum
Copy link

@digitalsanctum digitalsanctum commented Jan 12, 2020

Saw that workaround after I posted. Thanks!

@qwwdfsad qwwdfsad self-assigned this Jan 15, 2020
@caiodev
Copy link

@caiodev caiodev commented Jan 19, 2020

Having the same issue. But temporarily one can use runBlocking without having any issues

@DanielAsher
Copy link

@DanielAsher DanielAsher commented Jan 24, 2020

I've run into this issue testing a long running time-line schedule. Using runBlocking is not an option for me as the test would run too long.

@LouisCAD
Copy link
Contributor

@LouisCAD LouisCAD commented Jan 24, 2020

chrisbanes added a commit to chrisbanes/tivi that referenced this issue Feb 10, 2020
Had to migrate to runBlocking {} due to
Kotlin/kotlinx.coroutines#1204
@laim2003
Copy link

@laim2003 laim2003 commented Feb 6, 2021

@LouisCAD

@laim2003 JUnit, kotest and other testing frameworks allow you to generate tests dynamically.

When you combine this capability with a single entry-point with runBlocking, you can run tests against asynchronous code in parallel so long as there's no shared mutable state between the tests. This is also closer to real-world conditions, no hacks involved, so you test your code better than with hacks that don't happen when your final program runs.

So, IMO, saying testing asynchronous code is cancer is a bit of an overstatement. I think that with Kotlin, we actually have the best facilities in the industry for this, despite that rarely usable runBlockingTest function.

BTW, if you want to test flows, here's a very good third-party library made from the renowned devs from Square: https://github.com/cashapp/turbine

honestly my reply came from a short-term frustration so yes it was an overstatement. But thanks for the recommendations anyway

@samg-stripe
Copy link

@samg-stripe samg-stripe commented Feb 12, 2021

I'm also using kotlinx-coroutines-reactive and found that using the executor of the TestCoroutineDispatcher to create a scheduler for RxJava subscriptions allowed me to use runBlockingTest:

private val testDispatcher = TestCoroutineDispatcher()
private val scheduler = Schedulers.from(testDispatcher.asExecutor())

@Test
fun someTest() = testDispatcher.runBlockingTest {
    val x = SomeClassThatUsesRxJava(scheduler)  // Use this scheduler with Observable::subscribeOn
    x.awaitSomething()
}

Of course, this means that you must use this same scheduler throughout the class you're testing.

@thaibt
Copy link

@thaibt thaibt commented May 13, 2021

Hi guys, I published a post on stackoverlow and just saw your response about this issue. I tried to replace the runBlockingTest by runBlocking but my test seems to wait in an infinite loop.
Can someone help me pass this unit test please?

Same issue with this as of May 12. runBlockingTest throws error with job never completed. runBlocking results in the stuck being stuck in an infinite wait after the mocks are created.

EDIT: Found a way to bypass though by instead of Mocking my Mono/Flux requests. I gave it values instead so my awaitFirstOrNull method would actually return.

@siralam
Copy link

@siralam siralam commented Jun 9, 2021

Injecting dispatcher worked for me. The main point is to make sure you are using the same test dispatcher instance in the block inside runBlockingTest { }.

For my project, we have DI library to inject dispatchers, and when you override the dispatcher in tests, make sure you are using the same instance instead of creating new instances every time you ask for a dispatcher.

And then when you call runBlockingTest, make sure you call from your test dispatcher instance, e.g. coroutineRule.testDispatcher.runBlockingTest { ... } instead of just runBlockingTest {...}.

I spoke more on this SO thread.

@thanhnh98
Copy link

@thanhnh98 thanhnh98 commented Oct 21, 2021

You can get something from this issue

kotlin-hands-on/intro-coroutines#4

dkhalanskyjb added a commit that referenced this issue Nov 1, 2021
Implement a multiplatform runTest as an initial implementation of #1996.

Fixes #1204
Fixes #1222
Fixes #1395
Fixes #1881
Fixes #1910
Fixes #1772
dkhalanskyjb added a commit that referenced this issue Nov 17, 2021
Implement a multiplatform runTest as an initial implementation of #1996.

Fixes #1204
Fixes #1222
Fixes #1395
Fixes #1881
Fixes #1910
Fixes #1772
dkhalanskyjb added a commit that referenced this issue Nov 17, 2021
Implement a multiplatform runTest as an initial implementation of #1996.

Fixes #1204
Fixes #1222
Fixes #1395
Fixes #1881
Fixes #1910
Fixes #1772
dkhalanskyjb added a commit that referenced this issue Nov 19, 2021
Implement a multiplatform runTest as an initial implementation of #1996.

Fixes #1204
Fixes #1222
Fixes #1395
Fixes #1881
Fixes #1910
Fixes #1772
@evesforeva
Copy link

@evesforeva evesforeva commented Dec 4, 2021

runBlockingTest works for me when I switched from using Flow.first to turbine

@hantsy
Copy link

@hantsy hantsy commented Dec 6, 2021

Still encountered this issue when using Kotlin Coroutines 1.5.1/Micronaut 3.2/Kotest, I created a post on stackoverflow.

@qwwdfsad
Copy link
Member

@qwwdfsad qwwdfsad commented Dec 6, 2021

This issue is fixed in 1.6.0-RC where runBlockingTest was deprecated (for the sake of backwards compatibility) and runTest without the described pitfall was introduced

@CDRussell
Copy link

@CDRussell CDRussell commented Dec 6, 2021

I found it hard to find the official docs when searching for runTest, so posting a direct link to help others find the right docs they'll need when migrating away from runBlockingTest.

yorickhenning pushed a commit to yorickhenning/kotlinx.coroutines that referenced this issue Jan 28, 2022
This commit introduces the new version of the test module.
Please see README.md and MIGRATION.md for a thorough
discussion of the changes.

Fixes Kotlin#1203
Fixes Kotlin#1609
Fixes Kotlin#2379
Fixes Kotlin#1749
Fixes Kotlin#1204
Fixes Kotlin#1390
Fixes Kotlin#1222
Fixes Kotlin#1395
Fixes Kotlin#1881
Fixes Kotlin#1910
Fixes Kotlin#1772
Fixes Kotlin#1626
Fixes Kotlin#1742
Fixes Kotlin#2082
Fixes Kotlin#2102
Fixes Kotlin#2405
Fixes Kotlin#2462

Co-authored-by: Vsevolod Tolstopyatov <qwwdfsad@gmail.com>
@mnewlive
Copy link

@mnewlive mnewlive commented Feb 1, 2022

For me doesn't work this solution. Instead the test falls into eternal processing.

Here is my code:

@Test
fun `when is NULL zone`() = runBlockingTest {
    // Arrange
    accessZone.emit(AccessZone.NULL)

    viewModel = RouteScreenViewModel(
        authorizationFeature = authorizationFeature,
        authenticationFeature = authenticationFeature,
        mainScreenFeature = mainScreenFeature,
        interactor = interactor,
        splashScreenInteractor = splashScreenInteractor
    )

    // Action
    val destination = viewModel.destination.first()

    // Assert
    assertNull(destination)
}

And here is some code of my ViewModel, which i want to test:

init {
    viewModelScope.launch {
        interactor.zone.collect { zone ->
            logger.debug("zone = $zone")
            val nextDestination = when (zone) {
                NULL -> null
                MAIN_CONTENT, MAIN_CONTENT_DEMO -> mainScreenFeature.getRoute()
            }
            nextDestination?.let { destinationFlow.emit(it) }
        }
    }

UPD:
For example when i tried to test MAIN_CONTENT with runBlockingTest in this file, everything it works fine:

@Test
fun `when is MAIN_CONTENT zone`() = runBlockingTest {
    // Arrange
    accessZone.emit(AccessZone.MAIN_CONTENT)
    coEvery { mainScreenFeature.getRoute() } returns "getRoute"

    viewModel = RouteScreenViewModel(
        authorizationFeature = authorizationFeature,
        authenticationFeature = authenticationFeature,
        mainScreenFeature = mainScreenFeature,
        interactor = interactor,
        splashScreenInteractor = splashScreenInteractor
    )

    // Action
    val destination = viewModel.destination.first()

    // Assert
    assertEquals("getRoute", destination)
}

@PaulWoitaschek
Copy link
Contributor Author

@PaulWoitaschek PaulWoitaschek commented Feb 1, 2022

@mnewlive you should migrate to runTest

@mnewlive
Copy link

@mnewlive mnewlive commented Feb 1, 2022

@mnewlive you should migrate to runTest

Why? If my second test case works fine?
That is, will one test case run with runBlockingTest and the other with runTest?

@PaulWoitaschek
Copy link
Contributor Author

@PaulWoitaschek PaulWoitaschek commented Feb 1, 2022

Because it's deprecated and often time making wrong tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests