FreezingException when using Main dispatcher

[ychescale9]

👋

I'm running in to a FreezingException when integrating NativeCoroutines with Apollo3 which requires queries to be launched from Main dispatcher.

I couldn't reproduce it without apollo3 so I forked https://github.com/joreilly/MortyComposeKMM which is using Apollo3.

This is my branch with the crash: https://github.com/ychescale9/MortyComposeKMM/tree/reproduce-crash-with-NativeCoroutines-integration

This is the diff: ychescale9/MortyComposeKMM@dd65771#diff-c16bdea5c96b86a9098c45a11157d0a6f14338e025e52ca682b7753ccea205b2

Crash:

Uncaught Kotlin exception: kotlin.Throwable: The process was terminated due to the unhandled exception thrown in the coroutine [StandaloneCoroutine{Cancelling}@1160cc8, MainDispatcher]: freezing of CancellableContinuation(Shareable[kotlin.native.concurrent.WorkerBoundReference@11ebd88]){Active}@a1c408 has failed, first blocker is CancellableContinuation(Shareable[kotlin.native.concurrent.WorkerBoundReference@11ebd88]){Active}@a1c408
2021-09-12 19:34:36.176816+1000 iosApp[38612:22096996] [] nw_protocol_get_quic_image_block_invoke dlopen libquic failed
    at 0   shared                              0x000000010e32c5e2 kfun:kotlinx.coroutines#handleCoroutineExceptionImpl(kotlin.coroutines.CoroutineContext;kotlin.Throwable){} + 850 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/CoroutineExceptionHandlerImpl.kt:15:21)
    at 1   shared                              0x000000010e2879e7 kfun:kotlinx.coroutines#handleCoroutineException(kotlin.coroutines.CoroutineContext;kotlin.Throwable){} + 807 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/CoroutineExceptionHandler.kt:33:5)
    at 2   shared                              0x000000010e2774bf kfun:kotlinx.coroutines.StandaloneCoroutine.handleJobException#internal + 207 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Builders.common.kt:241:9)
    at 3   shared                              0x000000010e29491d kfun:kotlinx.coroutines.JobSupport.finalizeFinishingState#internal + 2381 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:230:59)
    at 4   shared                              0x000000010e2a07e4 kfun:kotlinx.coroutines.JobSupport.tryMakeCompletingSlowPath#internal + 1972 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:925:16)
    at 5   shared                              0x000000010e29ff02 kfun:kotlinx.coroutines.JobSupport.tryMakeCompleting#internal + 738 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:878:16)
    at 6   shared                              0x000000010e29f9b0 kfun:kotlinx.coroutines.JobSupport#makeCompletingOnce(kotlin.Any?){}kotlin.Any? + 544 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/JobSupport.kt:843:30)
    at 7   shared                              0x000000010e276043 kfun:kotlinx.coroutines.AbstractCoroutine#resumeWith(kotlin.Result<1:0>){} + 323 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/AbstractCoroutine.kt:104:21)
    at 8   shared                              0x000000010dfeaca3 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 1555 (/Users/teamcity1/teamcity_work/7c4fade2802b7783/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:43:32)
    at 9   shared                              0x000000010e339013 kfun:kotlinx.coroutines.internal.ShareableContinuation#resumeWith(kotlin.Result<1:0>){} + 595 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/internal/Sharing.kt:185:22)
    at 10  shared                              0x000000010e317abe kfun:kotlinx.coroutines.internal.ScopeCoroutine#afterResume(kotlin.Any?){} + 366 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/Scopes.kt:43:15)
    at 11  shared                              0x000000010e2760c4 kfun:kotlinx.coroutines.AbstractCoroutine#resumeWith(kotlin.Result<1:0>){} + 452 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/AbstractCoroutine.kt:106:9)
    at 12  shared                              0x000000010dfeaca3 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 1555 (/Users/teamcity1/teamcity_work/7c4fade2802b7783/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:43:32)
    at 13  shared                              0x000000010e30f0d8 kfun:kotlinx.coroutines.DispatchedTask#run(){} + 2616 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:103:25)
    at 14  shared                              0x000000010e33bb4a kfun:kotlinx.coroutines.DarwinMainDispatcher.dispatch$lambda-0#internal + 90 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt:35:19)
    at 15  shared                              0x000000010e33be60 kfun:kotlinx.coroutines.DarwinMainDispatcher.$dispatch$lambda-0$FUNCTION_REFERENCE$43.invoke#internal + 64 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt:34:51)
    at 16  shared                              0x000000010e33bf70 kfun:kotlinx.coroutines.DarwinMainDispatcher.$dispatch$lambda-0$FUNCTION_REFERENCE$43.$<bridge-UNN>invoke(){}#internal + 64 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt:34:51)
    at 17  shared                              0x000000010e33d099 _6f72672e6a6574627261696e732e6b6f746c696e783a6b6f746c696e782d636f726f7574696e65732d636f7265_knbridge8 + 185 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt:34:51)
    at 18  libdispatch.dylib                   0x000000010e9e3578 _dispatch_call_block_and_release + 12
    at 19  libdispatch.dylib                   0x000000010e9e474e _dispatch_client_callout + 8
    at 20  libdispatch.dylib                   0x000000010e9f2b3f _dispatch_main_queue_callback_4CF + 1152
    at 21  CoreFoundation                      0x00007fff203908f8 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 9
    at 22  CoreFoundation                      0x00007fff2038b169 __CFRunLoopRun + 2781
    at 23  CoreFoundation                      0x00007fff2038a1a7 CFRunLoopRunSpecific + 567
    at 24  GraphicsServices                    0x00007fff2b874d85 GSEventRunModal + 139
    at 25  UIKitCore                           0x00007fff246c14df -[UIApplication _run] + 912
    at 26  UIKitCore                           0x00007fff246c639c UIApplicationMain + 101
    at 27  iosApp                              0x000000010d925ddb main + 75 (/Users/.../MortyComposeKMM/iosApp/iosApp/AppDelegate.swift:<unknown>)
    at 28  libdyld.dylib                       0x00007fff2025abbd start + 1

This original repo is using this (which uses this dispatcher implementation) which seems to work.

[rickclephas]

Hey! Will have to take a closer look at this but I think the issue is in the suspend function.
While generating the getCharactersNative function the MortyRepository is probably being frozen (including the apolloClient property).
The nativeSuspend function intentionally freezes everything.

The difference with the original repo is that it uses a Flow (CommonFlow to be precise).
It also doesn't seem to freeze anything intentionally.

Did you already try using the characterPagingData Flow property?
If so, does it have the same freezing issue when used with KMP-NativeCoroutines?

[ychescale9]

Thanks for the quick response!

Yes using NativeFlow with createPublisher produces the same exception - this is the issue I saw originally in my work repo.

I added the following to MortyRepository:

val characterPagerFlow: Flow<PagingData<CharacterDetail>>
    get() = characterPager.pagingData.cachedIn(scope)

And creating the publisher from NativeFlow:

createPublisher(for: repository.characterPagerFlowNative)
    .assertNoFailure()
    .receive(on: DispatchQueue.main)
    .sink { completion in
        print("Received completion: \(completion)")
    } receiveValue: { value in
        print("Received value: \(value)")
    }
    .store(in: &subscriptions)

Crash:

Uncaught Kotlin exception: kotlin.Throwable: The process was terminated due to the unhandled exception thrown in the coroutine [StandaloneCoroutine{Cancelling}@3286528, MainDispatcher]: freezing of CancellableContinuation(Shareable[kotlin.native.concurrent.WorkerBoundReference@328df58]){Active}@29865c8 has failed, first blocker is CancellableContinuation(Shareable[kotlin.native.concurrent.WorkerBoundReference@328df58]){Active}@29865c8

---

Pushed a new commit with NativeFlow: ychescale9/MortyComposeKMM@21a3e6f

[rickclephas]

Alright there are a couple of things going on here.

First of all you are using v0.6.0 with Kotlin 1.5.21. Although that doesn't seem to cause any issues it would be better to use the Kotlin 1.5.20 compatibility version v0.6.0-kotlin-1.5.20.

Then there are actually 2 freezing issues (one with apollo3 and another with the multiplatform pager).
As I mentioned previously the NativeCoroutines functions intentionally freeze everything.
This allows the use of the native functions and properties from any thread in Swift.

This means the MortyRepository and the ApolloClient and Pager properties will be frozen.

The Apollo code is calling ensureNeverFrozen here which throws the exception you are seeing.
I am not sure why they are doing this.
If I downgrade to v3.0.0-dev14 (which doesn't include this call) the Apollo part is working fine.

But that brings me to the second freezing issue where the pager is trying to add items to a frozen list.
Which is causing another but similar crash:

Uncaught Kotlin exception: kotlin.Throwable: The process was terminated due to the unhandled exception thrown in the coroutine [StandaloneCoroutine{Cancelling}@15a8ff8, MainDispatcher]: mutation attempt of frozen kotlin.collections.ArrayList@e74008

Changing the pager to only use the StateFlow with immutable lists should fix this.

FYI both issues can be reproduced without KMP-NativeCoroutines by manually freezing the MortyRepository:

// commonMain
expect fun <T> T.freeze(): T

class MortyRepository {

    // this should be at the bottom of the MortyRepository
    init {
        freeze()
    }
}

// iosMain
actual fun <T> T.freeze(): T = freeze()

// androidMain
actual fun <T> T.freeze() = this

[rickclephas]

I have created issues for both the Apollo and Pager libraries.
Let me known if there is something I can improve in KMP-NativeCoroutines to help fix these (or other) freezing errors.

[ychescale9]

Thanks for the thorough investigation and creating those issues! 🙏