SuspendCallbacks can leak

[swankjesse]

I’ve got a Redwood application that creates and cancels flows, and in some situations I’ve got SuspendCallback objects that never get called back, and never get canceled. I suspect it’s a book-keeping issue with ZiplineScope, possibly exacerbated by our flows.

I’m able to create this leak by creating and closing services. The leaked callback is tracked in Endpoint.inboundServices; here’s a sample.

zipline/host-1029 -> {InboundService@56720} SuspendCallback/Call(receiver=zipline/js-865, function=suspend fun emit(T): kotlin.Unit, args=[HostConfiguration(darkMode=false)])

[swankjesse]

I traced through and learned what’s happening:

1. Outbound side makes a suspending call to a target service T that passes a service K by-reference, like FlowZiplineService.collect()
2. The call is encoded, which puts K in the outbound side’s inboundServices map
3. That coroutine suspends in OutboundCallHandler.callSuspending()
4. The target service T is closed by some other coroutine
5. The target service cannot decode the call because T is closed, and therefore doesn’t discover that K needs to be closed

I’m gonna fix #826 first, which will prevent that untimely suspension.

One other gotcha here: the outbound side does receive the ZiplineApiMismatchException crash because its coroutine is already canceled by the time that arrives. This is a problem because we don’t get a signal that a leak could have happened.

[damianw]

I found a similar issue with using Flows generally - it seems that both SuspendCallback and CancelCallbacks are leaked. I created a sample project based on the trivial example here: https://github.com/damianw/zipline-leaky-flow

Running the sample prints some leaks. Using the debugger, we can see that these were SuspendCallback and CancelCallback. Is this ultimately the same problem as described in this issue, or should I file a new issue?

[swankjesse]

I expect it’s likely the same issue. Could I talk you into contributing a test case?

[damianw]

Sure, will take a look. 👍

[damianw]

Added a test in #850, though I'm not sure that this is really the same issue anymore. Or maybe it is, and nothing is really leaking. Let me know what you think.

[swankjesse]

Confirmed fixed by the ZiplineScope features.