stdlib: DeepRecursiveFunction

[elizarov]

Introduces coroutine-based framework to execute deeply recursive functions that utilitie the heap and thus avoid StackOverflowError.

Fixes KT-31741

[LifeIsStrange]

I wonder what are the pros and cons (mainly about performance) of storing deeply recursive functions on the heap versus on a growable stack.
The latter is the solution that GCC has implemented ->
https://gcc.gnu.org/wiki/SplitStacks
By llvm ->
https://llvm.org/docs/SegmentedStacks.html
Could be used by Kotlin native
I don't know what would be the equivalent in JVM semantics

Edit1:
I believe there is, as always, a lot to learn from prior art.
It happens that rust has "solved" deeply recursive stack overflow just two weeks ago, what a coincidence :}
rust-lang/rust#40161
rust-lang/rust#55617
they are using this lib
https://docs.rs/stacker/0.1.9/stacker/

Ideally offloading to the heap should only be made once the function has exhausted its stack
Once a program has reached the end of its stack, a temporary stack on the heap is allocated and is switched to for the duration of a closure.
That way using the feature would be cost free when the feature is not needed, at runtime
Hence we could make all Kotlin functions automatically stackoverflow safe.
A flag to disable this behavior could be provided in order to debug unwanted deep recursivity
That would be I believe, a net improvement to the kotlin language.
Providing an optional function that this PR provide will be sadly underused thus preventing a wider set of developers from enjoying the recursive style

edit2: BTW out of the scope of this PR
But I wonder if tailcall couldn't be more permissive
I'm referring to this discussion - >
Sipkab/jvm-tail-recursion#3
where the developper shows an example of Kotlin performing suboptimal analysis.

[elizarov]

I wonder what are the pros and cons (mainly about performance) of storing deeply recursive functions on the heap versus on a growable stack.

This is mostly irrelevant w.r.t. this issue since we are running on a JVM that does not support any kind of segmented stacks or any similar solution. If it were, then we would not have a problem of running deeply recursive functions on JVM, but it is not and is not planned to in any conceivable future.

[udalov]

I'd prefer if we annotated the new internal API with @SinceKotlin("1.4") as well, to lessen the probability of accidentally generating access to it in the backend without checking if the API version is at least 1.4.

[elizarov]

Does the expect or actual version or both need to be annotated this way? Also, it is inline and its actuals are @InlineOnly. Does it really matter?

[udalov]

It's more of a custom than a requirement. Basically if I'm going to implement some new backend feature in 1.4.x and suddenly realize that using this intrinsic will help -- I'll happily use it. But with @SinceKotlin("1.4") on it, I'd think some more and either add a requirement on API version, or avoid using this function.

So since it's for documentation purposes only, I'd annotate both expect and actual. The fact that it's inline-only is not important, since the backend needs to generate a call to some intrinsic in the first place anyway (which can fail later during inlining, e.g. if the function body is not found).