Local lambda parameters (aka stable callsInPlace)

[YuliaKorolenko]

This is a discussion of the design proposal for introducing local and local once modifiers on lambda parameters:

fun <T> List<T>.getOrElse(index: Int, local defaultValue: (Int) -> T): T {
    return if (index in 0..<size) get(index) else defaultValue(index)
}

fun <T> run(local once block: () -> T): T { // or maybe just `once`
    return block()
}

Essentially, the proposal is about stabilizing the most commonly used and useful part of the callsInPlace contract, addressing limitations of the current approach. local and local once allows the compiler to perform certain smart-casts and assignments inside such lambdas.

This proposal is a logical continuation of the warning on the capture of mutable variables by escaping lambdas: that warning relies on understanding whether a lambda can escape, while this proposal provides a stable way to express that a lambda is local.

Note that this proposal works on top of our control-flow analysis and is compatible with Ross’s proposal, where local becomes part of the type system, if we decide to move further in that direction.

[kyay10]

However, a nullable lambda is a valid candidate for the UNKNOWN and AT_MOST_ONCE invocation kind, as the function would simply execute zero times within the caller.

I don't see why the rest of the invocation kinds aren't valid here.

[CLOVIS-AI]

If it's null, then it's never invoked (since it doesn't exist), and can't be used for smart-casting?

[kyay10]

once should absolutely be a shorthand for local once. local once will be used so, so often, so making it shorter is definitely a good idea

[CLOVIS-AI]

I'm in favor of keeping this to be decided in the future (e.g. during the experimental phase). The interaction between the new keyword local and inline and crossinline is already going to be subtle to teach to developers (since it mostly impacts safe-casts, which most developers don't think much about). I think it's best to avoid adding yet another combination, at least until people are familiar with local.

I agree once will be used more often and is shorter, but I also think the name is not as intuitive, and it's lifetime-adjacent, which is not something Kotlin developers usually think of.

[rnett]

The spec does not seem to imply that it is even possible to use once independently. If they are always tied together, i.e. we have local and local once as opposed to local and once, then I am in favor of making local once just once. I'm not sure what once could even mean for a non-local parameter - that it's only executed once, even if that execution may be after the function completes? How could that even be enforced, and is it valuable to the compiler to know?

[YuliaKorolenko]

We also did not find any really useful examples for once without local. If the lambda is not local and can escape, the once guarantee cannot be verified at compile time.

[kyay10]

What's the plan w.r.t. eventually getting rid of EXACTLY_ONCE and UNKNOWN? I don't think they should actually be gotten rid of. As you mention, there are functions that don't meet the strict requirements of local. Key among them is withContext.
You might say that maybe we can allow some suppression for these requirements, but:
Ross Tate's local likely wouldn't allow withContext's block to be considered local. His proposal mentions using local to get rid of runtime checking of flow invariants, which heavily suggests that changing coroutine context would not be allowed in a local-preserving way. Similar arguments apply to coroutineScope since it adds its own job, thus changing the context, and definitely violating flow invariants.

[YuliaKorolenko]

EXACTLY_ONCE and UNKNOWN, as well as contracts themselves, are not planned to be removed from the language. They can still be useful for experiments, for cases where users are sure about their code but the compiler cannot verify it, and for smaller use cases if local becomes different in the future.

[CLOVIS-AI]

When a callsInPlace contract can be safely replaced with local or local once, the IDE should suggest the corresponding modifier. Cases that do not satisfy the stricter local verification rules should not be migrated automatically.

Are there cases where a callsInPlace doesn't follow the new restrictions, but is legitimate?

My understanding is that an existing call that doesn't follow the restrictions in this document is a bug (that the compiler was not yet able to detect). If that's the case, maybe the IDE can be a bit more aggressive?

[YuliaKorolenko]

I also think most existing callsInPlace usages that do not follow these restrictions should be treated as bugs. And the current plan is to fix callsInPlace validation, forbid returns, allow nullability and other small things to align the rules with stricter local.

[CLOVIS-AI]

The nature of the KEEP process means most comments are often complaining about the new proposals; I'd like to say that I really like this KEEP. callsInPlace has always been mysterious to me, and I definitely did not understand the exact restrictions (e.g. the "may not catch an exception from the lambda").

I think the name local is very intuitive for what the restrictions are, and having the compiler check them will be a major improvement to correctly providing smart-cast-capable functions to library authors.

[CLOVIS-AI]

Can a local lambda be marked crossinline?

crossinline is only usable in inline methods, so by definition it already follows the restrictions of local. However, crossinline's main usage is to forbid non-local returns, so to me it seems that allowing it to be marked as local is incoherent.

[kyay10]

(at the risk of diverting the discussion) I think it's a chicken-or-egg problem. crossinline arose first and foremost, I think, as a way to get the benefits of inlining but in situations where the lambda is getting stored away somewhere. For technical reasons, it thus can't safely support non-local returns because it can easily leak (you need lifetimes or a system like it to prevent that). The fact that library developers then used it to prevent non-local returns even when the lambda wasn't leaked is a "happy accident".
But, of course, that's part of what crossinline means now, so that should likely be preserved.
To be clear, I don't think anything changes with this proposal at all w.r.t. crossinline. In fact, crossinline local, under this proposal, only has the library dev purpose. local has practically the same limitation as an inline lambda, so crossinline does not lift any limitations for the library writer.

I think the trickiness is in what exactly crossinline local should entail in Ross's world. Given that this will likely be an intermediary phase between our world and Ross's, I think it's perfectly reasonable to declare that crossinline local should continue to prevent explicit suspending and non-local returns (and breaks and continues), but it should also allow calling functions within the local lifetime (and hence it might implicitly use non-local control and suspension).

This would upset some users of crossinline that e.g. don't want suspension, but thus they shouldn't use local, and instead use the contract.

[CLOVIS-AI]

I think, as a way to get the benefits of inlining but in situations where the lambda is getting stored away somewhere.

Is there a chance we are confusing crossinline and noinline here? As far as I know, crossinline forbids non-local return but the lambda is still inlined normally, and noinline makes it a regular lambda.

I think local crossinline should not allow non-local return (but smart casts are still allowed), while local noinline has the exact same semantic behavior as a local lambda in a non-inline method.

[kyay10]

The funny thing is that non-inline local would actually allow non-local returns in Ross's world. That's why it's confusing.

Your behaviour assessment is absolutely correct under the current proposal. My point is that local in Ross's world allows for what inline allows now and more, so local crossinline and local noinline are interesting.

[YuliaKorolenko]

crossinline is only usable in inline methods, so by definition it already follows the restrictions of local.

crossinline does not imply local. It only forbids non-local returns, but the lambda can still escape, as in the example:

lateinit var saved: () -> Unit

inline fun store(crossinline block: () -> Unit) {
    saved = { block() }
}

But regular lambda parameters in inline functions will be implicitly local.

[YuliaKorolenko]

For this proposal, we would keep local crossinline simple: crossinline keeps its current meaning and local only shows the locality guarantee. We do not plan any special behavior around non-local returns or suspension here.

[rnett]

There will be functions with parameters that are local but are not marked as such, e.g. because they are from older libraries. What can we do to call them with a local lambda? In Ross's proposal, this would just be a cast to a new lifetime, but I don't see a clear equivalent here.

[kyay10]

If they're inline, they're local by default. Worst case, you can use callsInPlace instead for now.