[Proposal]: ref struct closures for lambdas

[agocke]

Discussion for #10209

[hez2010]

I think it doesn't have to be a ref struct. Being a struct is enough to let the JIT specialize the implementation.
Otherwise, it will not be able to use across suspension point like await or yield.

== UPDATE ==

That's all said. Allowing a ref struct closure can enable cases like capturing a ref/byref into the closure, which does have its benefit.

So, I would like to make it be something like:

• If there's any ref/byref being captured into the closure, the closure will be a ref struct
• Otherwise, the closure will be a struct

Or alternatively makes it a declaration site thing:

• If the generic constraint has a allows ref struct, the closure will be a ref struct
• Otherwise, the closure will be a struct

[timcassell]

Right, this looks like a neutered version of #8816.

[hez2010]

That's all said. Allowing a ref struct closure can enable cases like capturing a ref/byref into the closure, which does have its benefit.

So, I would like to make it be something like:

• If there's any ref/byref being captured into the closure, the closure will be a ref struct
• Otherwise, the closure will be a struct

Or alternatively makes it a declaration site thing:

• If the generic constraint has a allows ref struct, the closure will be a ref struct
• Otherwise, the closure will be a struct

[agocke]

It needs to be either a ref struct or the closure must itself be passed by ref to handle side effects of closed over value types

[GeeLaw]

Even if it's reference types, the references are values (reassigning referent). The closures should always capture non-byre(-like) locals using ref fields (or pointers, see below). Putting locals directly into the value closure is problematic when there're multiple closures capturing the same local (yes, one can do it with a lot of try-finally to transfer the values, but that's just complication and won't work well if inside a method, there are both delegate-based closures and value closures).

On top of capturing using ref fields or pointers, the value closures should still be passed by ref for performance reasons --- if there are many captures, the closure will be large.

---

On a second thought, all the locals should be put into a big struct (the "display struct"), and every closure in that scope should hold a pointer to the display struct. The closure types should always be byref-like to prevent accidental boxing. If the receiving method doesn't allows ref struct, there's no way to simultaneously achieve all of the following: (1) not allocating on managed heap; (2) safety, so that the locals cannot be accessed after the method returns; (3) capture by reference (the usual semantics for lambda).

[hez2010]

I misunderstood the original purpose of this proposal. The proposal was meant to support byref/byref-like local capture in lambdas, so I'm totally fine with the current design where closure type is a ref struct.

Well I hope that the lower strategy can be more flexible where it only emits ref struct when any byref/byref-like local being captured or any local being captured is mutated, and struct otherwise.

[GeeLaw]

This comment was moved to #10211. Feel free to mark as resolved to collapse.

[agocke]

It’s illegal to close over ref-like variables — this proposal doesn’t change that.

[GeeLaw]

The intention is to propose that it should be allowed, as there's no technical difficulty and it's useful.

[agocke]

You’ll need a separate proposal — I don’t want to deal with the complexity right now.

[huoyaoyuan]

A concern is type inference of lambda expressions. If we haven't done the var support of delegates in C# 10, we can now specify its type as the unspeakable closure type, like C++ or Rust.
Without the ability of declaring a variable with unspeakable type, we may loose some opportunity of optimizations. Local function can be a workaround, since the conversion from local function to the new delegate interface can be specified now.

[Richiban]

It's either missing from the examples or the title is misstated, but none of the examples from the proposal doc actually involve closures:

For instance, the following code

List<int> list = [1, 2, 3];

Console.Write(FirstOrNull(list, x => x % 2 == 0));

static int? FirstOrNull<TPred>(List<int> list, TPred pred)
    where TPred : allows ref struct, IFunc<int, bool>
{
    var filtered = new List<int>();
    foreach (var item in list)
    {
        if (pred.Invoke(item))
        {
            return item;
        }
    }
    return null;
}

would compile and produce the output "2". The generated code for the lambda would look equivalent to

ref struct __Closure : IFunc<int, bool>
{
    public bool Invoke(int x) => x % 2 == 0;
}