Closures

[bob-carpenter]

Closure spec.

[seantalts]

You can post a rendered link here as well:
Rendered

[seantalts]

This looks awesome! Thank you so much for writing this all up, and I'm excited to get these into Stan.

In my head the only thing we need to get nailed down before merging is the capture semantics (reference v. value), and I had a question about the shorthand syntax. I'd also like to give @VMatthijs a chance to look at this but he just moved to Europe yesterday so he might not be back online until next week sometime.

[seantalts]

C++ automatically deduces the return type - maybe we could too in all cases?

[seantalts]

I assume by "risky capture" here you're referring to the capture by reference semantics? My impression is that those semantics are often confusing and infrequently useful for the programmer, modulo performance concerns. Would you mind adding some discussion of performance and other tradeoffs between capture by value and capture by reference? I would guess that would be the most difficult part of this proposal and something we'd need to settle one way or the other before implementing closures.

I would guess we'd all agree not to use R's dynamic lexical scoping semantics 😅

[seantalts]

Is this also from C++? This seems confusing without the {}, in my humble opinion.

[seantalts]

Some templating survived here.

[seantalts]

The risk in capturing them by reference, not by value, right?

[seantalts]

Do we need to consider sized types for this proposal since we can't read in or write out functions from Stan?

[seantalts]

awesome :)

[seantalts]

I think this could be a separate issue on the stanc3 issue tracker - we could probably implement this subtyping in stanc3 right now without breaking anything or confusing anyone.

[seantalts]

To migrate some comments from the thread on Discourse here - you said there that you are proposing the capture-by-reference ([&] in C++) semantics for closures in this design doc, and that you weren't sure why we'd go with by-value. I still think it'd be good to have some discussion of the tradeoffs there in the doc, so I'll try to list some reasons I can think of to prefer capture-by-value:

1. I think many people would find capture-by-value less confusing, but I'm not sure about that. I certainly find immutability in a threaded context most appropriate, and this is the wisdom I have seen presented, e.g. in books like Java Concurrency in Practice, in the design of Clojure and Scala, in the implementations of most (all?) new parallelism frameworks recently such as Spark, Hadoop, Actor frameworks for various languages, Go channels, etc.
2. Semantics for threading will be hellacious if we don't make all variables in concurrent contexts immutable. In other words, we're opening up users to surprising race conditions in any parallel or concurrent context if we pass by reference and allow folks to mutate those captured variables within the lambda functions (which is what [&] allows in C++, if that's the proposed semantic). Making the referenced captures immutable within lambdas solves this issue but only in the context of threaded map_rect; if we start having other forms of concurrent programs it can become very confusing again.
3. Capture-by-reference semantics with mutable variables would add a source of non-determinism in a threaded context, making reproducibility harder.
4. We can pass these closures to threaded functions, but not to MPI, which destroys one of the major use-cases for lambdas with closures (a more user-friendly map_rect version).
5. We lose other immutability guarantees about Stan programs that we can use for optimization.

I claim we can have pass-by-value semantics with pass-by-reference performance in most cases by compiling down to closures with [&] wherever appropriate but creating copies if there's any chance the captured variable is mutated after the fact.

I'm a bit sleepy still - just to double-check, there aren't any additional considerations when creating a copy of a var, right?

[bob-carpenter]

On Aug 9, 2019, at 3:31 AM, seantalts ***@***.***> wrote: To migrate some comments from the thread on Discourse here - you said there that you are proposing the capture-by-reference ([&] in C++) semantics for closures in this design doc, and that you weren't sure why we'd go with by-value. I still think it'd be good to have some discussion of the tradeoffs there in the doc,

Sure. I literally couldn't imagine reasons we'd copy rather than use constant references.

so I'll try to list some reasons I can think of to prefer capture-by-value: • I think many people would find capture-by-value less confusing, but I'm not sure about that. I certainly find immutability in a threaded context most appropriate, and this is the wisdom I have seen presented, e.g. in books like Java Concurrency in Practice, in the design of Clojure and Scala, in the implementations of most (all?) new parallelism frameworks recently such as Spark, Hadoop, Actor frameworks for various languages, Go channels, etc.

I was assuming we'd pass by *constant* reference. I agree that adding mutable references to Stan will open a can of worms best left closed. I don't think you can just write [const &] in C++ to enforce that at the C++ level. I think all the rest of this is about immutability, not about reference vs. copy.

• Semantics for threading will be hellacious if we don't make all variables in concurrent contexts immutable. In other words, we're opening up users to surprising race conditions in any parallel or concurrent context if we pass by reference and allow folks to mutate those captured variables within the lambda functions (which is what [&] allows in C++, if that's the proposed semantic). Making the referenced captures immutable within lambdas solves this issue but only in the context of threaded map_rect; if we start having other forms of concurrent programs it can become very confusing again.

Agreed, let's make them immutable.

• Capture-by-reference semantics with mutable variables would add a source of non-determinism in a threaded context, making reproducibility harder.

Ditto.

• We can pass these closures to threaded functions, but not to MPI, which destroys one of the major use-cases for lambdas with closures (a more user-friendly map_rect version).

Is there a way to define the semantics for lambdas that lets them work neatly with MPI? I was trying to call out the difficulty with all of our higher order functions: integrate_ode: needs to know variables vs. data for Jacobians map_rect: needs to know variables from data for storing data with instances and for knowing which derivatives to calculate and presumably the algebraic solver will have similar issues.

• We lose other immutability guarantees about Stan programs that we can use for optimization. I claim we can have pass-by-value semantics with pass-by-reference performance in most cases by compiling down to closures with [&] wherever appropriate but creating copies if there's any chance the captured variable is mutated after the fact.

I was trying to suggest the former. Is there a case where you think allowing mutability of captured variables would be necessary? It doesn't add any functionality as you could always just assign the immutable variable to a local (though it's clunkier).

I'm a bit sleepy still - just to double-check, there aren't any additional considerations when creating a copy of a var, right?

Copies of var work just like the var---they're just pointers to implementations (vari*).

[wds15]

I was always assuming for the parallelization things that the lambdas would have immutable references to its environment (like we are settling on). Anything else would be a problem.

[seantalts]

Yeah, I didn't do a good job highlighting which issues occur due to lack of within-lambda immutability vs. lack of capture-by-value. If we assume within-lambda variable immutability (agreed it makes the most sense) we won't have issues with concurrency as long as it's confined to map_rect style functional concurrency. I prefer that style of concurrency and parallelism as far as user-defined constructs are concerned, but it does limit our ability to have the compiler automatically parallelize a Stan model.

I think the only thing we lose with capture-by-value would be some performance in the cases where a user writes a model that constructs a local variable (data and params are immutable already) that they want to use in a lambda function and then modify after the lambda has finished executing. But we would only lose that performance if they are actually using (accidentally or on purpose) the fact that the variable changes values in between calls to the lambda (or if the compiler can't detect that they aren't and is forced to conservatively make a copy). Do you know if anyone would want to program models like that on purpose? It seems like a confusing set of semantics to me.

On further thought I think the MPI issue could be solved with a very similar set of compiler analyses that we'd need to have cheap capture-by-value semantics.

So far capture-by-value seems to still be coming out ahead for me given that

1. the implementations for both seem equivalently complicated, assuming we need to support map_rect
2. capture-by-value still seems less confusing and like what people (non-R people?) would naturally expect
3. capture-by-value allows for easier whole-program parallelization efforts, I think

[seantalts]

I think either way we decide, it's good to add a section on capture-by-value vs. capture-by-reference. Would you like me to add that in?

[VMatthijs]

Does this mean you need a "sized function type" if you allow functions at the top level? What does that buy you over an unsized function type?

[VMatthijs]

Is it worth considering introducing a keyword for lambdas? For example \ or fun or function? I'm asking because the parentheses are already used for a lot of different things in Stan, as is, and probably will be used for more if we ever add tuples (unless we stick to structs instead). This is a genuine question. I'm not sure what I think, myself.

[VMatthijs]

Is it worth thinking about an implementation strategy of compiling away the higher-order functions using something like defunctionalization? That would keep the intermediate representation simpler and wouldn't add more overhead to maintaining the optimizations. On the other hand, it would require more fancy footwork and some thinking.

[VMatthijs]

Does this proposal immediately allow us to simplify the signatures of the ODE-solver and map_rect significantly, if we ask people to pass in a closure rather than a mere function?

[seantalts]

I think so, if we implement closures via something similar to lambda lifting but with automatic packing and unpacking of data and parameters.

[VMatthijs]

My feeling is that the copies would be so bad that we should not go with by value. It also feels more consistent with ordinary function argument which are passed by constant reference. Again, I feel like the enclosing variables should be passed by constant reference.

[seantalts]

We can avoid copies in cases where people don't mutate the variables in question in between calls to the lambda. I would say that is 99% of the use cases I've seen.

I don't think people can tell that functions are applied with constant reference vs by value in Stan currently.

[VMatthijs]

It would be very nice, but it'd be a lot of work to fix the whole language.

[seantalts]

We don't have to consider this as part of the closures proposal, I think :) But I'm curious (and maybe we should take this to a github issue) - what all would be required? I guess if C++ didn't support automated casting we'd have to generate code that did the casting?

[VMatthijs]

It shouldn't be too hard to catch the cases of dangling references and reject them at compile time.

[VMatthijs]

Thanks, @bob-carpenter ! This is great. Sorry to have taken so long to get to it. I've been swamped with work lately. Happy to discuss on hangouts at any point!

[bob-carpenter]

+ +## Variable declarations + +Variable declarations for functions follows the usual syntax, with a +sized type required for block variables, an unsized type for function Does this mean you need a "sized function type" if you allow functions at the top level? What does that buy you over an unsized function type?

That was totally unclear and I had to think about what I could've meant. I think I was just trying to stress that if you have something like an array of functions as a block variable, it needs to be sized real(real)[2] fs = { (real x).x^2, (real x).log(x) }; but it could be unsized as a function argument, as in real foo(real(real)[] fs, real x);

[bob-carpenter]

+## Lambda Expression Syntax + +Syntactically, a lambda expression is represented as a function argument +list followed by a function body. For example, in `(real u) { return +1 / (1 + exp(-u)); }`, the function argument list is `(real u)` and Is it worth considering introducing a keyword for lambdas? For example \ or fun or function?

I'm totall OK thinking about alternative syntax for types or lambdas. I was just following the way it's done in C++. Are you suggesting something like \real u { return u^2; } or function(real u) { return u^2; } instead of (real u) { return u^2; } ? I'm not so keen on backslash as it's a traditional escape.

I'm asking because the parentheses are already used for a lot of different things in Stan, as is, and probably will be used for more if we ever add tuples (unless we stick to structs instead). This is a genuine question. I'm not sure what I think, myself.

Are you imagining problems with parsing? - Bob

[bob-carpenter]

+## Implementation + +Lambdas can be mapped directly to C++ lambdas with default reference +closures. This will work because the scope of the compiled C++ is the +same as that of the Stan program. Is it worth thinking about an implementation strategy of compiling away the higher-order functions using something like defunctionalization?

Is that some way of unfolding a function like a macro? If not, is there a simple description or reference somewhere?

[bob-carpenter]

My feeling is that the copies would be so bad that we should not go with by reference.

I think the sign got garbled here.

It also feels more consistent with ordinary function argument which are passed by constant reference. Again, I feel like the enclosing variables should be passed by constant reference.

Yes, which would mean they couldn't be changed in the closure. The issue, as Sean pointed out, is that they might be changed out from under the closure. x = 2; int(int) f = (int y) { return x + y; } int z1 = f(2); // 4, as one would expect x = 3; int z2 = f(2); // 5 if pass by reference, 4 if pass by value

[bob-carpenter]

+4. Should full covariance and contravariance for function and array + types be required for assignment (including function calls)? We don't have to consider this as part of the closures proposal, I think :) But I'm curious (and maybe we should take this to a github issue) - what all would be required? I guess if C++ didn't support automated casting we'd have to generate code that did the casting?

Not for array types, but we should sort it out and do the right thing for function assignment. What would be required for array types is to basically allow assignment of int[] to double[], and make that automatic in function calls. I think that'd cover it.

[VMatthijs]

Yes, which would mean they couldn't be changed in the closure. The issue, as Sean pointed out, is that they might be changed out from under the closure. x = 2; int(int) f = (int y) { return x + y; } int z1 = f(2); // 4, as one would expect x = 3; int z2 = f(2); // 5 if pass by reference, 4 if pass by value

Agreed. They could be changed out from under the closure. But that's also true in Python. (Not that Python should be our model for good language design, but at least it's a language that tons of people seem to find intuitive to use.) Generally, I don't love mutable variables at all. But given that Stan is already a language with mutable variables and given that function arguments are currently passed by constant reference, I would expect closures also to work by constant reference as those variables are basically implicit function arguments. (At least, that's often how you implement them, using something like lambda-lifting or closure conversion.)

[VMatthijs]

I'm asking because the parentheses are already used for a lot of different things in Stan, as is, and probably will be used for more if we ever add tuples (unless we stick to structs instead). This is a genuine question. I'm not sure what I think, myself.
Are you imagining problems with parsing?

I don't think parsing would be a problem with the proposal as is, but it would become a problem if we ever want to move towards types being inferred more/being optional. I guess we could then let people write (auto x) {return x^2; } and it would still be OK.
Either way, I think we could get the formal parser to work with this. I was thinking more about keeping the language readable for people who aren't experts. In that respect, I'd like an informative keyword like function or fun or func. On the other hand, I can see that there might be an argument for brevity.

[VMatthijs]

Is that some way of unfolding a function like a macro? If not, is there a simple description or reference somewhere?

It's a whole program transformation for replacing all function types with a kind of variant type and replacing all higher-order functions with first order functions. There's a concise description with examples here: http://spivey.oriel.ox.ac.uk/corner/Defunctionalization_%28Programming_Languages%29 . I guess our problem is that this is only really easy to do in a language which already has support for variant types and tuples. You would compile away the higher-order types using the variant types and tuples and then I guess you can compile away those as well later (or give support for them in the runtime). Seeing that we don't have variant types and tuples to start with, perhaps this is not the most suitable implementation strategy for us. That's sad though as it would keep the MIR simpler. I would expect variants and tuples in an IR that you optimize on, but not closures and higher-order functions.

[bob-carpenter]

Yes, which would mean they couldn't be changed in the closure. The issue, as Sean pointed out, is that they might be changed out from under the closure. x = 2; int(int) f = (int y) { return x + y; } int z1 = f(2); // 4, as one would expect x = 3; int z2 = f(2); // 5 if pass by reference, 4 if pass by value Agreed. They could be changed out from under the closure. But that's also true in Python. (Not that Python should be our model for good language design, but at least it's a language that tons of people seem to find intuitive to use.)

I'm OK either way. I haven't been able to think of a reason we'd want to change a closed variable out from under a closure, but that doesn't mean there's not one.

Generally, I don't love mutable variables at all. But given that Stan is already a language with mutable variables and given that function arguments are currently passed by constant reference, I would expect closures also to work by constant reference as those variables are basically implicit function arguments. (At least, that's often how you implement them, using something like lambda-lifting or closure conversion.)

I don't know what those are. In C++, const or not and reference or not are just declarations on the member variable of the closure (the functor being created by the lambda), which get set when it's created.

[bob-carpenter]

We could add both variant types and tuples. I definitely want to add tuples and structs. I can't think of much use for variant types in a statistical model, but that doesn't mean we can't have them under the hood if they make code generation and optimization easier.

…

On Aug 30, 2019, at 5:49 AM, Matthijs Vákár ***@***.***> wrote: Is that some way of unfolding a function like a macro? If not, is there a simple description or reference somewhere? It's a whole program transformation for replacing all function types with a kind of variant type and replacing all higher-order functions with first order functions. There's a concise description with examples here: http://spivey.oriel.ox.ac.uk/corner/Defunctionalization_%28Programming_Languages%29 . I guess our problem is that this is only really easy to do in a language which already has support for variant types and tuples. You would compile away the higher-order types using the variant types and tuples and then I guess you can compile away those as well later (or give support for them in the runtime). Seeing that we don't have variant types and tuples to start with, perhaps this is not the most suitable implementation strategy for us. That's sad though as it would keep the MIR simpler. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[seantalts]

I feel pretty strongly that we shouldn't add more mutability unless there's a good reason for it because of its impacts on parallelism and the increased difficulty of reasoning about a program. Once you have closures with mutable state you effectively have an object system, which obviously some folks have historically thought was useful but is much more complex to reason about. Many of OO's early champions have backtracked away from objects with mutable state (see: the Java concurrency book).

I talked with a few users at Stan Con and all of them agreed that letting a variable change out from under a closure would be really confusing. That matches my feelings about it. I think folks also find Python's behavior here subtle and unintuitive.

I would expect closures also to work by constant reference as those variables are basically implicit function arguments. (At least, that's often how you implement them, using something like lambda-lifting or closure conversion.)

I would guess a very small percentage of our users are thinking about Stan closures being lambda-lifted or closure converted :P. I think we can easily create capture-by-value semantics that are free in the case where people don't mutate in between calls to a closure, so for me the question comes down to what kind of semantics you want users to reason about. Capturing by reference makes using closures for concurrent/parallel code really difficult both for the compiler writer and, in my opinion, the user reasoning about the language.

[seantalts]

I like the idea of adding a keyword for it. We could use fun so that when people want to google it, they get these results: https://www.google.com/search?q=stan+fun. I don't have a preference other than shorter is nicer.

[seantalts]

it's also worth noting that if we have closures as first class values (as they are in this proposal) then any implementation is required to create a data structure to hold the captured state anyway, and that could be equally by value or by reference. But yeah, I also just think giving users a consistent simple mental model for how closures work will work out better than assuming they know how they might be implemented in compiler textbooks. For either version, we can hopefully mitigate confusion by printing a warning if someone tries to mutate a captured variable in between calls to a closure.

[seantalts]

I wanna push back against requiring variables to be immutable before they are captured.

Is your preferred model that we throw an error if someone tries to change a variable that has been captured?

I'm not really sure how much more difficult it will make the implementation either. My criteria are still satisfied in both of those worlds - that we don't allow capturing variables that can then mutate in some other scope, and I think implementation will be easy enough for both so I'm happy either way.

Separately I do want to push back against thinking of a design doc as a perfect and final solution - it just has to represent an improvement and shouldn't serve to discourage future iterations and innovations, especially in a case like this where we're explicitly discussing leaving some work for the future.

[bbbales2]

Is your preferred model that we throw an error if someone tries to change a variable that has been captured?

Yeah, if that's what is convenient to do and there isn't some other flaw I'm not considering :P.

[seantalts]

It's an option with some convenience :p. What about if you weren't considering convenience?

…

On Mon, Sep 30, 2019 at 14:45 Ben Bales ***@***.***> wrote: Is your preferred model that we throw an error if someone tries to change a variable that has been captured? Yeah, if that's what is convenient to do and there isn't some other flaw I'm not considering :P. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#6?email_source=notifications&email_token=AAGET3DUJDRAO7YXBFRLYBDQMJCNRA5CNFSM4II6NOYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD76VW2A#issuecomment-536697704>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGET3FPKNBUFLYOHQFZOI3QMJCNRANCNFSM4II6NOYA> .

[bbbales2]

What about if you weren't considering convenience?

I'm only really considering convenience here :P. The two things I know I want this for are making ODEs really easy to program (no packing/unpacking) and then parallel evaluation of log densities with the new parallel stuff Sebastian has been talking about (in which case it solves the packing/unpacking problem for me).

[seantalts]

I'm only really considering convenience here :P. The two things I know I want this for are making ODEs really easy to program (no packing/unpacking) and then parallel evaluation of log densities with the new parallel stuff Sebastian has been talking about (in which case it solves the packing/unpacking problem for me).

Ohhh you meant Stan user programming convenience. That's the perspective I wanted - sorry, thought you were citing compiler implementation convenience.

[bbbales2]

Well here I am talking about compiler convenience: #6 (comment), but I answered your question in terms of user perspective cause I don't really know what's convenient or not language side :D.

[bob-carpenter]

On Sep 30, 2019, at 5:49 PM, seantalts ***@***.***> wrote: Ah yeah. I thought that's what your last commit had done, woops. Do you have a strong opinion on removing currying vs. no capture of mutables vs. full capture by value? If so I'm happy to go with it; if not my opinion is to go with the simplest one first (removing currying; only capture block variables) and show that off to users like Sebastian. This assumes it will be less work to go with the simplest and it will be easy to add additional capabilities later, which I think is true.

Sure. Let's start simple if that's easy enough with the parser. I'll revise the doc to reflect that. The bigger deal's going to be figuring out how we can rewrite map_rect and integrate_ode to use closures. I think that'll require more thought around how to specify the parameters if we capture them rather than packing/unpacking them. We might want to think about that. But then if this simple thing won't work for that, nothing will.

On Sep 30, 2019, at 6:51 PM, Ben Bales ***@***.***> wrote:

...

I don't want to lean on the model block structure given that that is already kind of awkward. For instance, if the thing I wanted captured by the closure was a function of parameters, it'd end up being a transformed parameter and end up in the output.

That's the current plan. If you need a function of parameters, it needs to be declared as a transformed parameter. That'll guarantee that variables that are captured never go out of scope (though copying them also gives us that guarantee).

This is from Bob: I updated to reflect a decision to only capture block variables and make captured variables immutable. I interpreted the last thing as somehow mark things that are captured as captured so that later assignments throw errors.

Yes. I'll clarify that was the intent.

Like this is allowed: x = 5 f = () { return x; } But this fails at compile time (x was once mutable but then became immutable once it was captured): x = 5; f = () { return x; } x = 2; I'm interpreting that differently from only allowing capture of things that are immutable.

I'll reword---the idea is rather that capturing a variable makes it immutable.

On Sep 30, 2019, at 8:05 PM, seantalts ***@***.***> wrote: For instance, if the thing I wanted captured by the closure was a function of parameters, it'd end up being a transformed parameter and end up in the output. Is this a big issue? Is it not worth putting out a simple version first with this constraint? It seems about as expensive as the copies it would require under the hood at first glance...

It's not the creation cost---it's the output that's annoying. We may be solving that problem separately in the interfaces going forward, but so far I think only RStan lets you filter what's output.

If so, or if Bob prefers, we can go with the "set anything captured to be immutable" version and just pay for copies in cases where we think the reference isn't going to be around long enough.

The idea was to do both---only allow capture of block variables, then make the block variable immutable after it's captured.

[seantalts]

I'm pretty sympathetic to Ben's programming convenience argument - I think that makes sense for a direction to go in. I'm happy with that (allowing folks to capture anything, if it's not a block variable we copy it, anything captured is immutable after capture) or with Bob's last proposal (only capturing block variables, immutable after capture). Bob I think it's up to you, this is your design doc :) We could even make it talk about a two phase implementation if you want. As the reviewer for this I guess I'm happy with all of the options we're currently talking about so just let me know when it's time to do the final check and merge.

Just to quickly throw out another option - we could change the definition of what is allowed to "only capturing variables that will be live as long as the closure is alive." This is somewhat confusing to explain properly but it would allow the above stuff (capturing transformed data and marking it immutable post-capture) as well as defining and capturing variables in the model block (and marking immutable post-capture). This is because all variables defined in the model block are local and don't leave the model block scope.

Maybe we can talk this out today at the meeting - I don't feel strongly about any of these options but would like to get one of these routes merged soon.

[bob-carpenter]

I created the branch and wrote the first draft, but the project is jointly owned by everyone and I don't get the final say. I specifically don't want to make the existence of a draft an argument against doing something alternative that's better.

All else being equal, I think it's going to be convenient to capture locals. I'm even OK letting undefined behavior rest if we can't catch captures that go out of scope. Anything involving scope is going to be complicated for most of our users, the majority of whom come from R which is the wild west with lexical dynamic scope and conditionals not opening scope blocks.

Capturing variables at the top-level model block won't go out of scope, but we can have local blocks in the model that will go out of scope.

model {
  real(real) f;
  if (theta > 0.5) {
    real j = 32;
    f = (real x) { return x + j; }
  } else {
    f = (real x) { return x^2; }
  }
  // j now out of scope, f broken

In general, we won't even be able to decide if f ever gets a definition if the block inside is complex enough.

[seantalts]

We can track what is variables are live at the time of the declaration and only allow those, right?

…

On Thu, Oct 3, 2019 at 09:49 Bob Carpenter ***@***.***> wrote: I created the branch and wrote the first draft, but the project is jointly owned by everyone and I don't get the final say. I specifically don't want to make the existence of a draft an argument against doing something alternative that's better. All else being equal, I think it's going to be convenient to capture locals. I'm even OK letting undefined behavior rest if we can't catch captures that go out of scope. Anything involving scope is going to be complicated for most of our users, the majority of whom come from R which is the wild west with lexical dynamic scope and conditionals not opening scope blocks. Capturing variables at the top-level model block won't go out of scope, but we can have local blocks in the model that will go out of scope. model { real(real) f; if (theta > 0.5) { real j = 32; f = (real x) { return x + j; } } else { f = (real x) { return x^2; } } // j now out of scope, f broken In general, we won't even be able to decide if f ever gets a definition if the block inside is complex enough. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#6?email_source=notifications&email_token=AAGET3G4YZOR4NJZV6NNWODQMXZ65A5CNFSM4II6NOYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAIINKA#issuecomment-537953960>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGET3DPSUO3BFUB25QB4E3QMXZ65ANCNFSM4II6NOYA> .

[bob-carpenter]

We can track what is variables are live at the time of the declaration and only allow those, right?

For capture by value, yes.

[seantalts]

@enetsee do you have any thoughts on this? Seems like we're between some rules intended to let folks capture things that we can nicely capture by reference versus just capturing anything they'd like and copying if necessary (and making anything captured immutable in the parent scope after capture).

[bob-carpenter]

I'm now worried that if we do have closures of locals, they'll still wind up being variable. Consider: real(int) f = (int x) { return (real y) { ... some expression involving closures of data ...}; }; Then we could have something like this: for (i in 1:10) map_rect(f(i), ...) // 10 different f(i) So one issue is how we're going to define whether something is constant and can thus be shipped off as data in a map-rect or integrate_ode call. Even worse, these conditionals can branch on parameter values.

…

On Oct 5, 2019, at 8:40 PM, seantalts ***@***.***> wrote: @enetsee do you have any thoughts on this? Seems like we're between some rules intended to let folks capture things that we can nicely capture by reference versus just capturing anything they'd like and copying if necessary (and making anything captured immutable in the parent scope after capture). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[seantalts]

I'm not sure I totally get the example, but it seems like maybe we should just go with capture-by-value and try to have the compiler be smart and save copies where it can. That seems to avoid a lot of these really tricky situations, right?

[bob-carpenter]

The example's important. Let's work backwards. For MPI, we need to have fixed data that we can ship to the core that's going to do the work on it. How do we maintain that with closures when the closures can close different pieces of data each trip through a program?

For example,

if (theta > 0)
  f = g;
else
  f = h;
map_rect(f, ...);

Now we don't know if g or h is going to be called in map_rect. This wasn't a problem before because there were no function variables. But even now (Stan 2.21), we could have something like:

for (i in 1:2)
  map_rect(f, ..., std::vector<int>{ i });

so that the integer data varies by loop instance. What happens now in this case?

[seantalts]

In the latter case I think it ships them at runtime. @wds15 can you confirm? I may be misremembering.

So for closures with values associated, in cases like that we'd need to essentially create first class closures that are runtime C++ objects with associated values and ship them over in the same way, right?

[seantalts]

@bob-carpenter once you decide which way you'd like to go we can merge this design doc and get started on the ragged one :)

[bob-carpenter]

What's left to decide? Now that I've finished std::complex for math, I plan to jump back into working on the language pretty much full time.

…

On Dec 16, 2019, at 1:26 PM, seantalts ***@***.***> wrote: @bob-carpenter once you decide which way you'd like to go we can merge this design doc and get started on the ragged one :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[wds15]

The design of map_rect requires you to assign a C++ type to every call of map_rect. That type is expected to label uniquely the data going into the function. So the call as outlined above would be illegal. map_rect will ship the data upon the first call and then not any more. Maybe we need additional safety checks in the parser to avoid such a call?

[bob-carpenter]

the call as outlined above

There were two---in a loop for repeated calls or in a conditional for unpredictable calls. Either way, do we need to ensure map_rect is only used from top-level scope (not nested in loops, conditionals, etc.) and that the function used is data? What else do we need to ensure.

Whatever we do going forward, if the above are illegal now, we need to make that clear to users. I hadn't realized they were illegal.

[seantalts]

There were two---in a loop for repeated calls or in a conditional for unpredictable calls. Either way, do we need to ensure map_rect is only used from top-level scope (not nested in loops, conditionals, etc.) and that the function used is data? What else do we need to ensure.

I think that's it. One day we could make the compiler smarter and generate the code to ship the data itself, similar to the way that @rok-cesnovar recently did the same thing for GPUs. But for now perhaps enforcing those conditions with nice error messages is a good thing to add, agreed.

What's left to decide?

What kind of capturing semantics you want to use - capture-by-value, capture-by-reference, or one of the few intermediate ones we discussed.

[wds15]

We should discuss this separately...but I wonder how @rok-cesnovar solved the issues with allowing shipping the correct way. We need to put some more restrictions on map_rect, but allowing it still to be called from within functions would be vital to keep my Stan programs alive at least.... but let's stop this side-track conversation here for now.

The closure stuff is something I am really looking forward to.

[bob-carpenter]

I thought the decision was to have pass-by-value semantics, but implement with pass-by-constant-reference wherever possible.

…

On Dec 17, 2019, at 11:47 AM, seantalts ***@***.***> wrote: There were two---in a loop for repeated calls or in a conditional for unpredictable calls. Either way, do we need to ensure map_rect is only used from top-level scope (not nested in loops, conditionals, etc.) and that the function used is data? What else do we need to ensure. I think that's it. One day we could make the compiler smarter and generate the code to ship the data itself, similar to the way that @rok-cesnovar recently did the same thing for GPUs. But for now perhaps enforcing those conditions with nice error messages is a good thing to add, agreed. What's left to decide? What kind of capturing semantics you want to use - capture-by-value, capture-by-reference, or one of the few intermediate ones we discussed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[seantalts]

Okay great! I hadn't realized we were good to go with this. I'll approve & merge. Thanks everyone!