New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add `let.Foo` sugar for continuation-passing-style #2140

Open
wants to merge 24 commits into
base: master
from

Conversation

Projects
None yet
@jaredly
Collaborator

jaredly commented Aug 14, 2018

This will allow us to get most of the benefits of lwt's ppx or ppx_let without baking in any specific implementation.

I also brought in some of the attribute handling changes in #1826 -- I didn't want to change too much, but it does make handling attributes much nicer.

thanks to @aantron for help with the implementation, and everyone who gave feedback & ideas on the concrete syntax.

Current (stripped-down) proposal

see bottom for original proposal

module Opt = {
  let let_ = Option.flatMap;
  let and_ = Option.pair;
};

let.Opt x = Some(10)
and.Opt y = Some(20)
and.Opt z = Some(30);
Some(x * y + z);

Potential future extensions

  • allow people to write lower-cased let.await and auto-capitalize -- part of the original proposal, but removed for simplicity of initial implementation -- supported in comment by @jordwalke
  • let._ that references a locally defined let_ instead of module-scoped: comment by @texastoland and comment by @hcarty
  • switch.Foo that would map to Foo.switch_(arg, fn): comment by @hcarty
  • relax the requirement of annotating ands: comment by @texastoland

References

  • the OCaml issue detailing a similar approach ocaml/ocaml#1947
  • the "percent less" PR that probably conflicts with this one #1735

original proposed syntax

let await = (value, continuation) => Js.Promise.then_(continuation, value);

let () = {
  let!await name = getName();
  let!await age = getAge(name);
  Promise.resolve({name, age})
}
@cristianoc

This comment has been minimized.

Show comment
Hide comment
@cristianoc
Contributor

cristianoc commented Aug 14, 2018

@aantron

This comment has been minimized.

Show comment
Hide comment
@aantron

aantron Aug 14, 2018

@jaredly There are a couple reasons to factor out the "monad product" that is done by and, so that it is its own function. Leaving the theoretical ones aside for now, here are the more practical ones:

One is motivated by your let!await example:

let () = {
  let!await name = getName();
  let!await age = getAge(name);
  {name, age}
}

It's probably just a minor oversight, but the second await is actually using Js.Promise.map, not then_, because it is returning unwrapped {name, age} (i.e., not Js.Promise.resolve({name, age})). So, I think the example would have to be:

let await = (value, continuation) => Js.Promise.then_(continuation, value);
let map = (value, continuation) => Js.Promise.map(continuation, value);

let () = {
  let!await name = getName();
  let!map age = getAge(name);
  {name, age}
}

If we want to be able to use and with both then_ and map, and don't take advantage of the "monad product," monad libs will have to define then_, then_2, then_3, map, map2, map3, etc. For some monad-like libs (applicatives), there might also be apply, so there will be apply2, apply3, etc. This causes code bloat and confusion, especially if the user has to ever manually hook all these up to the syntax, or if the user is writing some huge composition and the tuple gets too large.

Also, as can be seen from the option test case, in the PR as now, the user has to manually count how many ands they have, and pick the right suffix. If contributors want to insert or delete and extra and, they have to change the suffix on the first let:

let _ = { 
  let!opt x = Some(10); 
  let!opt2 a = Some(2)     /* <--- If I add an and, do I need to make this opt3? */
  and b = Some(5); 
  Some(a + x * b); 
}; 

That seems like something best left to the compiler.

aantron commented Aug 14, 2018

@jaredly There are a couple reasons to factor out the "monad product" that is done by and, so that it is its own function. Leaving the theoretical ones aside for now, here are the more practical ones:

One is motivated by your let!await example:

let () = {
  let!await name = getName();
  let!await age = getAge(name);
  {name, age}
}

It's probably just a minor oversight, but the second await is actually using Js.Promise.map, not then_, because it is returning unwrapped {name, age} (i.e., not Js.Promise.resolve({name, age})). So, I think the example would have to be:

let await = (value, continuation) => Js.Promise.then_(continuation, value);
let map = (value, continuation) => Js.Promise.map(continuation, value);

let () = {
  let!await name = getName();
  let!map age = getAge(name);
  {name, age}
}

If we want to be able to use and with both then_ and map, and don't take advantage of the "monad product," monad libs will have to define then_, then_2, then_3, map, map2, map3, etc. For some monad-like libs (applicatives), there might also be apply, so there will be apply2, apply3, etc. This causes code bloat and confusion, especially if the user has to ever manually hook all these up to the syntax, or if the user is writing some huge composition and the tuple gets too large.

Also, as can be seen from the option test case, in the PR as now, the user has to manually count how many ands they have, and pick the right suffix. If contributors want to insert or delete and extra and, they have to change the suffix on the first let:

let _ = { 
  let!opt x = Some(10); 
  let!opt2 a = Some(2)     /* <--- If I add an and, do I need to make this opt3? */
  and b = Some(5); 
  Some(a + x * b); 
}; 

That seems like something best left to the compiler.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 14, 2018

Collaborator
  1. Good call, I forgot to wrap in a resolve. let!map would also work

  2. about and

First off, I'm totally willing to explore a solution for and if we find that not having it is prohibitively annoying :) I think this solution is an excellent first step, and then we can later try to figure out how to get product right in the way that is least confusing.

Yes it's less "elegant" than a solution with the monad product, but I think it's also less confusing, and so I'd like to try this way first. We can then later add a and!prod syntax that's totally compatible with this current PR.

If we want to be able to use and with both then_ and map, and don't take advantage of the "monad product," monad libs will have to define then_, then_2, then_3, map, map2, map3, etc. This causes code bloat and confusion

It's boilerplate, but the best kind of boilerplate, in that it's trivial to get right, and probably won't need updating ever, because it lives in a library...

especially if the user has to ever manually hook all these up to the syntax

not sure what you mean by this. let!opt transforms to opt(x, y => z), so it's governed by normal scoping rules. I expect that people will do open Lwt.letSyntax at the top or something. You can also do let!Lwt.await x = y if you want.

or if the user is writing some huge composition and the tuple gets too large.

If they're writing some huge composition, you could build up the product yourself.

let a = b
and c = d
and e = f;
let!await ((a, c), e) = Lwt.await(Lwt.prod(Lwt.prod(a, c), e);

It doesn't seem to me like having massive compositions will be common enough for us to make things more complicated to support it, at least for this initial pass.

If I add an and, do I need to make this opt3?

Yup :) again, for this first pass I'm not too concerned about supporting support complex uses. We can add the and syntax afterwards if we find that it's super annoying not to have it.


All this to say:

  • I think this PR is quite straightforward for novices to understand (including potential error messages), and I'm super happy about that
  • adding and! syntax would make things more confusing
  • I'd like to see how things go without it, but I've also left the door open for implementing an and! syntax in a follow-up PR
Collaborator

jaredly commented Aug 14, 2018

  1. Good call, I forgot to wrap in a resolve. let!map would also work

  2. about and

First off, I'm totally willing to explore a solution for and if we find that not having it is prohibitively annoying :) I think this solution is an excellent first step, and then we can later try to figure out how to get product right in the way that is least confusing.

Yes it's less "elegant" than a solution with the monad product, but I think it's also less confusing, and so I'd like to try this way first. We can then later add a and!prod syntax that's totally compatible with this current PR.

If we want to be able to use and with both then_ and map, and don't take advantage of the "monad product," monad libs will have to define then_, then_2, then_3, map, map2, map3, etc. This causes code bloat and confusion

It's boilerplate, but the best kind of boilerplate, in that it's trivial to get right, and probably won't need updating ever, because it lives in a library...

especially if the user has to ever manually hook all these up to the syntax

not sure what you mean by this. let!opt transforms to opt(x, y => z), so it's governed by normal scoping rules. I expect that people will do open Lwt.letSyntax at the top or something. You can also do let!Lwt.await x = y if you want.

or if the user is writing some huge composition and the tuple gets too large.

If they're writing some huge composition, you could build up the product yourself.

let a = b
and c = d
and e = f;
let!await ((a, c), e) = Lwt.await(Lwt.prod(Lwt.prod(a, c), e);

It doesn't seem to me like having massive compositions will be common enough for us to make things more complicated to support it, at least for this initial pass.

If I add an and, do I need to make this opt3?

Yup :) again, for this first pass I'm not too concerned about supporting support complex uses. We can add the and syntax afterwards if we find that it's super annoying not to have it.


All this to say:

  • I think this PR is quite straightforward for novices to understand (including potential error messages), and I'm super happy about that
  • adding and! syntax would make things more confusing
  • I'd like to see how things go without it, but I've also left the door open for implementing an and! syntax in a follow-up PR
@aantron

This comment has been minimized.

Show comment
Hide comment
@aantron

aantron Aug 14, 2018

I'm all for merging all or parts of this soon, in the interest of not holding up evolution, but just saying :)

I do think the and part probably needs more exploration and experience to get right.

We could keep and without and!prod, by having

let await = {bind: (value, continuation) => ..., pair: (value, value') => ...}

and having let!await … and … take the right functions out of that record. Not saying that's a good idea, it's just a suggestion for keeping the syntax less visually complex (at a cost to library authors), because it doesn't require annotations on and. We would have some trouble switching to this later, if we think it's a good idea, if we commit this PR as is right now.

especially if the user has to ever manually hook all these up to the syntax

I mean when a library gets things slightly wrong, like when the author didn't realize they have a monad (or applicative, or whatever) and didn't make it compatible with the syntax, and the user has to write some glue code.

Another common case of this might be composed monads, like say there is an async monad that forgot to define an async+result monad, and now the user has to do it. This sort of thing comes up all the time in my OCaml experience.

Also, on the subject of boilerplate, since result has two binds (on the value and the error), and two maps, that is that much more boilerplate to define if we have to bake in the products. Maybe the failure case is not important enough for making it compatible with the syntax, I don't know.

If they're writing some huge composition, you could build up the product yourself.

Yes. But the point where you have to do this is not predictable to the user. And I can imagine large and-compositions would eventually happen with parsers and observables.

aantron commented Aug 14, 2018

I'm all for merging all or parts of this soon, in the interest of not holding up evolution, but just saying :)

I do think the and part probably needs more exploration and experience to get right.

We could keep and without and!prod, by having

let await = {bind: (value, continuation) => ..., pair: (value, value') => ...}

and having let!await … and … take the right functions out of that record. Not saying that's a good idea, it's just a suggestion for keeping the syntax less visually complex (at a cost to library authors), because it doesn't require annotations on and. We would have some trouble switching to this later, if we think it's a good idea, if we commit this PR as is right now.

especially if the user has to ever manually hook all these up to the syntax

I mean when a library gets things slightly wrong, like when the author didn't realize they have a monad (or applicative, or whatever) and didn't make it compatible with the syntax, and the user has to write some glue code.

Another common case of this might be composed monads, like say there is an async monad that forgot to define an async+result monad, and now the user has to do it. This sort of thing comes up all the time in my OCaml experience.

Also, on the subject of boilerplate, since result has two binds (on the value and the error), and two maps, that is that much more boilerplate to define if we have to bake in the products. Maybe the failure case is not important enough for making it compatible with the syntax, I don't know.

If they're writing some huge composition, you could build up the product yourself.

Yes. But the point where you have to do this is not predictable to the user. And I can imagine large and-compositions would eventually happen with parsers and observables.

@aantron

This comment has been minimized.

Show comment
Hide comment
@aantron

aantron Aug 14, 2018

Right now, I'm thinking we might want something like prod, then optional prod3, etc., because the extra products help with optimization. Not sure how to best implement that, though.

aantron commented Aug 14, 2018

Right now, I'm thinking we might want something like prod, then optional prod3, etc., because the extra products help with optimization. Not sure how to best implement that, though.

@aantron

This comment has been minimized.

Show comment
Hide comment
@aantron

aantron Aug 14, 2018

...and another option could be to define only bind/await/whatever, and have Reason translate let!await … and … always into a sequence of binds, unless the bind definition has some annotation attached, that gives hints about the names of the product functions. That completely removes the need for any boilerplate, both on the part of the library author, and on the part of the user adjusting opt2 to opt3 and so on. Boilerplate becomes optional, only for optimization.

aantron commented Aug 14, 2018

...and another option could be to define only bind/await/whatever, and have Reason translate let!await … and … always into a sequence of binds, unless the bind definition has some annotation attached, that gives hints about the names of the product functions. That completely removes the need for any boilerplate, both on the part of the library author, and on the part of the user adjusting opt2 to opt3 and so on. Boilerplate becomes optional, only for optimization.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 14, 2018

Collaborator

and having let!await … and … take the right functions out of that record.

Yeah, that does make the syntax simpler, but at the cost of making the transform more confusing. I like the simplicity of having the let!opt literally be a function reference that you can e.g. cmd-click, and error reporting is super straightforward.

large and-compositions would eventually happen with parsers and observables.

tbh if you're anding a ton of observables, you're in for a world of hurt 😂


but yeah, I think an and!foo syntax is a solid direction to go, and will be fully compatible with this PR.

Collaborator

jaredly commented Aug 14, 2018

and having let!await … and … take the right functions out of that record.

Yeah, that does make the syntax simpler, but at the cost of making the transform more confusing. I like the simplicity of having the let!opt literally be a function reference that you can e.g. cmd-click, and error reporting is super straightforward.

large and-compositions would eventually happen with parsers and observables.

tbh if you're anding a ton of observables, you're in for a world of hurt 😂


but yeah, I think an and!foo syntax is a solid direction to go, and will be fully compatible with this PR.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 14, 2018

Collaborator

...and another option could be to define only bind/await/whatever, and have Reason translate let!await … and … always into a sequence of binds

yuuup that gets into rather more magic than I'd want

Collaborator

jaredly commented Aug 14, 2018

...and another option could be to define only bind/await/whatever, and have Reason translate let!await … and … always into a sequence of binds

yuuup that gets into rather more magic than I'd want

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 14, 2018

Collaborator

I could imagine adding a "super-magic-monad" syntax later, but I want to keep this as straightforward and understandable to newcomers as possible, even though that means sacrificing some monady goodness.

Collaborator

jaredly commented Aug 14, 2018

I could imagine adding a "super-magic-monad" syntax later, but I want to keep this as straightforward and understandable to newcomers as possible, even though that means sacrificing some monady goodness.

@hcarty

This comment has been minimized.

Show comment
Hide comment
@hcarty

hcarty Aug 14, 2018

Contributor

Somewhat tangential to the main point of this PR - can we avoid overloading the meaning of ! as done here? It would be nice to either keep ! consistent as meaning "this is an explicit override/shadow/redefinition" or change the other uses of keyword! to use some other syntax. Like open^ Mod;, no seriously open Mod;, etc.

Contributor

hcarty commented Aug 14, 2018

Somewhat tangential to the main point of this PR - can we avoid overloading the meaning of ! as done here? It would be nice to either keep ! consistent as meaning "this is an explicit override/shadow/redefinition" or change the other uses of keyword! to use some other syntax. Like open^ Mod;, no seriously open Mod;, etc.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 14, 2018

Collaborator

I chose let! to mirror F#, and I think it looks nice. I'd be fine with no seriously open Mod :P

Collaborator

jaredly commented Aug 14, 2018

I chose let! to mirror F#, and I think it looks nice. I'd be fine with no seriously open Mod :P

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 14, 2018

Collaborator

or wait, what about like @override open Mod? That's both clear and doesn't require any parser changes

Collaborator

jaredly commented Aug 14, 2018

or wait, what about like @override open Mod? That's both clear and doesn't require any parser changes

@hcarty

This comment has been minimized.

Show comment
Hide comment
@hcarty

hcarty Aug 14, 2018

Contributor

@jaredly I like it - keyword! predates annotations and extension points on the OCaml side but the could work well here. There may need to be some special handling though since the compiler ignores unused/typo'd @annotations on its own.

It could be open%override, which would avoid the typo issue.

Contributor

hcarty commented Aug 14, 2018

@jaredly I like it - keyword! predates annotations and extension points on the OCaml side but the could work well here. There may need to be some special handling though since the compiler ignores unused/typo'd @annotations on its own.

It could be open%override, which would avoid the typo issue.

@jaredly jaredly referenced this pull request Aug 14, 2018

Open

Safe call operators ?. #2142

@IwanKaramazow
  • Can you add tests to make sure whitespace is interleaved correct?
let _ = {
  let!opt x = Some(10);

  let!opt2 a = Some(2)
  and b = Some(5);

  Some(a + x * b);
};
  • Can you add tests with extensions?
let!await%lwt
@aantron

This comment has been minimized.

Show comment
Hide comment
@aantron

aantron Aug 14, 2018

I want to keep this as straightforward and understandable to newcomers as possible

I think a far more straightforward thing to do is to give and no special treatment, and just expand let!foo … and … as if it was a series of lets.

Even simpler, we can reject and, and fail with an error message that links people to an issue for discussing how and should be handled.

aantron commented Aug 14, 2018

I want to keep this as straightforward and understandable to newcomers as possible

I think a far more straightforward thing to do is to give and no special treatment, and just expand let!foo … and … as if it was a series of lets.

Even simpler, we can reject and, and fail with an error message that links people to an issue for discussing how and should be handled.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 14, 2018

Collaborator

yeah, that makes sense

Collaborator

jaredly commented Aug 14, 2018

yeah, that makes sense

jaredly added some commits Aug 15, 2018

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 15, 2018

Collaborator

Ok, whitespace interleaving tested!

Collaborator

jaredly commented Aug 15, 2018

Ok, whitespace interleaving tested!

@jordwalke

This comment has been minimized.

Show comment
Hide comment
@jordwalke

jordwalke Aug 29, 2018

Member

The ppx_let example is using a lazy bind but that's not even the point of the Sketch, it's just showing how the types/transforms line up. How would exceptions propagate with backcalls? They must be turned into arguments passed to the callbacks, right?

Member

jordwalke commented Aug 29, 2018

The ppx_let example is using a lazy bind but that's not even the point of the Sketch, it's just showing how the types/transforms line up. How would exceptions propagate with backcalls? They must be turned into arguments passed to the callbacks, right?

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 29, 2018

but that's not even the point of the Sketch

Sorry I missed that I'm reading through it now.

How would exceptions propagate with backcalls?

In the case of bind (not the lazy version) it'd be 1:1.

let%bind file = Fs.readFile("path/to");
let%bind _ = Fs.writeFile("path/to");

vs

let file <= bind(Fs.readFile("path/to"));
let _ <= bind(Fs.writeFile("path/to"));

They must be turned into arguments passed to the callbacks, right?

Proper runtime exceptions I didn't consider until now. I suspect they'd be unaltered. If the user were to want better DX they could wrap their callback API in a higher order monadic function.

The difference seems to be callbacks are easier to make a mistake (as opposed to ppx_let) because you're not being forced down the narrow monadic road. I think the original proposal also allows arbitrary functions top though. I guess I should check if they enforce a signature. If not's it's the same thing.

Regardless syntactically my hesitations are:

  • Why is let.f not let? It's a let binding and a function call not a special kind of let.
  • Why is = a lie? In my own code let%map x = range(w) w is an array element not the array itself. Same with the original proposal.
  • Why is the let function annotated as if it were a member of the let module? Regardless whether tooling can navigate it feels incongruent.

texastoland commented Aug 29, 2018

but that's not even the point of the Sketch

Sorry I missed that I'm reading through it now.

How would exceptions propagate with backcalls?

In the case of bind (not the lazy version) it'd be 1:1.

let%bind file = Fs.readFile("path/to");
let%bind _ = Fs.writeFile("path/to");

vs

let file <= bind(Fs.readFile("path/to"));
let _ <= bind(Fs.writeFile("path/to"));

They must be turned into arguments passed to the callbacks, right?

Proper runtime exceptions I didn't consider until now. I suspect they'd be unaltered. If the user were to want better DX they could wrap their callback API in a higher order monadic function.

The difference seems to be callbacks are easier to make a mistake (as opposed to ppx_let) because you're not being forced down the narrow monadic road. I think the original proposal also allows arbitrary functions top though. I guess I should check if they enforce a signature. If not's it's the same thing.

Regardless syntactically my hesitations are:

  • Why is let.f not let? It's a let binding and a function call not a special kind of let.
  • Why is = a lie? In my own code let%map x = range(w) w is an array element not the array itself. Same with the original proposal.
  • Why is the let function annotated as if it were a member of the let module? Regardless whether tooling can navigate it feels incongruent.
@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 29, 2018

Sorry I replied before reading to the end. I just saw janestreet/ppx_let#8 too. Would that solved by an Error monad? Like you suggested in the ticket just add a function for binding on errors unless the the error monad does something fancier. I still think "backcalls" are doing the exact same thing as this proposal just explicitly. They can literally be rewritten 1:1.

texastoland commented Aug 29, 2018

Sorry I replied before reading to the end. I just saw janestreet/ppx_let#8 too. Would that solved by an Error monad? Like you suggested in the ticket just add a function for binding on errors unless the the error monad does something fancier. I still think "backcalls" are doing the exact same thing as this proposal just explicitly. They can literally be rewritten 1:1.

@jordwalke

This comment has been minimized.

Show comment
Hide comment
@jordwalke

jordwalke Aug 29, 2018

Member

Proper runtime exceptions I didn't consider until now. I suspect they'd be unaltered.

Unless you have some way to notify the callback function, the exceptions will be silently dropped, taking down the app with your callers having no way to catch them. That's why it's important to really explore this all the way with a real implementation.

Many of the syntactic objections you mentioned also are true of let%bind. I don't see how = is a lie here, either with ppx-let, or with this PR.

let.async result = readFile(path);

In that example result is equal to the result of reading the file.

Member

jordwalke commented Aug 29, 2018

Proper runtime exceptions I didn't consider until now. I suspect they'd be unaltered.

Unless you have some way to notify the callback function, the exceptions will be silently dropped, taking down the app with your callers having no way to catch them. That's why it's important to really explore this all the way with a real implementation.

Many of the syntactic objections you mentioned also are true of let%bind. I don't see how = is a lie here, either with ppx-let, or with this PR.

let.async result = readFile(path);

In that example result is equal to the result of reading the file.

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 29, 2018

Unless you have some way to notify the callback function, the exceptions will be silently dropped

Right but that's the way JS or OCaml works. The way to change that semantic is with a HOF that provides a monadic interface. That's why backcalls are more flexible besides simpler. You can pipe your function through a monadic HOF or not (your choice). This proposal just adds ceremony. In theory someone could provide a monadically broken function like in your Sketch and run into the same problem.

Many of the syntactic objections you mentioned also are true of let%bind.

That's my point. ppx_let is limited by PPX. You're not.

I don't see how = is a lie here

let.async result = readFile(path);

This illustrates the lie. It's not result. If async provides a monadic interface you should've written

let.async contents = readFile(path);

The result is handled by the bind. That's precisely why this is wrong. It's hard to see here because your monad has a 1:1 relationship with its result. It's much easier with an array and ppx_let

open PPXLetA;
let%map y = 0 --^ h;
let%map x = 0 --^ w;
y * w + x + 1;

x is most definitely not equal to the array produced by the range operator here. It's an element of that array. It's extracted from that array.

The thing both ppx_let and this proposal do is add ceremony to a desugaring that appears to do nothing more than unnest callbacks. Your rightful concern about error handling and ticket against ppx_let is easily resolved by calling a catch function on an Error monad. It's just a function. The ceremony of ppx_let provides principles but also restricts your use cases like generalized CPS. This proposal is arguably more limited in capability than ppx_let because any given module only exposes a single let_. What happens when I want to bind and map and catch in the same scope? That's precisely the use case that would enable list comprehensions for Reason.

As far as I can see backcalls are an un-magic desugaring for everything that ppx_let and this proposal want to do. ppx_lwt I don't know.

texastoland commented Aug 29, 2018

Unless you have some way to notify the callback function, the exceptions will be silently dropped

Right but that's the way JS or OCaml works. The way to change that semantic is with a HOF that provides a monadic interface. That's why backcalls are more flexible besides simpler. You can pipe your function through a monadic HOF or not (your choice). This proposal just adds ceremony. In theory someone could provide a monadically broken function like in your Sketch and run into the same problem.

Many of the syntactic objections you mentioned also are true of let%bind.

That's my point. ppx_let is limited by PPX. You're not.

I don't see how = is a lie here

let.async result = readFile(path);

This illustrates the lie. It's not result. If async provides a monadic interface you should've written

let.async contents = readFile(path);

The result is handled by the bind. That's precisely why this is wrong. It's hard to see here because your monad has a 1:1 relationship with its result. It's much easier with an array and ppx_let

open PPXLetA;
let%map y = 0 --^ h;
let%map x = 0 --^ w;
y * w + x + 1;

x is most definitely not equal to the array produced by the range operator here. It's an element of that array. It's extracted from that array.

The thing both ppx_let and this proposal do is add ceremony to a desugaring that appears to do nothing more than unnest callbacks. Your rightful concern about error handling and ticket against ppx_let is easily resolved by calling a catch function on an Error monad. It's just a function. The ceremony of ppx_let provides principles but also restricts your use cases like generalized CPS. This proposal is arguably more limited in capability than ppx_let because any given module only exposes a single let_. What happens when I want to bind and map and catch in the same scope? That's precisely the use case that would enable list comprehensions for Reason.

As far as I can see backcalls are an un-magic desugaring for everything that ppx_let and this proposal want to do. ppx_lwt I don't know.

@jordwalke

This comment has been minimized.

Show comment
Hide comment
@jordwalke

jordwalke Aug 29, 2018

Member

This illustrates the lie. It's not result. If async provides a monadic interface you should've written

Oh, I didn't mean this was a Result.result type. I just meant it's the conceptual return value of the function. Okay, now can you concede that it's not a lie? How does the following convey a lie?

let.async contents = readFile(path);

At the very least, it conveys no more of a lie than let%bind would. And I don't believe let%bind conveys a lie either.

Member

jordwalke commented Aug 29, 2018

This illustrates the lie. It's not result. If async provides a monadic interface you should've written

Oh, I didn't mean this was a Result.result type. I just meant it's the conceptual return value of the function. Okay, now can you concede that it's not a lie? How does the following convey a lie?

let.async contents = readFile(path);

At the very least, it conveys no more of a lie than let%bind would. And I don't believe let%bind conveys a lie either.

@jordwalke

This comment has been minimized.

Show comment
Hide comment
@jordwalke

jordwalke Aug 29, 2018

Member

This proposal is arguably more limited in capability than ppx_let because any given module only exposes a single let_. What happens when I want to bind and map and catch in the same scope? That's precisely the use case that would enable list comprehensions for Reason.

I think you didn't read the entire thread (and how could I expect anyone to given its length?). We since revised the PR after a few iterations to remove the specialness of let_. It is not the case that each module only expose a single let_. We revised it and my Sketch showed how a module can expose whatever it wants. This PR in its current form doesn't ever require you to define a module or any specially named member of let_. You can mix any number of monadic handlers and they're just plain identifiers that you specifically mention next to let. Like let.async and let.bind and let.errorAsync all just refer to async, bind, and errorAsync in scope. Given that, will you concede that this is not less flexible than ppx-let but more flexible?

Member

jordwalke commented Aug 29, 2018

This proposal is arguably more limited in capability than ppx_let because any given module only exposes a single let_. What happens when I want to bind and map and catch in the same scope? That's precisely the use case that would enable list comprehensions for Reason.

I think you didn't read the entire thread (and how could I expect anyone to given its length?). We since revised the PR after a few iterations to remove the specialness of let_. It is not the case that each module only expose a single let_. We revised it and my Sketch showed how a module can expose whatever it wants. This PR in its current form doesn't ever require you to define a module or any specially named member of let_. You can mix any number of monadic handlers and they're just plain identifiers that you specifically mention next to let. Like let.async and let.bind and let.errorAsync all just refer to async, bind, and errorAsync in scope. Given that, will you concede that this is not less flexible than ppx-let but more flexible?

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 29, 2018

How does the following convey a lie?

No as with my array example. If readFile were equal to contents it could never error. It's equal to some monadic result. Yes ppx_let is equationally wrong. The difference is they're limited where they can inject syntax.

Given that, will you concede that this is not less flexible than ppx-let but more flexible?

I read the entire thread but I clearly misunderstood. The desugaring of this proposal and ppx_let just unnests callbacks. AKAIK it literally does nothing else. In the case of ppx_let it roundabout calls a function hidden away in a module. It's the functions themselves that provide error handling etc. What is distasteful about just unnesting callbacks as backcalls, with normal functions, maintaining equational truth (since it's still a function just reversed)? It sounds like the only difference between this proposal and backcalls is syntax which was my original intuition?

texastoland commented Aug 29, 2018

How does the following convey a lie?

No as with my array example. If readFile were equal to contents it could never error. It's equal to some monadic result. Yes ppx_let is equationally wrong. The difference is they're limited where they can inject syntax.

Given that, will you concede that this is not less flexible than ppx-let but more flexible?

I read the entire thread but I clearly misunderstood. The desugaring of this proposal and ppx_let just unnests callbacks. AKAIK it literally does nothing else. In the case of ppx_let it roundabout calls a function hidden away in a module. It's the functions themselves that provide error handling etc. What is distasteful about just unnesting callbacks as backcalls, with normal functions, maintaining equational truth (since it's still a function just reversed)? It sounds like the only difference between this proposal and backcalls is syntax which was my original intuition?

@jordwalke

This comment has been minimized.

Show comment
Hide comment
@jordwalke

jordwalke Aug 29, 2018

Member

I think the several iterations that this PR underwent are adding far too much perceived complexity that isn't there. Right now, it's as simple as the following callback rewriting:

let.identifier x = expr;

identifier(expr, x => {

}); 

You can use any identifier or even long identifier Mod.ident. No modules require let_, or must have exactly 1/2 members. There is ultimate flexibility in naming and you can mix/match the identifiers in any sequence of bindings so that you can opt into error handling. As you wished: "it's just a function".

Why is there even an identifier rewriting? So that you can control the error handling more granularly using plain functions.

Member

jordwalke commented Aug 29, 2018

I think the several iterations that this PR underwent are adding far too much perceived complexity that isn't there. Right now, it's as simple as the following callback rewriting:

let.identifier x = expr;

identifier(expr, x => {

}); 

You can use any identifier or even long identifier Mod.ident. No modules require let_, or must have exactly 1/2 members. There is ultimate flexibility in naming and you can mix/match the identifiers in any sequence of bindings so that you can opt into error handling. As you wished: "it's just a function".

Why is there even an identifier rewriting? So that you can control the error handling more granularly using plain functions.

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 29, 2018

It sounds like the only difference between this proposal and backcalls is syntax which was my original intuition?

In that case I could rewrite

let.async contents = readFile(path);

as my original example

let.readFile result = path;

to opt out of the monadic interface then it's very clear why it's syntactically confusing relative to either of the following

let contents <= async(readFile(path));
let result <= readFile(path);

texastoland commented Aug 29, 2018

It sounds like the only difference between this proposal and backcalls is syntax which was my original intuition?

In that case I could rewrite

let.async contents = readFile(path);

as my original example

let.readFile result = path;

to opt out of the monadic interface then it's very clear why it's syntactically confusing relative to either of the following

let contents <= async(readFile(path));
let result <= readFile(path);
@jordwalke

This comment has been minimized.

Show comment
Hide comment
@jordwalke

jordwalke Aug 29, 2018

Member

What is distasteful about just unnesting callbacks as backcalls, with normal functions, maintaining equational truth (since it's still a function just reversed)?

If you have a full proposal that shows you would handle errors, and allow catching them, then please post that full proposal. I thought that a very simple backcall style transform would work, then I hit a wall when I wanted natural error handling when actually trying to build it. You might have more experience with that style of async API so please feel free to demonstrate a full example of handling errors (catching and propagating them without much nesting). Can you show what you'd write, and then what it gets transformed to?
(Maybe post it in Discord - this thread is already unwieldy).

Member

jordwalke commented Aug 29, 2018

What is distasteful about just unnesting callbacks as backcalls, with normal functions, maintaining equational truth (since it's still a function just reversed)?

If you have a full proposal that shows you would handle errors, and allow catching them, then please post that full proposal. I thought that a very simple backcall style transform would work, then I hit a wall when I wanted natural error handling when actually trying to build it. You might have more experience with that style of async API so please feel free to demonstrate a full example of handling errors (catching and propagating them without much nesting). Can you show what you'd write, and then what it gets transformed to?
(Maybe post it in Discord - this thread is already unwieldy).

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 29, 2018

You might have more experience with that style of async API

I'm positive I don't.

feel free to demonstrate a full example of handling errors

As mentioned it's easily implementable with the Error monad. The only reason it doesn't work with ppx_let is because they don't expose a bind function on errors. I'll fork your Sketch tomorrow but it's just functions not desugaring. My suggestion was just to change

let.f x = m
...

to

let x <= f(m)
...

because it's equationally true and more similar to what it desugars to

f(m, x => ...)

The clearest illustration of the problem is

/* x nor map(x) is an array */
let.map x = [||];

vs

/* x got mapped out of an array */
let x <= map([||]);

texastoland commented Aug 29, 2018

You might have more experience with that style of async API

I'm positive I don't.

feel free to demonstrate a full example of handling errors

As mentioned it's easily implementable with the Error monad. The only reason it doesn't work with ppx_let is because they don't expose a bind function on errors. I'll fork your Sketch tomorrow but it's just functions not desugaring. My suggestion was just to change

let.f x = m
...

to

let x <= f(m)
...

because it's equationally true and more similar to what it desugars to

f(m, x => ...)

The clearest illustration of the problem is

/* x nor map(x) is an array */
let.map x = [||];

vs

/* x got mapped out of an array */
let x <= map([||]);
@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 29, 2018

Collaborator

Hahah wow lots happened yesterday! I just want to interject that a huge goal of this PR is not just minimizing nesting (although that is also desired), but

  • to do so in a way that we can teach the 80% case without having to talk about category theory
  • is syntactically "loud" and "scannable" enough that you can see what's going in a codebase in terms of "what parts of this code have customised runtime semantics"

@texastoland it sounds like your proposal is basically the same as let->, but with a minor syntactic tweak. Is that how you see it?

Collaborator

jaredly commented Aug 29, 2018

Hahah wow lots happened yesterday! I just want to interject that a huge goal of this PR is not just minimizing nesting (although that is also desired), but

  • to do so in a way that we can teach the 80% case without having to talk about category theory
  • is syntactically "loud" and "scannable" enough that you can see what's going in a codebase in terms of "what parts of this code have customised runtime semantics"

@texastoland it sounds like your proposal is basically the same as let->, but with a minor syntactic tweak. Is that how you see it?

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 29, 2018

@jaredly my apology for the barrage. Yes that's what I originally understood and back to understanding. I'll summarize.

My concern (#2140 (comment)) is = is confusing because it's literally unequal e.g. map (#2140 (comment)) or ordinary async-less callbacks (#2140 (comment)). A more minor complaint is let.f looks like a module of let and not let at all. I think ppx_let is a really neat solution but they couldn't do anything about syntax.

Your idea is a really neat generalization of ppx_let. Without and it's basically LiveScript (FP CoffeeScript fork) backcalls (#2140 (comment)). Backcalls are a more literal desugaring but otherwise the same as let->. <= would parse unambiguously next to let but would look odd because it's ligature is less-than-or-equals (Haskell etc. use skinny arrows).

It looks like the implementation still includes the let_ and and_ interface? I think just let would be problematic for hosting multiple functions e.g. bind and map.

@jordwalke's separate concern is how to handle async exceptions. I won't have time to write it up formally today but after a good sleep I agree. The desguaring could catch unhandled exceptions and re-raise them one level up. It'd be even nicer if every computation were stored in a result (or Error monad lite) with a function to handle errors inline. That's high level but I haven't tried anything real yet.

texastoland commented Aug 29, 2018

@jaredly my apology for the barrage. Yes that's what I originally understood and back to understanding. I'll summarize.

My concern (#2140 (comment)) is = is confusing because it's literally unequal e.g. map (#2140 (comment)) or ordinary async-less callbacks (#2140 (comment)). A more minor complaint is let.f looks like a module of let and not let at all. I think ppx_let is a really neat solution but they couldn't do anything about syntax.

Your idea is a really neat generalization of ppx_let. Without and it's basically LiveScript (FP CoffeeScript fork) backcalls (#2140 (comment)). Backcalls are a more literal desugaring but otherwise the same as let->. <= would parse unambiguously next to let but would look odd because it's ligature is less-than-or-equals (Haskell etc. use skinny arrows).

It looks like the implementation still includes the let_ and and_ interface? I think just let would be problematic for hosting multiple functions e.g. bind and map.

@jordwalke's separate concern is how to handle async exceptions. I won't have time to write it up formally today but after a good sleep I agree. The desguaring could catch unhandled exceptions and re-raise them one level up. It'd be even nicer if every computation were stored in a result (or Error monad lite) with a function to handle errors inline. That's high level but I haven't tried anything real yet.

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 29, 2018

I wanted to point out another use case for this proposal. PureScript and Scala use do notation for list comprehensions. The PureScript example could be written using operators

/* example */
let factors = n => {
  0 -- n >>= a =>      /* for a in 0 to our number         */
  0 -- a >>= b =>      /* for b in 0 to a                  */
  a * b == n >>? () => /* if the product equals our number */
  return((a, b))       /* make a pair                      */
};

/* [[6,4],[8,3],[12,2],[24,1]] */
24 |> factors |> Js.log

Using ppx_let

let factors = n => {
  let%bind a = 0 -- n
  let%bind b = 0 -- a
  let%bind () = guard(a * b == n)
  (a, b)
};

Using this proposal (I think)

let yield = return
let for_ = bind
let if_ = (>>?)

let factors = n => {      /* alternatively:           */
  let.for_ a = 0 -- n     /* let a <= for_(0 -- n)    */
  let.for_ b = 0 -- a     /* let b <= for_(0 -- a)    */
  let.if_ () = a * b == n /* let _ <= if_(a * b == n) */
  yield((a, b))
};

Given

open Belt.Array

/* ppx_let */
let return = x => [|x|]
let bind = (x, f) =>
  reduce(x, [||], (acc, a) => concat(acc, f(a)))

/* MonadZero */
let guard = t => t ? [|()|] : [||]

/* infix */
let (>>=) = bind
let (>>?) = t => bind(guard(t))
let (--) = range

Playground

texastoland commented Aug 29, 2018

I wanted to point out another use case for this proposal. PureScript and Scala use do notation for list comprehensions. The PureScript example could be written using operators

/* example */
let factors = n => {
  0 -- n >>= a =>      /* for a in 0 to our number         */
  0 -- a >>= b =>      /* for b in 0 to a                  */
  a * b == n >>? () => /* if the product equals our number */
  return((a, b))       /* make a pair                      */
};

/* [[6,4],[8,3],[12,2],[24,1]] */
24 |> factors |> Js.log

Using ppx_let

let factors = n => {
  let%bind a = 0 -- n
  let%bind b = 0 -- a
  let%bind () = guard(a * b == n)
  (a, b)
};

Using this proposal (I think)

let yield = return
let for_ = bind
let if_ = (>>?)

let factors = n => {      /* alternatively:           */
  let.for_ a = 0 -- n     /* let a <= for_(0 -- n)    */
  let.for_ b = 0 -- a     /* let b <= for_(0 -- a)    */
  let.if_ () = a * b == n /* let _ <= if_(a * b == n) */
  yield((a, b))
};

Given

open Belt.Array

/* ppx_let */
let return = x => [|x|]
let bind = (x, f) =>
  reduce(x, [||], (acc, a) => concat(acc, f(a)))

/* MonadZero */
let guard = t => t ? [|()|] : [||]

/* infix */
let (>>=) = bind
let (>>?) = t => bind(guard(t))
let (--) = range

Playground

@jordwalke

This comment has been minimized.

Show comment
Hide comment
@jordwalke

jordwalke Aug 30, 2018

Member

@texastoland : You mentioned you haven't thought too much about the error handling. Aside from a nicer (imho) syntax than ppx-let, that error handling is the main reason I see this PR being better than ppx-let. If you have a proposal, please include how error propagation would work. Without it, a feature request is not really complete. The real problems come up when when handling errors and/or exceptions and it's that use case that I think this PR's justification becomes apparent.

Member

jordwalke commented Aug 30, 2018

@texastoland : You mentioned you haven't thought too much about the error handling. Aside from a nicer (imho) syntax than ppx-let, that error handling is the main reason I see this PR being better than ppx-let. If you have a proposal, please include how error propagation would work. Without it, a feature request is not really complete. The real problems come up when when handling errors and/or exceptions and it's that use case that I think this PR's justification becomes apparent.

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 30, 2018

that error handling is the main reason I see this PR being better than ppx-let

I don't see that being done here?

let continuation =
Exp.fun_
~loc:combinator.loc Nolabel None nested_pair_pattern rest_of_code
in
Exp.apply
~attrs:[simple_ghost_refmt_text_attr Reason_attrs.letCombinator]
~loc:let_bindings.lbs_loc
func
[(Nolabel, pairing_expression); (Nolabel, continuation)]

I think it's a separate issue from the syntax being unintuitive (more importantly unequational) though.

The desguaring could catch unhandled exceptions and re-raise them one level up. It'd be even nicer if every computation were stored in a result (or Error monad lite) with a function to handle errors inline.

I can write this up pretty trivially but I'd like to understand the status quo.

texastoland commented Aug 30, 2018

that error handling is the main reason I see this PR being better than ppx-let

I don't see that being done here?

let continuation =
Exp.fun_
~loc:combinator.loc Nolabel None nested_pair_pattern rest_of_code
in
Exp.apply
~attrs:[simple_ghost_refmt_text_attr Reason_attrs.letCombinator]
~loc:let_bindings.lbs_loc
func
[(Nolabel, pairing_expression); (Nolabel, continuation)]

I think it's a separate issue from the syntax being unintuitive (more importantly unequational) though.

The desguaring could catch unhandled exceptions and re-raise them one level up. It'd be even nicer if every computation were stored in a result (or Error monad lite) with a function to handle errors inline.

I can write this up pretty trivially but I'd like to understand the status quo.

@jordwalke

This comment has been minimized.

Show comment
Hide comment
@jordwalke

jordwalke Aug 30, 2018

Member

I don't see that being done here?

Did you read the entirety of my Sketch? (It included an example in the linked Sketch that explains why this PR is helpful).

Member

jordwalke commented Aug 30, 2018

I don't see that being done here?

Did you read the entirety of my Sketch? (It included an example in the linked Sketch that explains why this PR is helpful).

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Aug 30, 2018

Ah yes you articulated the problem awesome! I meant the current proposal doesn't address it either unless I misunderstood both the thread and this PR's changes. Regardless I agree and I'll publish a link concretely how I see it being desugared separate from any syntax concern.

texastoland commented Aug 30, 2018

Ah yes you articulated the problem awesome! I meant the current proposal doesn't address it either unless I misunderstood both the thread and this PR's changes. Regardless I agree and I'll publish a link concretely how I see it being desugared separate from any syntax concern.

@SanderSpies

This comment has been minimized.

Show comment
Hide comment
@SanderSpies

SanderSpies Aug 30, 2018

Contributor

Hopefully not adding too much noise to this discussion, but I thought this alternative syntax might be interesting:

let (x) = identifier(expr, $x);

The idea here is to look somewhat similar to a regexp search and replace.

It also allows you to put the callback argument in different places:

let (x) = identifier($x, expr); 

Edit: this syntax won't work as is - but I believe it's still an interesting direction to explore.

Contributor

SanderSpies commented Aug 30, 2018

Hopefully not adding too much noise to this discussion, but I thought this alternative syntax might be interesting:

let (x) = identifier(expr, $x);

The idea here is to look somewhat similar to a regexp search and replace.

It also allows you to put the callback argument in different places:

let (x) = identifier($x, expr); 

Edit: this syntax won't work as is - but I believe it's still an interesting direction to explore.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Aug 30, 2018

Collaborator

@SanderSpies that looks similar to @chenglou's proposal (comment) -- which was

let-> x = identifier(_, expr);

For me, the ability to put the callback argument in different places substantially hurts "grokkability", because now I have to hunt for where the callback function is being inserted, which in a function that accepts multiple handlers (e.g. an "on success" callback and an "on error" callback) could drastically change the execution semantics of the following code.

Your example also drops the let-> marker, which means, for me, it will be nearly impossible to look at a bunch of code and determine "what are the let bindings that will have very different execution semantics"

The code sample (taken from reason-language-server, which there is using a variant of ppx_let) that I'm using to evaluate different syntax proposals is this: (and I'm having to guess a little bit what you mean by $x)

let (uri, position) = try(Protocol.rPositionParams(params), $(uri, position));
let (text, verison, isClean) =
  try(maybeHash(state.documentText, uri) |> orError("No document text found"), $(text, verison, isClean));
let package = try(State.getPackage(uri, state), $package);
let offset =
  try(PartialParser.positionToOffset(text, position) |> orError("invalid offset"), $offset);
let {extra} = try(State.getDefinitionData(uri, state, ~package), ${extra});
let position = Utils.cmtLocFromVscode(position);

{
  let (commas, labelsUsed, lident, i) =
    opt(PartialParser.findFunctionCall(text, offset - 1), $(commas, labelsUsed, lident, i));
  let lastPos = i + String.length(lident) - 1;
  let pos =
    opt(PartialParser.offsetToPosition(text, lastPos) |?>> Utils.cmtLocFromVscode, $pos);
  let (_, loc) = opt(References.locForPos(~extra, pos), $(_, loc));
  let typ =
    opt(switch (loc) {
    | Typed(t, _) => Some(t)
    | _ => None
    }, $typ);
  let rec loop = t => ...
  let (args, rest) = loop(typ);
  let args = opt(args == [] ? None : Some(args), $args);
  Some(Ok(...));
} |? Ok((state, Json.Null));

How long does it take to spot all the $s? 😫

And here's that code in the let.foo style

let.try (uri, position) = Protocol.rPositionParams(params);
let.try (text, verison, isClean) =
  maybeHash(state.documentText, uri) |> orError("No document text found");
let.try package = State.getPackage(uri, state);
let.try offset =
  PartialParser.positionToOffset(text, position) |> orError("invalid offset");
let.try {extra} = State.getDefinitionData(uri, state, ~package);
let position = Utils.cmtLocFromVscode(position);

{
  let.opt (commas, labelsUsed, lident, i) =
    PartialParser.findFunctionCall(text, offset - 1);
  let lastPos = i + String.length(lident) - 1;
  let.opt pos =
    PartialParser.offsetToPosition(text, lastPos) |?>> Utils.cmtLocFromVscode;
  let.opt (_, loc) = References.locForPos(~extra, pos);
  let.opt typ =
    switch (loc) {
    | Typed(t, _) => Some(t)
    | _ => None
    };
  let rec loop = t => ...;
  let (args, rest) = loop(typ);
  let.opt args = args == [] ? None : Some(args);
  Some(Ok(...));
}
|? Ok((state, Json.Null));
Collaborator

jaredly commented Aug 30, 2018

@SanderSpies that looks similar to @chenglou's proposal (comment) -- which was

let-> x = identifier(_, expr);

For me, the ability to put the callback argument in different places substantially hurts "grokkability", because now I have to hunt for where the callback function is being inserted, which in a function that accepts multiple handlers (e.g. an "on success" callback and an "on error" callback) could drastically change the execution semantics of the following code.

Your example also drops the let-> marker, which means, for me, it will be nearly impossible to look at a bunch of code and determine "what are the let bindings that will have very different execution semantics"

The code sample (taken from reason-language-server, which there is using a variant of ppx_let) that I'm using to evaluate different syntax proposals is this: (and I'm having to guess a little bit what you mean by $x)

let (uri, position) = try(Protocol.rPositionParams(params), $(uri, position));
let (text, verison, isClean) =
  try(maybeHash(state.documentText, uri) |> orError("No document text found"), $(text, verison, isClean));
let package = try(State.getPackage(uri, state), $package);
let offset =
  try(PartialParser.positionToOffset(text, position) |> orError("invalid offset"), $offset);
let {extra} = try(State.getDefinitionData(uri, state, ~package), ${extra});
let position = Utils.cmtLocFromVscode(position);

{
  let (commas, labelsUsed, lident, i) =
    opt(PartialParser.findFunctionCall(text, offset - 1), $(commas, labelsUsed, lident, i));
  let lastPos = i + String.length(lident) - 1;
  let pos =
    opt(PartialParser.offsetToPosition(text, lastPos) |?>> Utils.cmtLocFromVscode, $pos);
  let (_, loc) = opt(References.locForPos(~extra, pos), $(_, loc));
  let typ =
    opt(switch (loc) {
    | Typed(t, _) => Some(t)
    | _ => None
    }, $typ);
  let rec loop = t => ...
  let (args, rest) = loop(typ);
  let args = opt(args == [] ? None : Some(args), $args);
  Some(Ok(...));
} |? Ok((state, Json.Null));

How long does it take to spot all the $s? 😫

And here's that code in the let.foo style

let.try (uri, position) = Protocol.rPositionParams(params);
let.try (text, verison, isClean) =
  maybeHash(state.documentText, uri) |> orError("No document text found");
let.try package = State.getPackage(uri, state);
let.try offset =
  PartialParser.positionToOffset(text, position) |> orError("invalid offset");
let.try {extra} = State.getDefinitionData(uri, state, ~package);
let position = Utils.cmtLocFromVscode(position);

{
  let.opt (commas, labelsUsed, lident, i) =
    PartialParser.findFunctionCall(text, offset - 1);
  let lastPos = i + String.length(lident) - 1;
  let.opt pos =
    PartialParser.offsetToPosition(text, lastPos) |?>> Utils.cmtLocFromVscode;
  let.opt (_, loc) = References.locForPos(~extra, pos);
  let.opt typ =
    switch (loc) {
    | Typed(t, _) => Some(t)
    | _ => None
    };
  let rec loop = t => ...;
  let (args, rest) = loop(typ);
  let.opt args = args == [] ? None : Some(args);
  Some(Ok(...));
}
|? Ok((state, Json.Null));
@SanderSpies

This comment has been minimized.

Show comment
Hide comment
@SanderSpies

SanderSpies Aug 31, 2018

Contributor

@jaredly I guess you are right, in this example it's somewhat hard to spot the replace locations.

Apologies for adding noise to the conversation, as it seems that Cheng's syntax version was in the same spirit and even better. The underscore has a different meaning for me, so that might be why I missed it.

Contributor

SanderSpies commented Aug 31, 2018

@jaredly I guess you are right, in this example it's somewhat hard to spot the replace locations.

Apologies for adding noise to the conversation, as it seems that Cheng's syntax version was in the same spirit and even better. The underscore has a different meaning for me, so that might be why I missed it.

@andreypopp

This comment has been minimized.

Show comment
Hide comment
@andreypopp

andreypopp Sep 3, 2018

Contributor

Tangentially related, with ppx_let I often find myself writing a series of let%bind () = ..; bindings:

let%bind () = Fs.unlink(p);
let%bind () = Fs.symlink(~target, p);

With the current proposal it would look like:

let.async () = Fs.unlink(p);
let.async () = Fs.symlink(~target, p);

Which looks great! But maybe there's a room for a special syntax:

async Fs.unlink(p);
async Fs.symlink(~target, p);

or something like that, which looks less noisy.

Contributor

andreypopp commented Sep 3, 2018

Tangentially related, with ppx_let I often find myself writing a series of let%bind () = ..; bindings:

let%bind () = Fs.unlink(p);
let%bind () = Fs.symlink(~target, p);

With the current proposal it would look like:

let.async () = Fs.unlink(p);
let.async () = Fs.symlink(~target, p);

Which looks great! But maybe there's a room for a special syntax:

async Fs.unlink(p);
async Fs.symlink(~target, p);

or something like that, which looks less noisy.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Sep 4, 2018

Collaborator

So, thinking about this some more, and talking to @texastoland, I'm now thinking about it this way: If we want to do "plain functions", then the name of the function needs to go to the right of the =. If we do a module, then it can go next to the let.
e.g. either we do something like let-> x = async(thing), or we do let.Async x = thing where Async is a module with a let_ (we could also auto-capitalize, allowing let.async, as discussed).

Having "plain functions" where the name is next to the let will inevitably be very confusing, as users will use all kinds of "callback functions" -- not just ones that "make sense" as monad transformers. e.g. let.readFile contents = "data.json" would be perfectly valid under the "plain functions let.foo" proposal, and is quite confusing to parse.

Collaborator

jaredly commented Sep 4, 2018

So, thinking about this some more, and talking to @texastoland, I'm now thinking about it this way: If we want to do "plain functions", then the name of the function needs to go to the right of the =. If we do a module, then it can go next to the let.
e.g. either we do something like let-> x = async(thing), or we do let.Async x = thing where Async is a module with a let_ (we could also auto-capitalize, allowing let.async, as discussed).

Having "plain functions" where the name is next to the let will inevitably be very confusing, as users will use all kinds of "callback functions" -- not just ones that "make sense" as monad transformers. e.g. let.readFile contents = "data.json" would be perfectly valid under the "plain functions let.foo" proposal, and is quite confusing to parse.

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Sep 4, 2018

let.Async x = thing where Async is a module with a let_

My idea was let.Async x <- await(thing) where Async is a module with an await. This is the least magic (<- could be replaced by another symbol except = or a right arrow). await(thing) is a typical function call (as opposed to the function and argument on opposite sides of =), there's no = to lie about equality as I've demonstrated in numerous examples, and @jaredly pointed out to me let.Async syntax resembles existing record.Module.field.

A friendlier option with more precedent would be to enclose the scope in a keyword (Haskell do, Scala for, F# named after module) and get rid of PPX-like lets altogether. I've already outlined a formal RFC (including error handling) with all my concerns and suggestions from this PR but it'll take a little bit to write up completely with examples. Tl;dr I like this PR's implementation but the syntax and modular interface are problematic for all use cases.

texastoland commented Sep 4, 2018

let.Async x = thing where Async is a module with a let_

My idea was let.Async x <- await(thing) where Async is a module with an await. This is the least magic (<- could be replaced by another symbol except = or a right arrow). await(thing) is a typical function call (as opposed to the function and argument on opposite sides of =), there's no = to lie about equality as I've demonstrated in numerous examples, and @jaredly pointed out to me let.Async syntax resembles existing record.Module.field.

A friendlier option with more precedent would be to enclose the scope in a keyword (Haskell do, Scala for, F# named after module) and get rid of PPX-like lets altogether. I've already outlined a formal RFC (including error handling) with all my concerns and suggestions from this PR but it'll take a little bit to write up completely with examples. Tl;dr I like this PR's implementation but the syntax and modular interface are problematic for all use cases.

@andreypopp

This comment has been minimized.

Show comment
Hide comment
@andreypopp

andreypopp Sep 4, 2018

Contributor

There's yet another advantage of using a module to represent semantics of CPS bindings.

Consider this piece of code with ppx_let:

let f = () => {
  let%bind value = ...;
}

which will be turned by refmt into:

let f = () => {
  let%bind value = ...;
  ()
}

(note the trailing () inserted by refmt) which will automatically lead to type error and this will be confusing! Instead if we keep all CPS semantics within a module we could have a monadic unit there or a return / pure. In this case the code:

let f = () => {
  let.Async value = ...;
}

could be transformed into:

let f = () => {
  let.Async value = ...;
  Async.return();
}

which won't lead to a type error.

This will require to supply:

let return : 'a => t('a)

from authors of modules which work with such syntax.

Contributor

andreypopp commented Sep 4, 2018

There's yet another advantage of using a module to represent semantics of CPS bindings.

Consider this piece of code with ppx_let:

let f = () => {
  let%bind value = ...;
}

which will be turned by refmt into:

let f = () => {
  let%bind value = ...;
  ()
}

(note the trailing () inserted by refmt) which will automatically lead to type error and this will be confusing! Instead if we keep all CPS semantics within a module we could have a monadic unit there or a return / pure. In this case the code:

let f = () => {
  let.Async value = ...;
}

could be transformed into:

let f = () => {
  let.Async value = ...;
  Async.return();
}

which won't lead to a type error.

This will require to supply:

let return : 'a => t('a)

from authors of modules which work with such syntax.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Sep 5, 2018

Collaborator

Hrmmmm I think I want to avoid magically adding a Something.return() anywhere.

Collaborator

jaredly commented Sep 5, 2018

Hrmmmm I think I want to avoid magically adding a Something.return() anywhere.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Sep 5, 2018

Collaborator

I made a ppx that implements the let.Module sugar https://github.com/jaredly/let-anything to exercise the idea a bit more.

Collaborator

jaredly commented Sep 5, 2018

I made a ppx that implements the let.Module sugar https://github.com/jaredly/let-anything to exercise the idea a bit more.

@jaredly

This comment has been minimized.

Show comment
Hide comment
@jaredly

jaredly Sep 12, 2018

Collaborator

Ok, I've been using let%Anything for a week and am pretty convinced it's the way to go -- let-> and friends are too "loose/permissive", and I think will just result in confusion, and won't adequately meet the need that started this whole thing -- namely providing a nice, standardized "async/await"-like functionality for dealing with promises.
In light of that, and some feedback I've gotten about "how do I handle exceptions?", I'm also going to add a try transformer that maps to TheModule.try_.
Stay tuned for the refresh.

Collaborator

jaredly commented Sep 12, 2018

Ok, I've been using let%Anything for a week and am pretty convinced it's the way to go -- let-> and friends are too "loose/permissive", and I think will just result in confusion, and won't adequately meet the need that started this whole thing -- namely providing a nice, standardized "async/await"-like functionality for dealing with promises.
In light of that, and some feedback I've gotten about "how do I handle exceptions?", I'm also going to add a try transformer that maps to TheModule.try_.
Stay tuned for the refresh.

@texastoland

This comment has been minimized.

Show comment
Hide comment
@texastoland

texastoland Sep 12, 2018

Tonight and tomorrow I'm writing up a thorough explanation in RFC form of @jaredly's implementation that answers "why not generalized CPS with arbitrary functions", "what about monads" (and how it relates to Haskell's do notation or idiom brackets), "what about F#'s builders", "what about async exceptions", "what could we improve incrementally" (mostly discussed above), etc. I hope it helps for clarification and more focused feedback since this thread is already lengthy.

texastoland commented Sep 12, 2018

Tonight and tomorrow I'm writing up a thorough explanation in RFC form of @jaredly's implementation that answers "why not generalized CPS with arbitrary functions", "what about monads" (and how it relates to Haskell's do notation or idiom brackets), "what about F#'s builders", "what about async exceptions", "what could we improve incrementally" (mostly discussed above), etc. I hope it helps for clarification and more focused feedback since this thread is already lengthy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment