Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature wish] introduce real pipe syntax # like elixir #1452

Closed
bobzhang opened this issue Oct 4, 2017 · 52 comments
Closed

[feature wish] introduce real pipe syntax # like elixir #1452

bobzhang opened this issue Oct 4, 2017 · 52 comments

Comments

@bobzhang
Copy link
Contributor

@bobzhang bobzhang commented Oct 4, 2017

Introduce a syntax # (was |.)

x 
#method0(a0, a1)
#method1(b0, b1)
#method2(c0, c1)

which is equivalent to

(method2 (method1 (method0 x a0 a1) b0 b1 ) c0 c1)

the reason is that x's type is usually known, so that by type flow, a0, a1 can have better auto-completion and less type annotation

Edit: maybe |. is too heavyweight, how about using ..

(x : M.t)
#method0 (a0,a1)
#method1 (b0,b1)
#method2 (c0,c1)

would be translated to

M.method2 (M.method1 (M.method0 (x, a0,a1), b0,b1) c0,c1)

Edit: I was convinced that t comes first seems to be better, it is easy to remember, and when we export functions to JS users, the API would be more familiar. Actually I think using # seems better

@OvermindDL1
Copy link

@OvermindDL1 OvermindDL1 commented Oct 4, 2017

What about overriding |> and such then when using a locally opened DSEL? Wouldn't that break then?

Loading

@bassjacob
Copy link

@bassjacob bassjacob commented Oct 4, 2017

would have to make it protected and not overridable I guess.

Loading

@chenglou
Copy link
Contributor

@chenglou chenglou commented Oct 5, 2017

What about this:

x
|> foo a
|> bar _ b
|> baz c

To mean:

(baz c (bar (foo a x) b))

Loading

@bassjacob
Copy link

@bassjacob bassjacob commented Oct 5, 2017

so _ in value position is used as a hole only for this purpose?

Loading

@hcarty
Copy link
Contributor

@hcarty hcarty commented Oct 5, 2017

What is the reasoning behind this rewriting?

Loading

@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Oct 5, 2017

The reason is that type inference flow from to right, having object as the first argument makes type inference, annotation, auto-completion works better

Loading

@OvermindDL1
Copy link

@OvermindDL1 OvermindDL1 commented Oct 5, 2017

@bassjacob I'm talking about using libraries that you don't control. It needs to be compatible with normal OCaml.

Loading

@hcarty
Copy link
Contributor

@hcarty hcarty commented Oct 5, 2017

@bobzhang How is that different after the rewrite? Is the compiler primitives behind |> insufficient for that purpose? This is an aspect of the language semantics I'm unfamiliar with.

Loading

@bassjacob
Copy link

@bassjacob bassjacob commented Oct 5, 2017

@OvermindDL1 great point! (I'm not for or against this change) but I think you could get around that by only treating non-module code this way. Seems like a pretty invasive change, but probably doable.

Loading

@OvermindDL1
Copy link

@OvermindDL1 OvermindDL1 commented Oct 5, 2017

In addition to the fact that many many existing ocaml code, including in it's standard library, also assume piping to the end.

Loading

@cullophid
Copy link

@cullophid cullophid commented Oct 5, 2017

Maybe thats the topic of a larger discussion... does the benefits of interop with ocaml libs outweigh the problems...
There are a lot of improvements that could be made to the standard library...

Loading

@OvermindDL1
Copy link

@OvermindDL1 OvermindDL1 commented Oct 5, 2017

Considering I am needing to compile for both the web (via bucklescript) and to native code (via the stock ocaml compiler), definitely yes. ^.^

Loading

@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Oct 5, 2017

Loading

@hcarty
Copy link
Contributor

@hcarty hcarty commented Oct 5, 2017

I still don't see the benefit the rewriting brings, even ignoring OCaml compatibility.

Loading

@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Oct 5, 2017

x |> f is not equivalent to f x, under my proposal it would be. Under current ocaml semantics, it works 95% of the cases, but it would cause surprises for the rest 5% (note it applies to native backend as well).

  1. OCaml optimizer comes very late( in the lambda level) to optimize |>, so it does incur some perf loss in some cases
  2. It has subtle difference on typing rules

|. can not be expressed as a function, it interacts better with type inference where the type information flows from left to right (+ better interaction with type based record disambiguation)

As I said, this does not make compatibility worse with OCaml given that ReasonML already has different keyword sets. I would suggest to make ReasonML a subset of OCaml in the future, which means removing some flexibility from OCaml to make it more friendly for tools(including IDE). Other changes I have in mind: removing open, include, customized operators etc

Loading

@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Oct 5, 2017

Removing features sometimes would make the language even better : )

Loading

@IwanKaramazow
Copy link
Contributor

@IwanKaramazow IwanKaramazow commented Oct 5, 2017

This is a beautiful idea to solve |> at the parsing level!

Loading

@hcarty
Copy link
Contributor

@hcarty hcarty commented Oct 5, 2017

@bobzhang Do you have some examples of the typing differences between %revapply and direct application? I haven't run into them before but I'd like to recognize them if I do in the future.

As for the proposed removals, I'll let it at "please no, don't break the language, let's solve this socially" until they're formally proposed.

Loading

@yyc-git
Copy link

@yyc-git yyc-git commented Oct 13, 2017

@bobzhang Could you desugar f @@ x in the parsing level?

Loading

@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Oct 13, 2017

Loading

@yyc-git
Copy link

@yyc-git yyc-git commented Oct 13, 2017

Thanks for your reply.
ok, I think use '<|' is better than '@@' too. so maybe not consider about '@@'.

Loading

@jordwalke
Copy link
Member

@jordwalke jordwalke commented Oct 14, 2017

Still thinking about this but regardless, Reason can create new syntactic forms and shortcuts and in doing so it would remain fully compatible with OCaml, and any OCaml backend. For example if |> is transformed at parse time into function application, we merely find another way to express the original |> which invokes a function named (|>). It's just a syntactic remapping of AST concepts.

But I don't fully understand the benefits of doing it at parse time, as opposed to a later stage. I don't really see many downsides except that you couldn't redefine |> (not a huge downside in my opinion). What's the benefits though?

Loading

@chenglou
Copy link
Contributor

@chenglou chenglou commented Oct 14, 2017

@jordwalke bob's last sentence makes sense. Also #1511

Loading

@jordwalke
Copy link
Member

@jordwalke jordwalke commented Oct 14, 2017

"x |> f is not equivalent to f x, under my proposal it would be. Under current ocaml semantics, it works 95% of the cases, but it would cause surprises for the rest 5% "

Bob, can you provide an example where it behaves unexpectedly?

Loading

@yyc-git
Copy link

@yyc-git yyc-git commented Oct 14, 2017

@jordwalke @bobzhang

label function with pipe operator(function compose) error! #1511

It maybe one example?

Loading

@cullophid
Copy link

@cullophid cullophid commented Nov 17, 2017

|. could be a footgun for newcomers.
It would let them write data first funcitons instead of data last.
Especially since data first is the standard in js

Loading

@chenglou
Copy link
Contributor

@chenglou chenglou commented Nov 17, 2017

@jordwalke are you onboard with |> as syntax? We can discuss about <| separately

Loading

@jordwalke
Copy link
Member

@jordwalke jordwalke commented Nov 17, 2017

I can't see anything wrong with doing so as long as we retain some way to also express the original form (losslessly converting from OCaml for example). Are there any other benefits or tradeoffs to be aware of besides better compiler output? @bobzhang could you comment about the 5% of the times that x |> f is not the same as f x in OCaml's semantics? I haven't run into one yet, and I want to be aware of the differences before implementing the feature.

Loading

@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Dec 8, 2017

@jordwalke

see different output for the v below

let f ?(x=1) z ?(y=2)  = 
   x + y + z   
(* let v = f 3    *)
let v = 3 |> f

Loading

@jordwalke
Copy link
Member

@jordwalke jordwalke commented Dec 8, 2017

Ah, that's a good example. The limitations of named argument when used as arguments to higher order functions like |> are getting in the way!

I think that's a good justification for applying |> at the syntax level. No one ever uses |> as an argument to List.map for example.

To complete this feature, here is what would need to be done:

  1. Determine how what |> parses to. I propose Pexp_apply but with a non-printed attribute like [@Reason.pipe_operator]. Then when printing, you know to print it as the pipe operator.
  2. Determine how to express the previous version of |> for completeness. So that we can convert perfectly from OCaml and back.
  3. Perform the special casing of Number 1 in the parser.

Loading

@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Dec 8, 2017

https://blog.janestreet.com/core-principles-uniformity-of-interface/
Actually I agreed with it that t comes first rule, so in that case: .. (or the original |.) seems more useful

Here is my motivating example that why t comes first is better, and why we need a pipe syntax operator

let f (xs: 'a list) (u : 'a -> bool) = 
  List.for_all u xs 

let f2  (u : 'a ->  bool) (xs: 'a list)= 
  List.for_all  u xs 

type t = { x : int ; y : int}
type u = { x : int ; y : int}

let t xs  = f (xs : t list)    (fun {x;y} -> x = y )

let t2 xs = 
  f2 (fun {x; y} -> x = y ) (xs : t list) (* would not compile *)

Loading

@jordwalke
Copy link
Member

@jordwalke jordwalke commented Dec 9, 2017

Here's a Reason rewriting of Bob's example in Reason:

let forAllItemsFunc = (xs: list('a), func: 'a => bool) => List.for_all(func, xs);
let forAllFuncItems = (func: 'a => bool, xs: list('a)) => List.for_all(func, xs);

type p1 = {x: int, y: int};
type p2 = {x: int, y: int};

let t = (items) => forAllItemsFunc(items: list(p1), ({x, y}) => x == y);

/* Does not compile - compiler expects items to be a list of p2 because
 * it inferred the first argument to be of type p2=>bool */
let t2 = (items) => forAllFuncItems(({x, y}) => x == y, items: list(p1));

It shows a situation where the "t comes first" API design guideline mentioned by Jane Street prevents an unnecessary type error. Type inference infers the type of the first argument, then the second, and then tries to substitute them in the argument position of the function's inferred type. I think that normally this wouldn't put "t comes first" at an advantage/disadvantage but OCaml has something called "type directed record field disambiguation", which allows the type system to know if x and y in let {x, y} = value refers to p1 or p2 based on the inferred type of value. Apparently the inferred types of values only propagate through inferred function types from left to right in function arguments, not right to left.

I think there are also cases where "t comes first" is put at a disadvantage for the same reasons as above (left-to-right bias of record field disambiguation). Here's an example where now "t comes first" won't compile but the other form will.

let forAllItemsFunc = (xs: list('a), func: 'a => bool) => List.for_all(func, xs);
let forAllFuncItems = (func: 'a => bool, xs: list('a)) => List.for_all(func, xs);
type p1 = {x: int, y: int};
type p2 = {x: int, y: int};

/** Does not compile for the same reason, but this time to the disadvantage of
 * "t comes first". */
let t = (items) => forAllItemsFunc([{x: 0, y: 0}], ({x, y}: p1) => x == y);

let t2 = (items) => forAllFuncItems(({x, y}: p1) => x == y, [{x: 0, y: 0}])

There are other reasons to favor "t comes first" for Reason - for example, JS developers are familiar with callbacks coming last. Iwan even built special printing support for it. There are some cases where "t comes last" is better - like when you have optional named arguments, the final unnamed t "fills in the defaults" without having to supply a final () argument.

Regardless, a good reason for |> being implemented in Reason is that higher order functions like |> trip up named arguments as Bob showed. If anyone wants to take a shot at implementing |> in Reason, let me know.

Loading

@glennsl
Copy link
Contributor

@glennsl glennsl commented Dec 9, 2017

Would |> at the syntax level allow variant constructors to be treated the same as functions?

I still occasionally get semi-surprised that this doesn't work:

getThing() |> doThis |> doThat |> Some

Perhaps because the syntactic similarity of function application and variant construction makes it seem like it should.

Loading

@jordwalke
Copy link
Member

@jordwalke jordwalke commented Dec 9, 2017

@glennsl Yeah, that should be possible as well. It would come with some complexities. For example, would we pretend that variants like Two(x, y) are curried, but only in this case of using |>?

Would we pretend that Two can be partially applied like getThing() |> doThis |> Two(y) - where it is equivalent to Two(y, getThing() |> doThis)? If so, people would probably wonder why it doesn't work everywhere else.

Loading

@glennsl
Copy link
Contributor

@glennsl glennsl commented Dec 9, 2017

Yeah, I'd say variants should not be treated as if they were curried. Mostly because the error message for a partially applied variant would likely be very confusing, but it also seems more consistent conceptually.

Loading

@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Dec 12, 2017

@jordwalke very few people write type annotations for callback instead of the data

Loading

@bobzhang bobzhang changed the title [feature wish] make |>, <| syntax instead of function [feature wish] introduce real pipe syntax # like elixir Dec 19, 2017
@Risto-Stevcev
Copy link

@Risto-Stevcev Risto-Stevcev commented Jan 15, 2018

@bobzhang It sounds like |> currently is like clojure's thread-last macro (->>), and this proposal is to introduce a change to make it like the thread-first macro (->) instead

Can we still keep thread-last? maybe the current syntax which is thread last can follow clojure's convention and be |>> if the plan is to change the behavior of |> to be thread-first

Removing the thread-last macro completely would mean that there's no way to pipe anything for a lot of APIs that are used to the default behavior for |>, and I think they both have their place

Loading

@jaredly
Copy link
Contributor

@jaredly jaredly commented Jan 15, 2018

Yeah we will definitely keep thread-last.
@bobzhang why the switch from |. to #?

Loading

@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Jan 16, 2018

@jaredly I expect it to be used frequently, saving one character seems worthwhile.

For the record, another reason is that avoid such ambiguity & more consistent interop with js libs

x |> append y
x # append y

Loading

@jaredly
Copy link
Contributor

@jaredly jaredly commented Jan 16, 2018

I'd probably vote against re-using # because of possible confusion, given that one will be a globally available thing, and ## is js-land only.

Loading

@Risto-Stevcev
Copy link

@Risto-Stevcev Risto-Stevcev commented Jan 16, 2018

My two cents:
In haskell+purescript, $ is the equivalent of |>, and # is <|. I prefer the arrow style (|>) over haskell's style because the symbol implies that some kind of piping is going on ($ is a currency 😂), and it also implies which direction it is (# is confusing because it doesn't tell you that it's $ flipped)
I don't mind sacrificing an extra character or two over having the extra clarity -- I use the ocaml style pipe operator in purescript code instead

Loading

@strega-nil
Copy link

@strega-nil strega-nil commented Jan 16, 2018

@Risto-Stevcev isn't $ the equivalent of <|? you write add a $ mul a b for a + (a * b). If it were the equivalent of |>, it'd be (a * b) (add a), which doesn't make a lot of sense - function application on natural numbers :P

Loading

@Risto-Stevcev
Copy link

@Risto-Stevcev Risto-Stevcev commented Jan 16, 2018

Ah yeah, sorry had the wrong way around. $ is <|, and # is |>. Forgot that haskell highly prefers the former and other fp languages prefer the latter

Loading

@rauschma
Copy link

@rauschma rauschma commented Feb 1, 2018

One more argument in favor of “main parameter first”: it works better with naming, because the “label” for that parameter is the function name and it’s therefore better if that label is located directly before it.

However, “positional last” is built deeply into the language. For example, you can’t erase the optional parameter y if you define the function as you did:

let f ?(x=1) z ?(y=2) = x + y + z

So that would have to be changed, too!

Loading

@rauschma
Copy link

@rauschma rauschma commented Feb 1, 2018

It may make sense to collect all reasons in favor of this significant change in a single document (which doesn’t have to be long). To convince critics and in order to document it for the future.

Loading

@strega-nil
Copy link

@strega-nil strega-nil commented Feb 1, 2018

I made a comment which I would now disagree with, so I deleted it.

I would put the jane street article in front, as to the reasons to make the change.

I also very much do not like the # syntax; I'd prefer something like <|.

Loading

@chenglou
Copy link
Contributor

@chenglou chenglou commented Feb 14, 2018

#1804 landed

Loading

@chenglou chenglou closed this Feb 14, 2018
@bobzhang
Copy link
Contributor Author

@bobzhang bobzhang commented Mar 1, 2018

for the record, rescript-lang/rescript-compiler#1671 is an example of cost of |>

Loading

@nkrkv
Copy link

@nkrkv nkrkv commented Mar 30, 2018

Does introduction of _ in #1804 make this proposal obsolete? I mean, is a counterpart for BS (|.) planed in ReasonML or not?

If not, am I understand correctly that the recommended pattern for practical usage is:

open Belt;

myList
|> List.filter(_, foo)
|> List.map(_, qux)
|> List.reduce(_, acc, bar)
|> Option.flatMap(_, baz)
|> Option.getWithDefault(_, 42);

Loading

@chenglou
Copy link
Contributor

@chenglou chenglou commented Mar 30, 2018

You can use the same |. in Reason

Loading

@nkrkv
Copy link

@nkrkv nkrkv commented Mar 30, 2018

😮 Indeed! Never thought about it as saw no mentions in Re-specific changelogs and docs.

Works excellent. Will it be kept or is it a highly-experimental feature intensive usage of which is discouraged for now?

The very hot discussion in this issue interrupted suddenly and I’m not sure about the current official position/recommendation of RE authors.

Loading

@chenglou
Copy link
Contributor

@chenglou chenglou commented Mar 30, 2018

We'll keep it. So not highly experimental. Even if it was, we'd provide a good migration path just in case

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet