Skip to content
Permalink
Browse files

Add pipeline operator [Feature #15799]

  • Loading branch information...
nobu committed Apr 23, 2019
1 parent e717d6f commit f169043d81524b5b529f2c1e9c35437ba5bc3a7a
Showing with 47 additions and 1 deletion.
  1. +12 −0 NEWS
  2. +1 −0 defs/id.def
  3. +1 −0 ext/ripper/eventids2.c
  4. +24 −1 parse.y
  5. +2 −0 test/ripper/test_scanner_events.rb
  6. +7 −0 test/ruby/test_syntax.rb
12 NEWS
@@ -48,6 +48,18 @@ sufficient information, see the ChangeLog file or Redmine
" # This has been warned since 2.4
EOS

* Pipeline operator is added.

This code equals to the next code.

foo()
|> bar 1, 2
|> display

foo()
.bar(1, 2)
.display

=== Core classes updates (outstanding ones only)

Enumerable::
@@ -111,6 +111,7 @@ token_ops = %[\
OROP ||
ANDDOT &.
METHREF .:
PIPE |>
]

class KeywordError < RuntimeError
@@ -262,6 +262,7 @@ static const struct token_assoc {
{tDSTAR, O(op)},
{tANDDOT, O(op)},
{tMETHREF, O(op)},
{tPIPE, O(op)},
{tSTRING_BEG, O(tstring_beg)},
{tSTRING_CONTENT, O(tstring_content)},
{tSTRING_DBEG, O(embexpr_beg)},
25 parse.y
@@ -1002,7 +1002,7 @@ static void token_info_warn(struct parser_params *p, const char *token, token_in
%type <node> literal numeric simple_numeric ssym dsym symbol cpath
%type <node> top_compstmt top_stmts top_stmt begin_block
%type <node> bodystmt compstmt stmts stmt_or_begin stmt expr arg primary command command_call method_call
%type <node> expr_value expr_value_do arg_value primary_value fcall rel_expr
%type <node> expr_value expr_value_do arg_value primary_value fcall rel_expr pipeline
%type <node> if_tail opt_else case_body case_args cases opt_rescue exc_list exc_var opt_ensure
%type <node> args call_args opt_call_args
%type <node> paren_args opt_paren_args args_tail opt_args_tail block_args_tail opt_block_args_tail
@@ -1060,6 +1060,7 @@ static void token_info_warn(struct parser_params *p, const char *token, token_in
%token <id> tANDDOT RUBY_TOKEN(ANDDOT) "&."
%token <id> tCOLON2 RUBY_TOKEN(COLON2) "::"
%token <id> tMETHREF RUBY_TOKEN(METHREF) ".:"
%token tPIPE RUBY_TOKEN(PIPE) "|>"
%token tCOLON3 ":: at EXPR_BEG"
%token <id> tOP_ASGN "operator-assignment" /* +=, -= etc. */
%token tASSOC "=>"
@@ -1095,6 +1096,7 @@ static void token_info_warn(struct parser_params *p, const char *token, token_in
%nonassoc modifier_if modifier_unless modifier_while modifier_until
%left keyword_or keyword_and
%right keyword_not
%left tPIPE
%nonassoc keyword_defined
%right '=' tOP_ASGN
%left modifier_rescue
@@ -2269,12 +2271,29 @@ arg : lhs '=' arg_rhs
/*% %*/
/*% ripper: ifop!($1, $3, $6) %*/
}
| pipeline
| primary
{
$$ = $1;
}
;

pipeline : arg tPIPE operation2 opt_paren_args
{
/*%%%*/
$$ = new_command_qcall(p, ID2VAL(idPIPE), $1, $3, $4, Qnull, &@3, &@$);
/*% %*/
/*% ripper: command_call!($1, ID2VAL(idPIPE), $3, $4) %*/
}
| arg tPIPE operation2 opt_paren_args brace_block
{
/*%%%*/
$$ = new_command_qcall(p, ID2VAL(idPIPE), $1, $3, $4, $5, &@3, &@$);
/*% %*/
/*% ripper: method_add_block!(command_call!($1, ID2VAL(idPIPE), $3, $4), $5) %*/
}
;

relop : '>' {$$ = '>';}
| '<' {$$ = '<';}
| tGEQ {$$ = idGE;}
@@ -8924,6 +8943,10 @@ parser_yylex(struct parser_params *p)
SET_LEX_STATE(EXPR_BEG);
return tOP_ASGN;
}
if (c == '>') {
SET_LEX_STATE(EXPR_DOT);
return tPIPE;
}
SET_LEX_STATE(IS_AFTER_OPERATOR() ? EXPR_ARG : EXPR_BEG|EXPR_LABEL);
pushback(p, c);
return '|';
@@ -573,6 +573,8 @@ def test_op
scan('op', 'obj.:foo')
assert_equal [],
scan('op', %q[`make all`])
assert_equal %w[|>],
scan('op', %q[x|>y])
end

def test_symbeg
@@ -1379,6 +1379,13 @@ def test_numbered_parameter
assert_syntax_error('@1', /outside block/)
end

def test_pipeline_operator
assert_valid_syntax('x |> y')
x = nil
assert_equal("121", eval('x = 12 |> pow(2) |> to_s(11)'))

This comment has been minimized.

Copy link
@eregon

eregon Jun 13, 2019

Member
# today
x = (12 ** 2).to_s(11)
# the new syntax
x = 12 |> pow(2) |> to_s(11)

Isn't it just longer and less clear and less readable?

This comment has been minimized.

Copy link
@amrabdelwahab

amrabdelwahab Jun 13, 2019

Pipe operator in the functional paradigm makes more sense as it replaces something like:

to_s(pow(12, 2), 11) but I honestly don't see it applicable on replacing chaining methods in OOP

This comment has been minimized.

Copy link
@shioyama

shioyama Jun 13, 2019

Exactly, isn't the proposed usage going to confuse people? It feels like it should be the opposite.

This comment has been minimized.

Copy link
@simi

simi Jun 13, 2019

🤔 This is just test case, not hint how to use it in production code.

This comment has been minimized.

Copy link
@amrabdelwahab

amrabdelwahab Jun 13, 2019

@simi we're commenting about the feature itself, Using a pipe operator to replace method call doesn't make sense and the usual use case of pipe operators is to pass the output of a function as an argument to the next one

This comment has been minimized.

Copy link
@amrabdelwahab

amrabdelwahab Jun 13, 2019

So far the only "change" this operator introduces is gives you the ability to chain methods without parentheses

This comment has been minimized.

Copy link
@nbulaj

nbulaj Jun 13, 2019

Whooa, hold on, so it's not something like the replacement of:

scrap(parse_document(load_http("http://domain.com")))

to

load_http("http://domain.com")
  |> parse_document
  |> scrap

??

If not, don't see any good reason to alias . operator to |> :\

assert_equal(12, x)
end

private

def not_label(x) @result = x; @not_label ||= nil end

29 comments on commit f169043

@watsy0007

This comment has been minimized.

Copy link

replied Jun 13, 2019

awesome!

@anildigital

This comment has been minimized.

Copy link

replied Jun 13, 2019

👍

@igbanam

This comment has been minimized.

Copy link

replied Jun 13, 2019

😍

@simi

This comment has been minimized.

Copy link

replied Jun 13, 2019

clap

@banister

This comment has been minimized.

Copy link

replied Jun 13, 2019

I don't understand this, isn't the whole point of the pipeline operator that it passes the LHS as the arguments to the RHS? i.e 16 |> Math.sqrt #=> 4

Isn't the way it's implemented here almost exactly the same as . (with a very minor difference on parenthesis use) ?

@tom-lord

This comment has been minimized.

Copy link

replied Jun 13, 2019

To echo the above -- In my opinion, this feature is:

  1. Fairly pointless, as it adds more verbose alternate syntax to a solved problem.
  2. Invalid. This is not how pipeline operators are "supposed to" work. (I.e. what a pipeline operator means in other languages.)

Your implementation is:

foo |> bar 1, 2 == foo.bar(1,2)

But actually, what a pipeline operator is supposed to mean is:

foo |> bar 1, 2 == bar(foo, 1,2)
@cweilemann

This comment has been minimized.

Copy link

replied Jun 13, 2019

Another -1 to this change - it makes zero sense.

Why is Ruby trying to be Elixir? Ruby is an OO language. The dot operator is designed for this purpose. The pipe operator is for functional programming.

@iNecas

This comment has been minimized.

Copy link

replied Jun 13, 2019

👎 as well, it's just causes false expectations for fairly common use of that operator not just in JS, the whole ML world is using it the same way http://blog.shaynefletcher.org/2013/12/pipelining-with-operator-in-ocaml.html, https://fsharpforfunandprofit.com/posts/function-composition/#composition-vs-pipeline). IMO introducing this operator would make sense only when doing first some work on making functions more first-class citizens in the language (which I'm not sure how would be done with the current way of doings things).

@AlexWayfer

This comment has been minimized.

Copy link

replied Jun 13, 2019

And -1 from me. |> as alternative for . — what?

We have the same thing in JavaScript (experimental):

And it works as expected by many people, not as in this commit. Please, remake or revert these changes.

@ttilberg

This comment has been minimized.

Copy link

replied Jun 13, 2019

Another -1 to this change - it makes zero sense. Why is Ruby trying to be Elixir? Just don’t.

I think this would be far more acceptable if it was in fact similar to Elixir by providing functionality beyond aliasing .. With a lot of data processing tasks, I often come across the desire to transform simple data in the style of Elixir -- but this feature doesn't allow that. It doesn't pass the return value as an argument to some other arbitrary function --> It requires that whatever you are calling next is still a method defined on the return value.

I think this feature as it stands is overwhelmingly confusing in 2019.

@mame

This comment has been minimized.

Copy link
Member

replied Jun 13, 2019

Most Ruby developers won't read this discussion. It would be good to write it in the Redmine ticket.

I understand the first impression: the new syntax (|>) just duplicates the existing one (.). However, I noticed this might be similar to the relation between do ... end and { ... }. So, for example, we may use |> for multi-line method chain and . for single-line chain?

foo.bar.baz
foo
|> bar
|> baz

It somewhat makes sense because . is easy to overlook:

foo.
bar.
baz
foo
.bar
.baz

But in principle, I'm against adding a symbol keyword. I'm yet unsure if it is good or not. I'd like to consider this feature for a while.

@cweilemann

This comment has been minimized.

Copy link

replied Jun 13, 2019

I think this would be far more acceptable if it was in fact similar to Elixir by providing functionality beyond aliasing .. With a lot of data processing tasks, I often come across the desire to transform simple data in the style of Elixir -- but this feature doesn't allow that. It doesn't pass the return value as an argument to some other arbitrary function --> It requires that whatever you are calling next is still a method defined on the return value.

Exactly. That’s pretty much my point. This simply tries to make Ruby look like Elixir by implementing a functional operator familiar in Elixir. It doesn’t even operate the same way as Elixir’s pipe operator, as has been pointed out in many previous comments above.

@ignatiusreza

This comment has been minimized.

Copy link

replied Jun 13, 2019

👎 on current implementation..

👍 to javascript version of pipeline operator enhanced with https://github.com/tc39/proposal-partial-application, which provide answer to whether return value should be pass in as first or last argument..

As for usage of pipeline operator in OOP language, I find myself writing private methods using functional style more and more.. especially useful when breaking down complex method into smaller "helper" methods..

@meinac

This comment has been minimized.

Copy link

replied Jun 13, 2019

I don't know if this is only a foundation for a useful pipe operator which takes the output of the first function and passes it to the second function as argument but if not this operator is just useless.

@Donavan

This comment has been minimized.

Copy link

replied Jun 13, 2019

Why would you add something called a pipe operator that's not actually a pipe operator?

@DaniG2k

This comment has been minimized.

Copy link

replied Jun 13, 2019

Not a fan of this. It's:

  1. Mimicking Elixir for no good reason
  2. Not operating as a pipe operator should
  3. Trying to solve a solved problem
@aeroastro

This comment has been minimized.

Copy link

replied Jun 13, 2019

I believe the current implementation reflects Ruby's internal architecture. A method takes the receiver (or self) as the implicit first argument. So, it is "a pipeline operator" from the viewpoint of some Ruby committers.

However, I believe this is not so natural nor practical from the viewpoint of most Ruby users.

Since Ruby is elegantly crafted, most Ruby users can forget about the implicit first argument (e.g. no need for my $self = shift;). They can focus on writing OO codes on explicit arguments. Even when directly calling Method#call, we are free from writing boilerplates. 🎉

In this situation, it seems natural to expect the LHS result to be passed as the first explicit argument of RHS when using a pipeline operator |>.

I want the new pipeline operator to be just as elegant and practical as the current overall Ruby design, hopefully from the viewpoint of Ruby users. ❤️

@baweaver

This comment has been minimized.

Copy link
Contributor

replied Jun 13, 2019

I will write more on my thoughts later, but the example given of aliasing like do .. end vs { ... } is unconvincing as that already causes confusion for more Junior engineers.

In short I agree with most of the opinions presented here already that we should emulate Elixir and Javascript's style of pipeline operator.

As is we are adding sugar without substance, which I would advocate against as it does not increase the expressiveness of the language.

@shuber

This comment has been minimized.

Copy link

replied Jun 13, 2019

Check out this operator-less pipe_operator proof of concept:

# before
JSON.parse(Net::HTTP.get(URI.parse(url)))

# after
url.pipe { URI.parse; Net::HTTP.get; JSON.parse }
"https://api.github.com/repos/ruby/ruby".pipe do
  URI.parse
  Net::HTTP.get
  JSON.parse.fetch("stargazers_count")
  yield_self { |n| "Ruby has #{n} stars" }
  Kernel.puts
end
#=> Ruby has 15120 stars
@hanachin

This comment has been minimized.

Copy link
Contributor

replied Jun 14, 2019

I realize if I want to pass LHS value as the argument to other object's method, I can take the method from other Object and pass the method object to "then" method as a block.

16 |> then(&Math.:sqrt)
# => 4.0
require "net/http"
require "json"

"https://api.github.com/repos/ruby/ruby"
  |> then(&URI.:parse)
  |> then(&Net::HTTP.:get)
  |> then(&JSON.:parse)
  |> fetch("stargazers_count")
  |> then(&"Ruby has %d stars".:%)
  |> then(&self.:puts)

or use then with named parameters.

require "net/http"
require "json"

"https://api.github.com/repos/ruby/ruby"
  |> then { URI.parse @1 }
  |> then { Net::HTTP.get @1 }
  |> then { JSON.parse @1 }
  |> fetch("stargazers_count")
  |> then { "Ruby has %d stars" % @1 }
  |> then { puts @1 }

or refine #> for shorthand

using Module.new {
  revapply = Module.new {
    def >(f)
      f.call(self)
    end
  }
  refine(Object) { include revapply }
  refine(Comparable) { include revapply }
  refine(Float) { include revapply }
  refine(Integer) { include revapply }
}

16 |>> Math.:sqrt |>> self.:puts
# 4.0

require "net/http"
require "json"

"https://api.github.com/repos/ruby/ruby"
  |>> URI.:parse
  |>> Net::HTTP.:get
  |>> JSON.:parse
  |> fetch("stargazers_count")
  |>> "Ruby has %d stars".:%
  |>> self.:puts
# Ruby has 15854 stars

or convert argument to Proc by .:itself then Proc#>> then call it.

require 'net/http'
require 'uri'
require 'json'

"https://api.github.com/repos/ruby/ruby".:itself
  |>>> URI.:parse
  |>>> Net::HTTP.:get
  |>>> JSON.:parse
  |> call
  |> fetch("stargazers_count")
  |> then { puts @1 }
@baweaver

This comment has been minimized.

Copy link
Contributor

replied Jun 14, 2019

I have captured my thoughts on this here: https://dev.to/baweaver/ruby-2-7-the-pipeline-operator-1b2d

I believe the alias functionality is actually a very good thing when combined with implementations from other languages to make what could potentially be a massive win for the Ruby language in terms of expressiveness.

As it exists now, I do not believe this is a good idea.

@shioyama

This comment has been minimized.

Copy link

replied Jun 14, 2019

@baweaver This is fantastic:

def double(n)
  n * 2
end

increment = -> n { n + 1 }

5 |> double |> double |> increment |> double
# => 42

I actually think a lot of people giving this feature thumbs up are assuming this is the usage, when in fact it is not.

@kubakrzempek

This comment has been minimized.

Copy link

replied Jun 14, 2019

Isn't the nature of the pipeline operator captured in then blocks? If the pipe would alias the blocks then that'd make sense. In the current form, I don't see how the pipe is beneficial.

@avit

This comment has been minimized.

Copy link

replied Jun 14, 2019

@kubakrzempek Yes. I was also hoping then (previously Object#yield_self / Object#as) could support a true pipeline syntax like this, suggested here:

https://bugs.ruby-lang.org/issues/10095#note-22

@zernie

This comment has been minimized.

Copy link

replied Jun 14, 2019

This is quite literally NOT a pipeline operator, which makes this feature somewhat unnecessary and unexpectedly confusing.
@baweaver's suggestion of making |> an alias to then is great and fairly easy to implement.

@rbq

This comment has been minimized.

Copy link

replied Jun 14, 2019

Is this some sort of belated April fools' prank? Could we please also introduce a this keyword that points to a random object in memory?

@konsolebox

This comment has been minimized.

Copy link

replied Jun 14, 2019

|> aliased to then looks better.

@weshatheleopard

This comment has been minimized.

Copy link

replied Jun 14, 2019

So another way to inflate the "I committed XXX lines yesterday" count? :)

@intpl

This comment has been minimized.

Copy link

replied Jun 15, 2019

this is contrary to the line "a programmer's best friend" in logo on the front page of https://www.ruby-lang.org/

  1. aliasing '.' to '|>' is just plain useless and very few people will use it. this is wasted potential, because Elixir (+others) syntax is just plain gold and I would LOVE to have it in Ruby. this would ease up writing Service Objects in some cases or maybe even introduce new design patters e.g. "one object oriented service objects" (bad name sorry). some gems would even disappear, e.g. this one: https://rubygems.org/gems/solid_use_case/versions/2.1.1

  2. what about programmers working both with Ruby and JS at the same time? and we are many, really. how can we be productive when we need to switch between meaning of the same operator all the time?

  3. what about only-JS programmers that just have to read some of the backend side of their projects to understand something? or maybe change some simple code? i imagine the hell they are gonna be in when having to read code with this "new" operator.

Please sign in to comment.
You can’t perform that action at this time.