New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Hack-style Pipelining #84

Open
zenparsing opened this Issue Jan 22, 2018 · 87 comments

Comments

Projects
None yet
@zenparsing
Copy link
Member

zenparsing commented Jan 22, 2018

The current proposal (in which the RHS is implicitly called with the result of the LHS) does not easily support the following features:

  • Piping into a function with multiple arguments
  • Calling methods on a piped value
  • Awaiting a piped value

In order to better support these features the current proposal introduces special-case syntax and requires the profilgate use of single-argument arrow functions within the pipe.

This proposal modifies the semantics of the pipeline operator so that the RHS is not implicitly called. Instead, a constant lexical binding is created for the LHS and then supplied to the RHS. This is similar to the semantics of Hack's pipe operator.

Runtime Semantics

PipelineExpression : PipelineExpression |> LogicalORExpression
  1. Let left be the result of evaluating PipelineExpression.
  2. Let leftValue be ? GetValue(left).
  3. Let oldEnv be the running execution context's LexicalEnvironment.
  4. Let pipeEnv be NewDeclarativeEnvironment(oldEnv).
  5. Let pipeEnvRec be pipeEnv's EnvironmentRecord.
  6. Perform ! pipeEnvRec.CreateImmutableBinding("$", true).
  7. Perform ! pipeEnvRec.InitializeBinding("$", leftValue);
  8. Set the running execution context's LexicalEnvironment to pipeEnv.
  9. Let right be the result of evaluating LogicalORExpression.
  10. Set the running execution context's LexicalEnvironment to oldEnv.
  11. Return right.

Example

anArray
  |> pickEveryN($, 2)
  |> $.filter(...)
  |> shuffle($)
  |> $.map(...)
  |> Promise.all($)
  |> await $
  |> $.forEach(console.log);

Advantages

  • All use cases described above are supported with no special-case syntax.
  • By removing the implicit-call behavior, we remove a potential source of confusion for developers.

Disadvantages

  • By requiring an explicit function call, functional programming patterns that rely heavily on currying will be less ergonomic.

Notes

The choice of "$" for the lexical binding name is somewhat arbitrary: it could be any identifier. It should probably be one character and should ideally stand out from other variable names. For these reasons, "$" seems ideal. However, this might result in a conflict for users that want to combine both jQuery and the pipeline operator. Personally, I think it would be a good idea to discourage usage of "$" and "_" as variable names with global meanings. We have modules; we don't need jQuery to be "$" anymore!

@littledan

This comment has been minimized.

Copy link
Member

littledan commented Jan 22, 2018

Thanks for writing this out in such a detailed way. I didn't understand exactly what you were suggesting before. I like how this direction is very explicit and general; it completely handles nested subexpressions in a way that I didn't previously understand was possible.

The cost seems to be the loss of terse syntax for x |> f |> g, which becomes x |> f($) |> g($). I'm not sure how important conciseness for that case is.

Bikeshedding: I'm not sure if we want to use _ or $ when they are so widely used for libraries. There are lots of other names possible, or we could go with something currently untypable like <>.

@zenparsing

This comment has been minimized.

Copy link
Member

zenparsing commented Jan 22, 2018

I'm not sure if we want to use _ or $ when they are so widely used for libraries.

True, using $ or _ will probably make some people unhappy. On the other hand, we should prioritize the future over the past.

Another thought: a pipe operator with these semantics would obviate the need for the current :: proposal. We could re-purpose binary :: for just method extraction:

class XElem extends HTMLElement {
  _onClick() {
    // ...
  }

  connectedCallback() {
    this.addEventListener('click', this::_onClick);
  }

  disconnectedCallback() {
    // Assuming this::_onClick === this::_onClick
    this.removeEventListener('click', this::_onClick);
  }
}

which would be pretty nice and intuitive.

@gilbert

This comment has been minimized.

Copy link
Collaborator

gilbert commented Jan 22, 2018

Nicely done! The loss of terse syntax for unary and curried functions is arguably significant, but in return you get pretty much everything else one could ask for 😄

Some more fun features of this syntax:

let lastItem =
  getArray() |> $[$.length-1]

let neighbors =
  array.indexOf('abc') |> array.slice($ - 1, $ + 2)

let silly =
  f |> $(x, y)
@isiahmeadows

This comment has been minimized.

Copy link
Contributor

isiahmeadows commented Jan 22, 2018

This looks a lot like a variant of the partial application proposal, just using a $ (a valid identifier) instead of a ? (which is what that proposal uses).

Maybe we could take some of the features proposed here and integrate them into that proposal, but from the surface, this looks flat out identical to that proposal mod the change in syntactic marker.

@mAAdhaTTah

This comment has been minimized.

Copy link
Contributor

mAAdhaTTah commented Jan 22, 2018

I guess I'm surprised $ would even be possible. Something like the $.map example would be especially confusing because jQuery has a $.map. Could the grammar specify whether that's supposed to call immediately or not? If someone did this:

const $ = require('ramda') // using ramda cuz I'm familiar

[1, 2] |> $.map(x => x + 1)

it can't tell when it should use the return value of Ramda's curried map or the Array.prototype.map. Is using a valid variable name explicitly intended?

@isiahmeadows My understanding is this proposal explicitly evolved out of prior attempts to combine the two proposals.

Given that, though is the intention to actually combine the proposals or use the placeholder semantics defined as part of the pipeline proposal as a springboard for adding partial application to the language generally?

@js-choi

This comment has been minimized.

Copy link
Contributor

js-choi commented Jan 22, 2018

I'm not actually proposing this, but, for what it's worth, the Clojure programming language has a similar pipelining feature called the as-> macro. It has a somewhat different approach, though: it requires explicitly specifying the placeholder argument at the beginning of the pipeline—which may be any valid identifier.

That is, the example in the original post would become something like:

anArray |> ($) { // The $ may be any arbitrary identifier
  pickEveryN($, 2),
  $.filter(...),
  shuffle($),
  $.map(...),
  Promise.all($),
  await $,
  $.forEach(console.log)
}

…but the $ could instead be _ or x or or any other valid identifier. That's the advantage: It doesn't implicitly clobber a specially privileged identifier; it requires an explicit identifier for its placeholder argument binding. The downside is that it needs to have its own block—which would contain a list of expressions through which to pipe placeholder data, and within which the placeholder identifier would resolve—rather than the current |> binary operator. That's probably not tenable, I'm guessing.

Having said all that, like @isiahmeadows I too am wondering if ?, as a nonidentifier, would not be a better choice here. The relationship between this proposal and the partial-application proposal may need to be once again considered.

@JAForbes

This comment has been minimized.

Copy link

JAForbes commented Jan 23, 2018

I see the benefit of this for multi arg functions, but I don't understand why we need a placeholder for single arg functions? A lot of the FP community in JS has moved or is moving to unary only functions, and so for us it'd be quite natural to do x |> f |> g |> h, littering that expression with $ (or some other symbol) for every application would be pretty noisy, so I'd like to know the benefit.

@JAForbes

This comment has been minimized.

Copy link

JAForbes commented Jan 23, 2018

I'm also really surprised await is even a part of this proposal.

anArray
  |> pickEveryN($, 2)
  |> $.filter(...)
  |> shuffle($)
  |> $.map(...)
  |> Promise.all($)
  |> await $
  |> $.forEach(console.log);

Should just be with lodash/fp, ramda, sanctuary ( etc )

anArray
  |> pickEveryN(2)
  |> filter(f)
  |> shuffle
  |> map(f)
  |> Promise.all
  |> then( forEach(console.log) )

|> is a very simple operator, it just applies a function with the result of the previous value, it doesn't need any other semantics.

If we want to use legacy OO style libraries with this syntax, we should have a separate proposal (like the placeholder proposal), and it should not be pipeline specific, it should be language wide. The worst case scenario with arrow functions is completely fine. And the fact it's not as convenient as unary functions is a good thing, it's a subtle encouragement to think compositionally and design API's that enable that.

E.g. let's say we're using pipeline with no placeholder, with no await, with standard lodash and bluebird.

anArray
  |> x => pickEveryN(x, 2)
  |> x => filter(x, f)
  |> shuffle
  |> x => map(x, f)
  |> Promise.all
  |> then( xs => xs.forEach(console.log) )

It's not bad for an API that is not designed for unary application.

I think we may be over-complicating things to support functionality we don't need to support. The work-around of using arrow functions is still simple to follow, and any special placeholder syntax should be language wide anyway and probably beyond the scope of this proposal.

@mAAdhaTTah

This comment has been minimized.

Copy link
Contributor

mAAdhaTTah commented Jan 23, 2018

Unfortunately, latest guidance from TC39 is they want await support. This is a syntax that will need to be solved before it can advance.

@TehShrike

This comment has been minimized.

Copy link
Collaborator

TehShrike commented Jan 23, 2018

Is nobody at TC39 willing to oppose await in this proposal? It seems like a bad idea to take this super-handy operator common to other languages and bolt on this awkward feature.

@TehShrike

This comment has been minimized.

Copy link
Collaborator

TehShrike commented Jan 23, 2018

To be fair, I haven't heard the discussion, so maybe I'm missing something?

@TehShrike

This comment has been minimized.

Copy link
Collaborator

TehShrike commented Jan 23, 2018

The notes from the discussion seem more negative towards the idea of bolting on await/yield than anything http://tc39.github.io/tc39-notes/2017-11_nov-29.html#interaction-of-await-and-pipeline

@JAForbes

This comment has been minimized.

Copy link

JAForbes commented Jan 23, 2018

Yeah I agree, and the arguments for await are either not justifications or they are reaching for exotic scenarios.

KCL: There are valid usecases for await expressions in a pipeline, an async function that returns a function could be used here.

This is supported by the above |> then( f => f(x) ). Also if exotic examples are required to justify this syntax, then it's not really justifiable.

DE: Yes, I think it'd be very strange if we didn't allow this.

I think we need more guidance than this.

Its a very simple operator, we don't need to complicate it. I think @zenparsing's suggestion is an elegant solution for the constraints tc39 have provided, but I don't think the constraints have any merit and we should seek further guidance.

If we ship it with await banned, and then people want to add await later if/when there's a legitimate justification, we can. It won't break any code. But I personally doubt there will be a justification, await solves a different set of problems.

I'd be in favour of some static functions on the Promise object (e.g. Promise.then ) and ban await for now with an option to add it later if there's ever a need.

@bakkot

This comment has been minimized.

Copy link

bakkot commented Jan 23, 2018

I don't think this is the right thread to revisit await. As a committee member, though, I agree with the assessment that it's something this proposal needs to handle if it's to get through; if you want to discuss that further, I think a new issue would be appropriate.

@mAAdhaTTah

This comment has been minimized.

Copy link
Contributor

mAAdhaTTah commented Jan 23, 2018

Personally, I think part of what makes the pipeline operator so elegant is its simplicity. I'd be extremely happy simply writing code like this:

anArray
  |> x => pickEveryN(x, 2)
  |> x => filter(x, f)
  |> shuffle
  |> x => map(x, f)
  |> Promise.all
  |> then(xs => xs.forEach(console.log))

This builds on syntax I'm already familiar and comfortable with (arrow functions), themselves already beautifully simple and elegant, and it feels like a natural evolution for them. Introducing both the |> and a pipeline-only placeholder syntax feels like... too much. I don't think the extra syntax is needed, and I think we'd be better off treating that as a different issue.

I still think I prefer inlining await into the pipeline, despite ASI hazards, but in the interest in exhausting all options, what about maintaining the elegance of the current approach, but use the placeholder syntax to "solve" await:

anArray
  |> x => pickEveryN(x, 2)
  |> x => filter(x, f)
  |> shuffle
  |> x => map(x, f)
  |> Promise.all
  |> await ?
  |> xs => xs.forEach(console.log)

such that the placeholder is only used in await in a pipeline? For the most common case (sync pipelining), we still get a great syntax with arrow functions, and still looks pretty clean in the async case, and perhaps could be useful as a "foot in the door" for placeholders without requiring fully implementing that proposal, which would give developers an opportunity to start developing an intuition for it.

Thoughts?

@zenparsing

This comment has been minimized.

Copy link
Member

zenparsing commented Jan 23, 2018

@JAForbes Javascript is a big, huge tent, filled with many people that don't consider OOP "legacy". We need to consider everyone under that tent when designing syntax.

@mAAdhaTTah I don't think we can assume yet that "arrow functions without parens" syntax is going to work out: see #70 (comment). If we have to rewrite your example with the parentheses, it doesn't look as attractive:

anArray
  |> (x => pickEveryN(x, 2))
  |> (x => filter(x, f))
  |> shuffle
  |> (x => map(x, f))
  |> Promise.all
  |> await
  |> (xs => xs.forEach(console.log))
@JAForbes

This comment has been minimized.

Copy link

JAForbes commented Jan 23, 2018

@zenparsing

Javascript is a big, huge tent, filled with many people that don't consider OOP "legacy". We need to consider everyone under that tent when designing syntax.

Let me clarify what I mean by legacy. I don't mean legacy today, I mean legacy after this feature lands and is adopted by the community.

This feature will be so transformative for the community, it's hard to over estimate. Currently if you want a library to support some functionality you need to open a pull request and add a method to their prototype. This feature allows us to do all of that in user land by just piping their library into our own functions.

Inevitably that will happen.

That is going to affect the status quo, and we should design for that reality not for what is currently normal.

If we have to rewrite your example with the parentheses, it doesn't look as attractive:

I think it still looks fine, and it keeps this language feature simple. Let's keep the semantics simple and ship it.

@zenparsing

This comment has been minimized.

Copy link
Member

zenparsing commented Jan 23, 2018

@JAForbes Can you provide an example, using the semantics of this proposal, that shows how it is unergonomic for some use cases?

@bakkot

This comment has been minimized.

Copy link

bakkot commented Jan 23, 2018

I don't mean legacy today, I mean legacy after this feature lands and is adopted by the community.

I really don't think this feature will drive OO programing out of JavaScript.

Currently if you want a library to support some functionality you need to open a pull request and add a method to their prototype.

See the interfaces proposal for a different approach to this problem.

@JAForbes

This comment has been minimized.

Copy link

JAForbes commented Jan 23, 2018

@bakkot I didn't say it would drive OO programming out of JS. I'm saying the release of this operator will change ecosystem norms for data transforms. You can still have classes, and OO and all that stuff, but for data transformation |> will have a massive affect on the community. Designing for an ecosystem that doesn't have |> yet is a mistake. Libraries will change, assumptions will change, patterns will be built entirely around this feature because it's that much of a big deal.

E.g. waiting for Array utilities to land becomes less of a concern, we can do that in userland. Or waiting for Promise, Observable, Iterator functions etc. We can add features immediately and get the same convenience we are used to with chaining. It relieves pressure on library authors, on language designers, it gives more power to the community. We're going to see the pipeline being used in all kind of innovative contexts e.g. decorating hyperscript and jsx. It's hard to predict beyond that, but this feature is a huge deal and it will have drastic affects on the JS ecosystem.

So let's design for that. Just supporting the core functionality will bring a lot of value with very little risk. I'm not at all saying OO will become legacy. I get why you are reading what I'm saying that way, and I apologise sincerely for being unclear but that is not what I am trying to communicate at all. I just want to make sure we don't bake in things that make the feature confusing for an imaginary use case when a simpler proposal provides so much value and is so much easier to teach and adopt.

Designing languages is like designing infrastructure for a city. You need to plan for the future. Future economies, future culture, future populations, future needs. I'm saying, let's not bind ourselves to our assumptions about usage based on the present day, let's keep our options open and add affordances for await and methods if the community needs it. In the mean time, arrow functions work just fine and there's a whole suite of libraries that are made to work perfectly with this feature. Let's ship the simplest possible operator, and give ourselves room to extend if we need it.

@JAForbes

This comment has been minimized.

Copy link

JAForbes commented Jan 23, 2018

@zenparsing

Can you provide an example, using the semantics of this proposal, that shows how it is unergonomic for some use cases?

I think your proposal is a brilliant design for the constraints that have been presented. It really is elegant. But I want to ensure that there is a legitimate need for a compromise.

The common case for pipeline execution is composing sync functions. This proposal solves for a lot of contexts but sacrifices the absolute simplicity of x |> f |> g |> h, and instead it becomes x |> f($) |> g($) |> h($). It's not at all unreadable, or unergonomic. But its crossing a threshold of simplicity that I would want to avoid unless it's absolutely necessary. It makes it that much scarier for beginners, that much more intimidating.

If it turns out in #86 there are some solid justifications, I'd support your proposal because I think it's probably the best compromise that could be concocted. But I don't want to compromise without justification. And I'd really like us to ship the simplest possible operator.

@gilbert

This comment has been minimized.

Copy link
Collaborator

gilbert commented Jan 23, 2018

But its crossing a threshold of simplicity that I would want to avoid unless it's absolutely necessary.

I think you need to define what you mean here. "Simple" is often treated as an always-good attribute instead of the tradeoff that it truly is. Aren't function parameters simple? Then why complicated them with default values? Aren't variable assignments simple? Then why complicate them with destructuring? Aren't script tags simple? Then why complicate things with modules?

Reconsidering the meaning of the word, both the original and this Hack-style proposal are equally simple. They both have a simple rule: "call the right-hand side" and "create a binding for the right-hand side", respectively. It just so happens that Hack-style is more ergonomic for a much broader variety of use cases, but at the cost of being more verbose for the simplest use case (unary function calls) Edit: and also curried functions, to be fair.

@gilbert

This comment has been minimized.

Copy link
Collaborator

gilbert commented Jan 23, 2018

Going back to pointing out issues with this proposal, what happens when the temporary identifier is not used on the right hand side? Example:

var result = x |> f($) |> g

Should this result in a syntax error, or would result now point to the function g? Or even crazier, would the parser detect this and invoke g like in the original proposal?

I think this is important to address since I imagine some will forget to pass $ into a righthand expression.

@bakkot

This comment has been minimized.

Copy link

bakkot commented Jan 23, 2018

I agree with the concerns about the choice of binding.

That aside, I want to point out another nice synthesis (with a proposed feature): this works nicely with do expressions. For example:

anArray
 |> pickEveryN($, 2)
 |> do { let [fst, snd, ...rest] = $; [snd, fst, ...rest]; } 
 |> $.map(whatever)

Maybe that do would be better extracted to a function, but I think that's true of most uses of do expressions, so whatever.

@JAForbes

This comment has been minimized.

Copy link

JAForbes commented Jan 23, 2018

@gilbert

I think you need to define what you mean here.

Yeah good point. I think this proposal is simple in the abstract. It's design is simple. But it's design also inherits the complexity of the surrounding language, unlike a traditional |> which only can compose functions.

Additionally in terms of actual engine implementation and in terms of the things a user can do with this new syntax, it's more complex. There will be reams of edge cases and blog posts demystifying it's use in corner cases, because it's wide open in what it permits. There'll be deopts for years to come because it will be so hard to predict behavior. Where as in lining a composition pipeline is far more straight forward.

You are right simplicity is a trade off. I think this design is elegant given the constraints. But that's why I'm questioning the justification for those constraints in #86.

I also do not want to have to include a placeholder for every composition, particular when my functions are already unary anyway and I use composition exhaustively. I'll have ? all over my code base to support a use case I don't have, and that I'm not convinced is a good idea anyway. It feels like a repeat of A+ where we forever inherit awkward code because of a supposed need the community didn't ask for.

Yes shipping |> without await may inhibit future designs, but it doesn't rule out the possibility of supporting await. Especially if await expressions are banned from pipeline in the interim.

@isiahmeadows

This comment has been minimized.

Copy link
Contributor

isiahmeadows commented Jan 23, 2018

Edit: Made a few small clarifications.

Here's my thoughts on await:

  1. Why special case it to partial application, if it only works within async functions?
  2. It could easily be "shimmed"* for non-async contexts through const then = f => async (...args) => f(...args), although this wouldn't translate well to things like generators/etc.

Now, to resolve 1, you could alter await behavior to work this way:

  • x |> (await f) does what you'd expect: it awaits f and invokes x with the awaited result.
    • Parentheses are required if you want to directly call the previously awaited result.
  • x |> await f is equivalent to await f(x).
    • This keeps partial application separate from the pipeline operator.
  • x |> await f is only valid within async functions for obvious reasons.

I know that complicates a potential use case, but it does so in favor for another more frequent one.

Similarly, I'll propose a few more useful additions:

  • x |> yield f is equivalent to yield f(x), available in generators only.
  • x |> yield* f is equivalent to yield* f(x), available in generators only.
  • x |> do ..., etc. perform the right hand side (whether it be a function, await, yield, or yield*), but return the argument passed in, not the result.
    • This lets you do things like call a void function or yield a result mid-flight, while still being able to continue the pipeline with the same value.
      • This would come in handy with array manipulation and DOM manipulation.
    • There's substantial precedent in libraries:
      • Consider the common tap function/method available in numerous utility libraries (like Lodash, RxJS, Underscore, Bluebird, among others).
      • A lot of each utility functions return the instance iterated.
      • Consider the vast number of fluent chaining APIs (notably jQuery here).
    • If you've got a better keyword than do, I'm flexible. (There is void, a little-known keyword that could see a little more use, but that's two more characters I have to type...)
    • Note: this can only be implemented as a function if await/yield/etc. aren't available within pipelines.

* I mean this exceptionally loosely, in the sense that es5-sham "shims" Object.getOwnPropertyDescriptor.

@zenparsing

This comment has been minimized.

Copy link
Member

zenparsing commented Jan 23, 2018

@isiahmeadows

The "Hack-style" proposal here isn't trying to introduce any kind of partial application to the pipeline operator. Under this "Hack-style" proposal, the pipe operator simply creates a constant variable binding for the left operand when evaluating the right operand. Also, see #83 for a longer discussion on the await syntax as it fits into the current (implicit-call) semantics.

@JAForbes

I'll have ? all over my code base to support a use case I don't have

But other people will have those use cases. Should we make the syntax unergonomic/confusing for them?

Also, are we sure that we even need syntax for the implicit-call semantics? If you are really using composition for everything, can we use a function for that?

let result = purePipe(anArray,
  pickEveryN(2),
  filter(f),
  shuffle,
  map(f),
  Promise::all, // Future method-extraction syntax
  then(xs => xs.forEach(console.log)),
);

You could even combine that with |>:

anArray 
  |> purePipe($)
  |> $( pickEveryN(2),
        filter(f),
        shuffle,
        map(f) )
  |> Promise.all($)
  |> await $
  |> $.forEach(console.log);
@zenparsing

This comment has been minimized.

Copy link
Member

zenparsing commented Jan 23, 2018

@gilbert

I think this is important to address since I imagine some will forget to pass $ into a righthand expression.

For this reason I would say that a syntax error is probably the best option.

@mAAdhaTTah

This comment has been minimized.

Copy link
Contributor

mAAdhaTTah commented Jan 23, 2018

Is a placeholder required for unary functions? Is there a reason we couldn't allow mixed-use? Modified example from OP w/ ? instead:

anArray
  |> pickEveryN(?, 2)
  |> ?.filter(...)
  |> shuffle // Doesn't require the ?
  |> ?.map(...)
  |> Promise.all // neither does this
  |> await ? // maybe even drop it here?
  |> ?.forEach(console.log);

Are we mostly concerned here about developer confusion?

@js-choi

This comment has been minimized.

Copy link
Contributor

js-choi commented Feb 2, 2018

The second-to-last comment I made above shows a bunch of rough equivalences between nested Hack/binding-style pipelines and nested IIFEs. This isn’t totally correct, though. I had forgotten that await only affects the innermost function, and because of that await messes up the IIFE equivalency.

It’s more accurate if you use do expressions instead: Thus if we pretend that ^^ and _^^ a valid variable name, then the following are equivalent:

x |> f(^^)
( ^^ => f(^^) )(x)
do { const ^^ = x; f(^^) }
(10 |> ^^ + 1) |> ^^ + 2
( ^^ => ^^ + 2 )( ( ^^ => ^^ + 1 )( 10 ) ) // => 13
do {
  const ^^ = do {
    const ^^ = 10;
    do {
      const _^^ = ^^;
      do { const ^^ = _^^; ^^ + 1 }
    };
  };
  ^^ + 2
} // => also 13
10 |> (^^ + 1 |> ^^ + 2)
( ^^ => ( ^^ => ^^ + 2 )( ^^ + 1 ) )( 10 ) // => also 13
do {
  const ^^ = 10;
  do {
    const _^^ = ^^;
    do { const ^^ = _^^ + 1; ^^ + 2 }
  }
} // => also 13
10 |> (^^ + 1 |> ^^ + 2) |> ^^ + 0
( ^^ => ^^ )( ( ^^ => ( ^^ => ^^ + 2 )( ^^ + 1 ) )( 10 ) ) // => also 13
do {
  const ^^ = do {
    const ^^ = 10;
    do {
      const _^^ = ^^;
      do { const ^^ = _^^ + 1; ^^ + 2 }
    }
  };
  ^^ + 0
} // => also 13
10 |> function () { return ^^ + 1 + 2 } |> ^^()
( ^^ => ^^() )( ( ^^ => function () { return ^^ + 1 + 2 } )( 10 ) ) // => also 13
do {
  const ^^ = do {
    const ^^ = 10;
    function () { return ^^ + 1 + 2 }
  };
  ^^()
} // => also 13
10 |> function () { return ^^ + 1 |> ^^ + 2 } |> ^^()
( ^^ => ^^() )( ( ^^ => function () { return ( ^^ => ^^ + 2 )( ^^ + 1 ) } )( 10 ) ) // => also 13
do {
  const ^^ = do {
    const ^^ = 10;
    function () {
      return do {
        const _^^ = ^^;
        do {
          const ^^ = _^^ + 1; ^^ + 2
        }
      }
    }
  };
  ^^()
} // => also 13
value |> somethingWithCallback(^^, data => data |> ^^.filter(...))
( ^^ => somethingWithCallback(^^, data => ( ( ^^.filter(...) )( data ) )) )( value )
do {
  const ^^ = value;
  somethingWithCallback(^^, data => do {
    const ^^ = data; ^^.filter(...)
  })
}
value |> g(^^, await (f(x => ^^) |> ^^ + 1 |> h(^^)))
// Nested IIFEs would not work here.
do {
  const ^^ = value;
  g(^^, await do {
    const _^^ = ^^;
    do {
      const ^^ = f(x => _^^);
      do {
        const _^^ = ^^;
        do { const ^^ = _^^ + 1; h(^^) }
      }
    }
  })
}

Here, a dummy variable _^^ is used only because declarations that shadow an outer lexical context’s variable cannot use that outer context’s variable of the same name in its assignment; the declaration immediately shadows the variable of the same name with an undefined variable. In contrast, IIFEs separate the argument values from the parameter-variable declarations; the argument values are in the outer lexical context and the parameter-variable declarations are in the inner context.

@pygy

This comment has been minimized.

Copy link
Contributor

pygy commented Feb 2, 2018

value
  |> g(^^, await (
    f(x => ^^)
      |> ^^ + 1
      |> h(^^)
    ))

Assuming that ^^ is lexically scoped to the RHS of the pipes (keeping in line with left-associativity), it desugars to this in F# style:

value
  |> (_1 => g(_1, await (
    f(x => _1)
      |> (_2 => _2 + 1)
      |> (_3 => f(_3))
  )))

which is equivalent to this after inlining expressions:

( _1 => g(
  )(value);

or

do {
  const _1 = value;
  g(
    _1,
    await h((f(x => _1) + 1))
  );
}

The only places where inlining isn't possible is the cases where there are multiple references to ^^ within a given scope.
Then you'd have to desugar to an IIFE or a do expression.

FWIW, if #23 (comment) were to be accepted, that pipeline could be expressed as

value
  |> x => g(x, await (
    f(_ => x)
      |> y => y + 1
      |> h
    ))

F#-style which is more readable to me because the bindings are explicit. Those ^^ and assorted |: or |>> delve too deep for me into Sigilistan.

Edit: formating and typos

@js-choi

This comment has been minimized.

Copy link
Contributor

js-choi commented Feb 2, 2018

@pygy: The only places where inlining isn't possible is the cases where there are multiple references to ^^ within a given scope.
Then you'd have to desugar to an IIFE or a do expression.

Your analysis is correct. Assuming the absence of do expressions, Hack-style pipes are more powerful than F#-style pipes, in that the latter cannot inline RHS arrow functions with multiple references to their LHS’s value, without creating new function objects—and without doing more complex movement of autogenerated lexical bindings to the innermost surrounding valid block, without disturbing the original pipe expression’s result in situ. Without creating new function objects or moving autogenerated lexical bindings outward, do expressions would be required: e.g., x |> y => [y, y] “inlined” into do { _1 = x; [_1, _1] }.

@pygy: F#-style which is more readable to me because the bindings are explicit.

The problem with requiring all placeholder bindings in non-call expressions to be explicit is precisely the same problem as requiring all F#-style pipes to have placeholders. It’s really verbose for every step. That would be why F#-style pipes might still be standardized even if Hack-style pipes were also standardized (as #89 formally proposes). The terseness goal applies to both.

@pygy: Those ^^ and assorted |: or |>> delve too deep for me into Sigilistan.

The Hack-style pipe need not be |:, and the placeholder need not be ^^. Those are bikeshedding problems; there may well be alternative tokens that are more readable and less Sigilistani. Although I don’t know whether said bikeshedding should go in this issue or a new issue.


For people reading in the future: @gilbert has created a summary wiki page of the current proposals. There is also a new issue (#89) specifically regarding mixing the Hack-style/parameterized/binding pipe and the F#-style/tacit/call pipe.

@ljharb

This comment has been minimized.

Copy link
Member

ljharb commented Feb 3, 2018

Is there any use case for having only hack-style pipelining and not using a placeholder?

@zenparsing

This comment has been minimized.

Copy link
Member

zenparsing commented Feb 3, 2018

@ljharb I don't think so - in such a case you'd essentially be starting a new pipe. It should probably be a syntax error if you don't use the binding. Is that what you were thinking, or something else?

@ljharb

This comment has been minimized.

Copy link
Member

ljharb commented Feb 3, 2018

Yep - i would hope it’s a syntax error.

@rpamely

This comment has been minimized.

Copy link

rpamely commented Feb 3, 2018

Besides the this identifier, is there any other precedence of implicitly bound values in JavaScript? I cannot recall any, which makes the hack syntax quite exceptional for the language.

@js-choi

This comment has been minimized.

Copy link
Contributor

js-choi commented Feb 3, 2018

The placeholder variable would be at least somewhat less “weird” than this, in that it would come from a simple compile-time lexical binding, like with let and const—rather than a special runtime call-site binding, like with this. In that lexical sense, at least, there is predecence. But, I don’t recall any other other precedent for the concept of implicit binding. Edit: Of course, super, arguments, new.target, and import.meta exist…Those are also precedent. There is also no predecent for the tacit calling in the F#-style pipeline either, for what it’s worth…

I’m hoping that, by using an invalid identifier that is well understood not to be a normal variable, the Hack style would still be clear, unambiguous, and readable. I don’t think this is as true as for making the placeholder an already-valid identifier such as $…but hopefully there would be an invalid identifier prettier than ^^. Maybe ><, since <> would conflict with JSX and legacy E4X.

@ljharb

This comment has been minimized.

Copy link
Member

ljharb commented Feb 3, 2018

super, arguments, new.target, import.meta?

@stephaneraynaud

This comment has been minimized.

Copy link

stephaneraynaud commented Feb 6, 2018

From what I understand, in the Hack-style proposal:
LHS |> RHS is defined as ($ => eval(RHS))(LHS)

while in the F#-style proposal:
LHS |> RHS is defined as RHS(LHS)

I think that's why people talk about "simplicity" of the F#-style proposal.

@js-choi

This comment has been minimized.

Copy link
Contributor

js-choi commented Feb 6, 2018

@bisouduperou: That is not completely true. Proposal 2: Hack Style Only would be equivalent to nested do expressions. eval is not involved. This would naturally enable function-scoped expressions such as await, yield, and yield *, which would not work with nested arrow functions. For instance, if we pretend that a ## nullary operator is the Hack-pipe placeholder (cf. #91 for bikeshedding) but that ## is also bindable using const (which it wouldn’t), and if we make |> left associative, then these would be exactly equivalent:

LHS |> RHS
do { const ## = LHS; RHS }
x + 1 |> await ##
do { const ## = x + 1; await ## }
x |> await ## |> [0, ##, ## * 2]
do { const ## = do { const ## = x + 1; await ## }; [0, ##, ## * 2] }

This syntactic transformation is branchless; there are no syntactic special cases; its cyclomatic complexity is 1.

In contrast, Proposal 1: F-sharp Style Only would indeed often be equivalent to RHS(LHS)…but it would require special casing of LHS |> await (and maybe other types of expressions like LHS |> .RHS, as has been proposed in #75 (comment) and #23 (comment)). So these would be equivalent:

LHS |> RHS

LHS.RHSWithoutPeriod // if RHS starts with “.”
await LHS // else if RHS is “await”
RHS(LHS) // else

…which is not as conceptually simple and uniform as Proposal 2’s transform. It demands a syntactic cyclomatic complexity of n, where n is the number of special cases for the RHS. If only await is accomodated, then this syntax has a cyclomatic complexity of 2 without accomodating methods, instanceof, arithemetic operations, yield, etc. If await and methods are accomodated (as in the example above), then syntactic cyclomatic complexity is 3, while still not accomodating instanceof, arithemetic operations, yield, etc.

Proposal 3: Split Mix would be equivalent to the above two syntaxes for their respective operators, although the F-sharp pipe operator probably would lose its special cases, since they would be made useless by the separate Hack pipe operator. Each of the two pipe operator’s cyclomatic complexity would be 1, at the cost of having two pipe operators.

Proposal 4: Smart Mix would be equivalent to:

LHS |> RHS
do { const ## = LHS; RHS } // if RHS contains “##”
RHS(LHS) // else

I personally think Proposal 4 is a good compromise, with a syntactic cyclomatic complexity of only two, while keeping RHS(LHS) pithy, but also accomodating any possible expression, and adding one not two new operators (not including a placeholder nullary operator). But I could be wrong in my opinion. I just want the syntax to have versatility without complex, ad-hoc special casing.

@mAAdhaTTah

This comment has been minimized.

Copy link
Contributor

mAAdhaTTah commented Feb 6, 2018

@js-choi FWIW, I'm not in love with the .method() syntax for F# because of the additional complexity you mention. While we do special case await (and potentially other unary-like operators, so not instanceof, but maybe import or typeof), thinking about those like functions reduces how difficult it is to learn / use them.

(As an aside, I'm not sure I understand the usage of "cyclomatic" complexity here, which I colloquially understood as being about # of potential code paths in a piece of code.)

@js-choi

This comment has been minimized.

Copy link
Contributor

js-choi commented Feb 6, 2018

@mAAdhaTTah: Sorry; by “cyclomatic complexity” of the syntax I’m speaking from the parser/compiler’s perspective (as well as the programmer’s perspective while they reason about what their code means). Each special case of the syntactic transformation is yet another parsing-time branch that makes the syntax more expressive but also makes it more complex to parse and to reason about. The syntax should balance between simplicity of parsing (for both compiler and programmer) and expressiveness/versatility, as well as it can.

@gilbert

This comment has been minimized.

Copy link
Collaborator

gilbert commented Feb 7, 2018

FWIW I ran into another case where this would be useful: usage with the keyword new.

var school_uids = await Group.findAll(...)
  |> $$.map(g => g.school_uid)
  |> new Set($$)
@MrVichr

This comment has been minimized.

Copy link

MrVichr commented May 2, 2018

Hi all,
I'm not an expert on language design, but find it ridiculous that after 3 years there is still no agreement on the syntax. I admit I haven't read everything, but I did quite a bit and there doesn't seem to be any chance of a compromise.
Can we just write down the use cases, sort them by priority and try to satisfy as many as possible? I tried it, and concluded that a placeholder is necessary for all but the simplest cases.

After giving it some more thought, I've arrived at something similar to Hack-style:

  1. Every function has a local variable named ?, initialized with caller's ?.
  2. Every semicolon or newline sets this variable to the value of previous expression.
  3. Because JavaScript doesn't allow standalone RHS, |> can be used to initialize ?. The pipe operator isn't even necessary.

That's pretty much it. I'm not saying this is the best proposal ever, but anyone can learn it in a minute, and it can do pretty much anything.

//parameter piping
1; twice(?); console.log(?)  // 2

//this piping
[3, 2, 1]; ?.sort(); console.log(?)  // [1, 2, 3]

//multipiping not possible (easily)
1; 2; let x=?+?  // 4 not 3

let mult = by => ? * by
4; mult(5)  // ? = 20

I think it should be possible to make a new ? at every { block } or even at every ( expression ) that's not a function's parameters.

// in case anyone finds this useful
3; (1; ? + 2 ) * ?  //? = (1+2)*3

It might even replace the ::, if the to-be-bound functions are rewritten to operate on ? rather than this. Borrowing the example from #107,

function doubleSay() { return ? + ", " + ?; }
function capitalize() { return ?[0].toUpperCase() + ?.substring(1); }
function exclaim() { return ? + '!'; }

"hello"; doubleSay(); capitalize(); let result=exclaim(); 

result // "Hello, hello!"

One less-than-awesome feature is that the injected ? gets overwritten after first line, but you can make a backup copy if needed.

@charmander

This comment has been minimized.

Copy link

charmander commented May 2, 2018

Because JavaScript doesn't allow standalone RHS

It does.

@masaeedu

This comment has been minimized.

Copy link

masaeedu commented Sep 9, 2018

I haven't read the full thread very closely, but to respond to "doesn't handle multi argument functions", those can be handled quite easily by uncurrying and uncurrying. For example:

const add = x => y => x + y

// A multi-argument function
const lift2 = f => xs => ys => xs.reduce((p, x) => [...p, ...ys.map(f(x))], [])

const curry = f => x => y => f([x, y])
const uncurry = f => ([x, y]) => f(x)(y)

const xs = [1, 2, 3]
const ys = [4, 5, 6]

// Still works fine with the pipeline operator
const result = [xs, ys] |> uncurry(lift2(add))
console.log(result)
// => [ 5, 6, 7, 6, 7, 8, 7, 8, 9 ]

Note that uncurrying into arrays like this isn't the only valid form of uncurrying, a very frequently used pattern in JS is to collapse a function that has 20 different arguments into a function that takes a single argument with 20 different properties.

@masaeedu

This comment has been minimized.

Copy link

masaeedu commented Sep 9, 2018

I'd also like to propose a new $= operator for people who want a metavariable like syntax to represent the previous result in a pipeline. I've taken the liberty of going through all the browsers and implementing it, so you can use it in all browsers today! (you're welcome)

Let's translate @js-choi's example from above:

let timer =
  planets
  |> pickRandom($)
  |> "Hello " + $
  |> setTimeout(() => console.log($), 1000);

, which can be translated to:

let timer = (
  $= planets,
  $= pickRandom($),
  $= "Hello " + $,
  setTimeout(() => console.log($), 1000)
)

The nice thing about the $= operator is that after much careful design and analysis on my part, I was able to get it to work well with all the other JS language features!

// Works nicely with new, await etc.!
let googlecom = async () => (
  $= await fetch("https://www.google.com"),
  $= await $.text(),
  new DOMParser().parseFromString($, "text/html")
)

googlecom().then(console.log, console.error)

Now that this new operator is available in major browsers, we can have a compromise where the |> operator can be used whenever we want to avoid referring to $ everywhere, and the $= operator can be used when we prefer the pointed style.

@kurtmilam

This comment has been minimized.

Copy link

kurtmilam commented Sep 9, 2018

@masaeedu you can already do something similar to $= with the sequence operator:

const greetRandom = async ($) => (
    $= await pickRandom($),
    $= "Hello " + $,
    setTimeout(() => console.log($), 1000)
)
greetRandom(['mercury', 'venus', 'earth', 'mars'])

It's more of a pipe than a pipeLine, but I find it handy. h/t @gilbert , although I'm having trouble finding the repo where he outlined this syntax.

I included async ... await just to illustrate that it works.

@masaeedu

This comment has been minimized.

Copy link

masaeedu commented Sep 9, 2018

@kurtmilam Ah, yes, that's quite nice (although to clarify, you can already do something identical to the $= operator using the $= operator; the $= operator works in browsers today). We should open two more competing proposals called the $= operator and the "smart" $= operator.

@kurtmilam

This comment has been minimized.

Copy link

kurtmilam commented Sep 9, 2018

Maybe all we need is a new sequence operator. AFAIK, the main complaint about the current one is that it's confusing to developers, since the comma means different things in different contexts.

@gilbert

This comment has been minimized.

Copy link
Collaborator

gilbert commented Sep 10, 2018

@masaeedu Duplicate of this comment :) That issue also links to the proposal you're thinking of, @kurtmilam

@kurtmilam

This comment has been minimized.

Copy link

kurtmilam commented Sep 10, 2018

@gilbert That's the one - thanks for linking it!

As a fan of the F# pipeline proposal, I'd really like to know whether there are any good arguments against @masaeedu 's suggestion that TC39 go with an F# style pipeline and recommend the existing sequence operator for all of the additional operations a Hack style pipeline would enable.

In other words, I don't see $= as just a bit of silly fun. Rather, it seems to me that everything(?) a Hack style pipeline would enable is already possible using the existing sequence operator.

@ljharb

This comment has been minimized.

Copy link
Member

ljharb commented Sep 10, 2018

$ is an identifier, and thus imo is not available for use in an operator.

@pygy

This comment has been minimized.

Copy link
Contributor

pygy commented Sep 10, 2018

@ljharb Coffee? (-;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment