Draft proposal for smart pipelines, including formal specification.
Switch branches/tags
Nothing to show
Clone or download

readme.md

Smart pipelines

ECMAScript Stage-0 Proposal. Living Document. J. S. Choi, 2018-02.

📖 Table of Contents


This document is an explainer for the formal specification of a proposed smart pipe operator |> in JavaScript, along with several other additional features. The specification is divided into one Stage-0 Core Proposal plus six mutually independent-but-compatible Additional Features:

Name Status Features Purpose
Core Proposal Stage 0 Infix pipelines … |> …
Lexical topic #
Unary function/expression application
Additional Feature BC None Bare constructor calls … |> new … Tacit application of constructors
Additional Feature BA None Bare awaited calls … |> await … Tacit application of async functions
Additional Feature BP None Block pipeline steps … |> {…} Application of statement blocks
Additional Feature PF None Pipeline functions +> Partial function/expression application
Function/expression composition
Method extraction
Additional Feature TS None Pipeline try statements Tacit application to caught errors
Additional Feature NP None N-ary pipelines (…, …) |> …
Lexical topics ##, ###, and ...
N-ary function/expression application

The Core Proposal is currently at Stage 0 of the TC39 process and is planned to be presented, along with a competing proposal, to TC39 by Daniel “littledan” Ehrenberg of Igalia. The Core Proposal is a variant of the first pipe-operator proposal also championed by Ehrenberg; this variant is listed as Proposal 4: Smart Mix in the pipe-proposal wiki. The variant resulted from previous discussions in the previous pipe-operator proposal, discussions which culminated in an invitation by Ehrenberg to try writing a specification draft.

The additional features are not part of the Stage-0 Core Proposal. They are included to illustrate possible separate follow-up proposals for the case in which the Core Proposal advances past Stage 1. Together, the Core Proposal and the additional features demonstrate a unified vision of a future in which composition, partial application, method extraction, and error handling are all tersely expressible with the same simple pipeline/topic concept.

An update to the existing pipeline Babel plugin is also being developed jointly between the author of this proposal and James DiGioia, the author of the competing proposal. The update will support both this proposal and the other proposal, configurable with a flag.

You can take part in discussions on the GitHub issue tracker. When you file an issue, please note in it that you are talking specifically about “Proposal 4: Smart Mix”.

This specification uses # as its topic reference. However, this is not set in stone. In particular, @ or ? could also be used. Bikeshedding discussions over what characters to use for the topic token has been occurring on GitHub at tc39/proposal-pipeline-operator issue #91.

The Core Proposal is formally specified in in the draft specification.

Motivation

This section gives a brief overview of the motivations behind the smart pipe operator’s Core Proposal, as well the additional features listed above. Examples from real-world libraries are juxtaposed with their original versions. The original versions have been lightly edited (e.g., breaking up lines, removing semicolons), in order to fit their horizontal widths into this table. Examples that use additional features are included only to illustrate the power of the pipeline/topic concept and are always simply rewritable into forms that use only the Core Proposal.

With smart pipelines Status quo

The infix “smart” pipe operator |> proposed here would provide a backwards- and forwards-compatible style of chaining nested expressions into a readable, left-to-right manner.

Using a zero-cost abstraction, nested data transformations become untangled into short steps.

console.log(
  await stream.write(
    new User.Message(
      capitalize(
        doubledSay(
          await promise
            || throw new TypeError(
              `Invalid value from ${promise}`)
        ), ', '
      ) + '!'
    )
  )
);

Nested, deeply composed expressions occur often in JavaScript. They occur whenever any single value must be processed by a series of data transformations, whether they be operations, functions, or constructors. Unfortunately, these deeply nested expressions often result in messy spaghetti code, due to their mixing of prefix, infix, and postfix expressions together. Writing such code requires many nested levels of indentation and parentheses. Reading such code requires checking both the left and right of each subexpression to understand its data flow.

promise
|> await #
|> # || throw new TypeError(
  `Invalid value from ${promise}`)
|> doubleSay(#, ', ')
|> capitalize
|> # + '!'
|> new User.Message(#)
|> await stream.write(#)
|> console.log;

With smart pipelines, the code above becomes terser and, literally, more straightforward. Prefix, infix, and postfix expressions would be less tangled together in threads of spaghetti. Instead, data values would be piped from left to right through a single flat thread of postfix expressions, with a single level of indentation and four fewer pairs of parentheses – essentially forming a reverse Polish notation.

The resulting code’s terseness and flatness may be both easier for the JavaScript developer to read and to edit. This uniform postfix notation preserves locality between related code; the reader may follow the flow of data more easily through this single flattened thread of postfix operations. And the developer may more easily add or remove operations at the beginning, end, or middle of the thread, without changing the indentation of unrelated lines.

console.log(
  await stream.write(
    new User.Message(
      capitalize(
        doubledSay(
          await promise
            || throw new TypeError(
              `Invalid value from ${promise}`)
        ), ', '
      ) + '!'
    )
  )
);

Compared with the pipeline version, the original code requires additional indentation and grouping on each step. This requires four more levels of indentation and four more pairs of parentheses.

In addition, much related code is here separated by unrelated code. Rather than a uniform postfix chain, operations appear either before the previous step’s expression (await stream.write(…),new User.Message(…), capitalize(…), doubledSay(…), await …) but also after (… || throw new TypeError(), … + '!'). An additional argument to function calls (such as , in doubledSay(…, ', ')) is also separated from its function calls, forming another easy-to-miss “postfix” argument.

A pipeline is made of a head expression, followed by a chain of postfix expressions called pipeline stages. Each stage has its own inner lexical scope, within which a special topic reference # is defined. This # is a reference to the lexical topic of the pipeline (# itself is called a topic reference).

input
|> # + 1 // step 1
|> f(x, #, y) // step 2
|> await g(#, z) // step 3
|> console.log(`${#}!`); // step 4
  1. The head expression to the left of the pipeline steps is first evaluated.
  2. It then is inputted into pipeline step 1, becoming that step’s lexical topic. A new lexical environment is created, scoped only to pipeline step 1, and within which # is immutably bound to the topic. Using that topic binding, the first pipeline step is then evaluated; the current lexical environment is then reset back to before.
  3. The result of the first pipeline step becomes the input to step 1. A new lexical environment is created, scoped only to pipeline step 2, and whose topic binding is the result of evaluating step 1.
  4. And so forth, until step 4 is evaluated, with the result of step 3 as its input. The result of step 4 is the result of the entire pipeline.
console.log(`${ // step 4
  await g( // step 3
    f(x, // step 2
      input + 1, // step 1
      y), // step 2
    z) // step 3
}!`); // step 4
input |> (# = 50);
// 🚫 Reference Error:
// Cannot assign to topic reference.

The topic binding is immutable, established only once per lexical environment. It is an error to attempt to assign a value to it using =, whether inside or outside a pipeline step.

This chained pipeline:

input
|> # - 3
|> -#
|> # * 2
|> Math.max(#, 0)
|> console.log;

…is equivalent to the tangled nested expression:

console.log(
  Math.max(
    -(input - 3) * 2,
    0
  )
);

The syntax is statically term rewritable into already valid code in this way, with theoretically zero runtime cost.

Similar use cases appear numerous times in JavaScript code, whenever any input is transformed by expressions of any type: function calls, property calls, method calls, object constructions, arithmetic operations, logical operations, bitwise operations, typeof, instanceof, await, yield and yield *, and throw expressions.

promise
|> await #
|> # || throw new TypeError()
|> doubleSay(#, ', ')
|> capitalize
|> # + '!'
|> new User.Message(#)
|> await stream.write(#)
|> console.log;

Note that, in the example above, it is not necessary to include the parenthesized argument (#) for capitalize and console.log. They were tacitly implied, forming a tacit unary function call. In other words, the example above is equivalent to the version in the next row.

console.log(
  await stream.write(
    new User.Message(
      capitalize(
        doubledSay(
          await promise
            || throw new TypeError(
              `Invalid value from ${promise}`)
        ), ', '
      ) + '!'
    )
  )
);
promise
|> await #
|> # || throw new TypeError(
    `Invalid value from ${#}`)
|> doubleSay(#, ', ')
|> capitalize(#)
|> # + '!'
|> new User.Message(#)
|> await stream.write(#)
|> console.log(#);

This version is equivalent to the version above, except that the |> capitalize(#) and |> console.log(#) pipeline steps explicitly include optional topic references #, making the expressions slightly wordier than necessary.

console.log(
  await stream.write(
    new User.Message(
      capitalize(
        doubledSay(
          await promise
            || throw new TypeError(
              `Invalid value from ${promise}`)
        ), ', '
      ) + '!'
    )
  )
);

Being able to automatically detect this bare style is the smart part of the “smart pipe operator”. The styles of functional programming, dataflow programming, and tacit programming may particularly benefit from bare pipelines and their terse function application.

const object = input
|> f
|> # + 2
|> # * 3
|> -#
|> g(#, x)
|> o.unaryMethod
|> await asyncFunction(#)
|> await o.asyncMethod(#)
|> new Constructor(#);

This pipeline is a very flat expression, with only one level of indentation, and with each transformation step on its own line.

Note that … |> f is a bare unary function call. This is the same as … |> f(#), but the topic reference # is unnecessary; it is invisibly, tacitly implied. The same goes for o.unaryMethod, which is a unary function call on o.unaryMethod.

This is the smart part of the smart pipe operator, which can distinguish between two syntax styles (bare style vs. topic style) by using a simple rule: bare style uses only identifiers and dots – and never parentheses, brackets, braces, or other operators. And topic style always contains at least one topic reference. For more information, see the reference below about the smart step syntax.

const object =
  new Constructor(
    await o.asyncMethod(
      await asyncFunction(
        o.unaryMethod(
          g(
            -(f(input) + 2)
              * 3,
            x
          )
        )
      )
    )
  );

In contrast to the version with pipes, this code is deeply nested, not flat.

The expression has two levels of indentation instead of one. Reading its data flow requires checking both the beginning and end of each expression, and each step expression gradually increases in size.

Inserting or removing any step of the data flow also requires changes to the indentation of any previous steps’ lines.

input |> x + 50 |> f |> g(x, 2);
// 🚫 Syntax Error:
// Topic-style pipeline step
// `|> x + 50`
// binds topic but contains
// no topic reference.
// 🚫 Syntax Error:
// Topic-style pipeline step
// `|> g(x, 2)`
// binds topic but contains
// no topic reference.

In order to fulfill the goal of “don’t shoot me in the foot”, when a pipeline step is in topic style but it contains no topic reference, that is an early error. Such a degenerate pipeline step has a very good chance of actually being an accidental bug. (Note that the bare-style pipeline step |> f is not an error. The bare style is not supposed to contain any topic references #.)

For instance, this code may be clear enough:

input |> object.method;

It is a valid bare-style pipeline. Bare style is designed to be strictly simple: it must either be a simple reference or it is not in bare style.

It means:

object.method(input);

But this code would be less clear. That is why it is an early Syntax Error:

input |> object.method();
// 🚫 Syntax Error:
// Topic-style pipeline step
// `|> object.method()`
// binds topic but contains
// no topic reference.

It is an invalid topic-style pipeline. It is in topic style because it is not a simple reference; it has parentheses. And it is invalid because it is in topic style yet it does not have a topic reference.

Had that code not been an error, it could reasonably mean either of these lines:

object.method(input);
object.method()(input);

Instead, the developer must clarify what they mean, using a topic reference, into either of these two valid topic-style pipelines:

input |> object.method(#);
input |> object.method()(#);

The reading developer benefits from explicitness and clarity, without sacrificing the benefits of untangled flow that pipelines bring.

object.method(input);
object.method()(input);

Adding other arguments:

input |> object.method(x, y);
// 🚫 Syntax Error:
// Topic-style pipeline step
// `|> object.method(x, y)`
// binds topic but contains
// no topic reference.

…would make this problem of semantic ambiguity worse. But the reader is protected from this ambiguity by the same early error.

That code could have any of these reasonable interpretations:

object.method(input, x, y);
object.method(x, y, input);
object.method(x, y)(input);

Both inserting the input as the first argument and inserting it as the last argument are reasonable interpretations, as evidenced by how other programming languages’ pipe operators variously do either. Or it could be a factory method that creates a function that is in turn to be called with a unary input argument.

The writer must clarify which of these reasonable interpretations is correct:

input |> object.method(#, x, y);
input |> object.method(x, y, #);
input |> object.method(x, y)(#);
object.method(input, x, y);
object.method(x, y, input);
object.method(x, y)(input);

And this example’s ambiguity would be even worse:

input |> await object.method(x, y);
// 🚫 Syntax Error:
// Topic-style pipeline step
// `|> await object.method(x, y)`
// binds topic but contains
// no topic reference.

…were it not an invalid topic-style pipeline.

It could reasonably mean any of these lines:

await object.method(input, x, y);
await object.method(x, y, input);
await object.method(x, y)(input);
(await object.method(x, y))(input);

So the developer must clarify their intent using one of these lines:

input |> await object.method(#, x, y);
input |> await object.method(x, y, #);
input |> await object.method(x, y)(#);
input |> (await object.method(x, y))(#);
await object.method(input, x, y);
await object.method(x, y, input);
await object.method(x, y)(input);
(await object.method(x, y))(input);
function doubleSay (str, separator) {
  return `${str}${separator}${string}`;
}

function capitalize (str) {
  return str[0].toUpperCase()
    + str.substring(1);
}

promise
|> await #
|> # || throw new TypeError()
|> doubleSay(#, ', ')
|> capitalize
|> # + '!'
|> new User.Message(#)
|> await stream.write(#)
|> console.log;

This pipeline is also relatively flat, with only one level of indentation, and with each transformation step on its own line.

… |> capitalize is a bare unary function call equivalent to … |> capitalize(#).

function doubleSay (str, separator) {
  return `${str}${separator}${str}`;
}

function capitalize (str) {
  return str[0].toUpperCase()
    + str.substring(1);
}

console.log(
  await stream.write(
    new User.Message(
      capitalize(
        doubledSay(
          await promise
            || throw new TypeError(
              `Invalid value from ${promise}`)
        ), ', '
      ) + '!'
    )
  )
);

This deeply nested expression has four levels of indentation instead of two. Reading its data flow requires checking both the beginning of each expression (new User.Message, capitalizedString, doubledSay, await promise and end of each expression (|| throw new TypeError(), , ', ', + '!')).

x =|> f(#, #);
x =|> [#, # * 2, # * 3];

The topic reference may be used multiple times in a pipeline step. Each use refers to the same value (wherever the topic reference is not overridden by another, inner pipeline’s topic scope). Because it is bound to the result of the topic, the topic is still only ever evaluated once.

{
  const $ = …;
  x = f($, $);
}
{
  const $ = …;
  x = [$, $ * 2, $ * 3];
}

This is equivalent to assigning the topic value to a unique variable, then using that variable multiple times in an expression.

promise
|> await #
|> # || throw new TypeError()
|> `${#}, ${#}`
|> #[0].toUpperCase() + #.substring(1)
|> # + '!'
|> new User.Message(#)
|> stream.write
|> console.log;

When tiny functions are only used once, and when their bodies would be obvious and self-documenting in meaning, then they might be ritual boilerplate that a developer may prefer to inline: trading off self-documentation for localization of code.

{
  const promiseValue = await promise
    || throw new TypeError();
  const doubledValue =
    `${promiseValue}, ${promiseValue}`;
  const capitalizedValue
    = doubledValue[0].toUpperCase()
      + doubledValue.substring(1);
  const exclaimedValue
    = capitalizedValue + '!';
  const userMessage =
    new User.Message(exclaimedValue);
  const writeValue =
    stream.write(userMessage);
  console.log(writeValue);
}

Using a sequence of variables instead has both advantages and disadvantages. The variable names may be self-documenting. But they also are verbose. They visually distract from the crucial data transformations (overemphasizing the expressions’ nouns over their verbs), and it is easy to typo their names.

promise
|> await #
|> # || throw new TypeError()
|> normalize
|> `${#}, ${#}`
|> #[0].toUpperCase() + #.substring(1)
|> # + '!'
|> new User.Message(#)
|> await stream.write(#)
|> console.log;

With a pipeline, there are no unnecessary variable identifiers. Inserting a new step in between two steps (or deleting a step) only touches one new line. Here, a call of a function normalize was inserted between the second and third steps.

{
  const promiseValue = await promise
    || throw new TypeError();
  const normalizedValue = normalize();
  const doubledValue =
    `${normalizedValue}, ${normalizedValue}`;
  const capitalizedValue =
    doubledValue[0].toUpperCase()
      + doubledValue.substring(1);
  const exclaimedValue =
    capitalizedValue + '!';
  const userMessage =
    new User.Message(exclaimedValue);
  const writeValue =
    stream.write(userMessage);
  console.log(writeValue);
}

This code underwent a similar insertion of normalize. With a series of variables, inserting a new step in between two other steps (or deleting a step) requires editing the variable names in the following step.

input |> f |> [0, 1, 2, ...#] |> g;

A topic-style pipeline step may contain array literals. These may be flattened, just like any other sort of expression.

g([0, 1, 2, ...f(input)]);

A topic-style pipeline step may also contain object literals. However, pipeline steps that are entirely object literals must be parenthesized. It is similar to how arrow functions distinguish between object literals and blocks.

input |> f |> ({ x: #, y: # }) |> g;

This fulfills the goal of forward compatibility with Additional Feature BP, which introduces block pipeline steps. (It is expected that block pipelines would eventually be much more common than pipelines with object literals.)

input |> f |> { x: #, y: # } |> g;
// 🚫 Syntax Error:
// Unexpected token `{`.
// Cannot parse base expression.
{
  const $ = f(input);
  g({ x, $: y: f($) });
}
f = input
|> f
|> (x => # + x);

A topic-style pipeline step may contain an inner arrow function. Both versions of this example result in an arrow function in a closure on the previous pipeline’s result input |> f.

{
  const $ = f(input);
  x => $ + x;
}

The arrow function lexically closes over the topic value, takes one parameter, and returns the sum of the topic value and the parameter.

input
|> f
|> settimeout(() => # * 5)
|> processIntervalID;

This ability to create arrow functions, which do not lexically shadow the topic, can be useful for using callbacks in a pipeline.

{
  const $ = f(input);
  const intervalID = settimeout(() => $ * 5);
  processIntervalID(intervalID);
}

The topic value of the second pipeline step (here represented by a normal variable $) is still lexically accessible within its body, an arrow function, in both examples.

input
|> f
|> (() => # * 5)
|> settimeout
|> processIntervalID;

The arrow function can also be created on a separate pipeline step.

{
  const $ = f(input);
  const callback = () => $ * 5;
  const intervalID = settimeout(callback);
  processIntervalID(intervalID);
}

The result here is the same.

input
|> f
|> () => # * 5
|> settimeout
|> processIntervalID;
// 🚫 Syntax Error:
// Unexpected token `=>`.
// Cannot parse base expression.

Note, however, that arrow functions have looser precedence than the pipe operator. This means that if a pipeline creates an arrow function alone in one of its steps, then the arrow-function expression must be parenthesized. (The same applies to assignment and yield operators, which are also looser than the pipe operator.) The example above is being parsed as if it were:

(input |> f |> ()) =>
  (# * 5 |> settimeout |> processIntervalID);
// 🚫 Syntax Error:
// Unexpected token `=>`.
// Cannot parse base expression.

The arrow function must be parenthesized, as with any other looser-precedence expression:

input
|> (f, g)
|> (() => # * 5)
|> settimeout
|> processIntervalID;

Both the head and the steps of a pipeline may contain nested inner pipelines.

x = input
|> f(x =>
  # + x |> g |> # * 2)
|> #.toString();

A nested pipeline works consistently. It merely shadows the outer context’s topic with the topic within its own steps’ inner contexts.

{
  const $ = input;
  x = f(x =>
    g($ + x) * 2
  ).toString();
}
x = input
|> # ** 2
|> f(x => #
  |> g(#, x)
  |> [# * 3, # * 5]);
{
  const $ = input ** 2;
  x = f(x => {
    const _$ = g($, x);
    return [_$ * 3, _$ * 5];
  });
}

Four kinds of statements cannot use an outside context’s topic in their expressions. These are:

  • function definitions (including those for async functions, generators, and async generators; but not arrow functions, as explained above),
  • class definitions,
  • for and while statements,
  • catch clauses (but see Additional Feature TS), and
  • with statements.

This behavior is in order to fulfill the goals of simple scoping and of “don’t shoot me in the foot”: it prevents the origin of any topic from being difficult to find. It also fulfills the goal of forward compatibility with future additional features.

x = input |> function () { return #; };
// 🚫 Syntax Error:
// Lexical context `function () { return #; }`
// contains a topic reference
// but has no topic binding.
// 🚫 Syntax Error:
// Pipeline step `|> function () { … }`
// binds topic but contains
// no topic reference.
x = input |> class { m: () { return #; } };
// 🚫 Syntax Error:
// Pipeline step `|> class { … }`
// binds topic but contains
// no topic reference.
x = input
|> await f(#, 5)
|> () => {
  if (#)
    return # + 30;
  else
    return #;
}
|> g;

Any other nested blocks may contain topic references from outer lexical environments. These include arrow functions, if statements, try statements and their finally clauses (though not their catch clauses), switch statements, and bare block statements.

x = g(await f(input, 5) + 30);

A function definition that is a pipeline step may contain topic references in its default parameters’ expressions, because their scoping is similar to that of the outside context’s: similar enough such that also allowing topic references in them would fulfill the goal of simple scoping. However, as stated above, the function body itself still may not contain topic references.

value
|> processing
|> function (x = #) { return x; }
function (x = processing(value)) {
  return x;
}

The same applies to the parenthesized antecedents of for and while loops.

input
|> process
|> (x, y) => {
  for (const element of #)
    …
}
input
|> process
|> (x, y) => {
  let element;
  while (element = getNextFrom(#))
    …
}
(x, y) => {
  for (const element
    of process(input))
    …
}
(x, y) {
  let element;
  while (element =
    getNextFrom(input))
    …
}

WHATWG Fetch Standard (Core Proposal only)

The WHATWG Fetch Standard contains several examples of using the DOM fetch function, resolving its promises into values, then processing the values in various ways. These examples may become more easily readable with smart pipelines.

With smart pipelines Status quo
'/music/pk/altes-kamuffel'
|> await fetch(#)
|> await #.blob()
|> playBlob;
fetch('/music/pk/altes-kamuffel')
  .then(res => res.blob())
  .then(playBlob);
playBlob(
  await (
    await fetch('/music/pk/altes-kamuffel')
  ).blob()
);
'https://example.com/'
|> await fetch(#, { method: 'HEAD' })
|> #.headers.get('content-type')
|> console.log;
fetch('https://example.com/',
  { method: 'HEAD' }
).then(response =>
  console.log(
    response.headers.get('content-type'))
);
'https://example.com/'
|> await fetch(#, { method: 'HEAD' })
|> #.headers.get('content-type')
|> console.log;
console.log(
  (await
    fetch('https://example.com/',
      { method: 'HEAD' }
    )
  ).headers.get('content-type')
);
'https://example.com/'
|> await fetch(#, { method: 'HEAD' })
|> #.headers.get('content-type')
|> console.log;
{
  const url = 'https://example.com/';
  const response =
    await fetch(url, { method: 'HEAD' });
  const contentType =
    response.headers.get('content-type');
  console.log(contentType);
}
'https://pk.example/berlin-calling'
|> await fetch(#, { mode: 'cors' })
|> (
  #.headers.get('content-type')
  |> # && #
    .toLowerCase()
    .indexOf('application/json')
    >= 0
  )
  ? #
  : throw new TypeError()
|> await #.json()
|> processJSON;
fetch('https://pk.example/berlin-calling',
  { mode: 'cors' }
).then(response => {
  if (response.headers.get('content-type')
    && response.headers.get('content-type')
      .toLowerCase()
      .indexOf('application/json') >= 0
  )
    return response.json();
  else
    throw new TypeError();
}).then(processJSON);

jQuery (Core Proposal only)

As the single most-used JavaScript library in the world, jQuery has provided an alternative human-ergonomic API to the DOM since 2006. jQuery is under the stewardship of the JS Foundation, a member organization of TC39 through which jQuery’s developers are represented in TC39. jQuery’s API requires complex data processing that becomes more readable with smart pipelines.

With smart pipelines Status quo
return data
|> buildFragment([#], context, scripts)
|> #.childNodes
|> jQuery.merge([], #);

The path that a reader’s eyes must trace while reading this pipeline moves straight down, with some movement toward the right then back: from data to buildFragment (and its arguments), then .childNodes, then jQuery.merge. Here, no one-off-variable assignment is necessary.

parsed = buildFragment(
  [ data ], context, scripts
);
return jQuery.merge(
  [], parsed.childNodes
);

From jquery/src/core/parseHTML.js. In this code, the eyes first must look for data – then upwards to parsed = buildFragment (and then back for buildFragment’s other arguments) – then down, searching for the location of the parsed variable in the next statement – then right when noticing its .childNodes postfix – then back upward to return jQuery.merge.

(key |> toType) === 'object';
key |> toType |> # === 'object';

|> has a looser precedence than most operators, including ===. (Only assignment operators, arrow function =>, yield operators, and the comma operator are any looser.)

toType(key) === 'object';

From jquery/src/core/access.js.

context = context
|> # instanceof jQuery
    ? #[0] : #;
context =
  context instanceof jQuery
    ? context[0] : context;

From jquery/src/core/access.js.

context
|> # && #.nodeType
  ? #.ownerDocument || #
  : document
|> jQuery.parseHTML(match[1], #, true)
|> jQuery.merge;
jQuery.merge(
  this, jQuery.parseHTML(
    match[1],
    context && context.nodeType
      ? context.ownerDocument
        || context
      : document,
    true
  )
);

From jquery/src/core/init.js.

match
|> context[#]
|> (this[match] |> isFunction)
  ? this[match](#);
  : this.attr(match, #);

Note how, in this version, the parallelism between the two clauses is very clear: they both share the form match |> context[#] |> something(match, #).

if (isFunction(this[match])) {
  this[match](context[match]);
} else
  this.attr(match, context[match]);
}

From jquery/src/core/init.js. Here, the parallelism between the clauses is somewhat less clear: the common expression context[match] is at the end of both clauses, at a different offset from the margin.

elem = match[2]
|> document.getElementById;
elem = document.getElementById(match[2]);

From jquery/src/core/init.js.

// Handle HTML strings
if (…)
  …
// Handle $(expr, $(...))
else if (!# || #.jquery)
  return context
  |> # || root
  |> #.find(selector);
// Handle $(expr, context)
else
  return context
  |> this.constructor
  |> #.find(selector);

The parallelism between the final two clauses becomes clearer here too. They both are of the form return context |> something |> #.find(selector).

// Handle HTML strings
if (…)
  …
// Handle $(expr, $(...))
else if (!context || context.jquery)
  return (context || root).find(selector);
// Handle $(expr, context)
else
  return this.constructor(context)
    .find(selector);

From jquery/src/core/init.js. The parallelism is much less clear here.

Underscore.js (Core Proposal only)

Underscore.js is another utility library very widely used since 2009, providing numerous functions that manipulate arrays, objects, and other functions. It too has a codebase that transforms values through many expressions – a codebase whose readability would therefore benefit from smart pipelines.

With smart pipelines Status quo
function (obj, pred, context) {
  return obj
  |> isArrayLike
  |> # ? _.findIndex : _.findKey
  |> #(obj, pred, context)
  |> (# !== void 0 && # !== -1)
      ? obj[#] : undefined;
}
function (obj, pred, context) {
  var key;
  if (isArrayLike(obj)) {
    key = _.findIndex(obj, pred, context);
  } else {
    key = _.findKey(obj, pred, context);
  }
  if (key !== void 0 && key !== -1)
    return obj[key];
}
function (obj, pred, context) {
  return pred
  |> cb
  |> _.negate
  |> _.filter(obj, #, context);
}
function (obj, pred, context) {
  return _.filter(obj,
    _.negate(cb(pred)),
    context
  );
}
function (
  srcFn, boundFn, ctxt, callingCtxt, args
) {
  if (!(callingCtxt instanceof boundFn))
    return srcFn.apply(ctxt, args);
  var self = srcFn
  |> #.prototype |> baseCreate;
  return self
  |> srcFn.apply(#, args)
  |> _.isObject(#) ? # : self;
}
function (
  srcFn, boundFn,
  ctxt, callingCtxt, args
) {
  if (!(callingCtxt instanceof boundFn))
    return srcFn.apply(ctxt, args);
  var self = baseCreate(srcFn.prototype);
  var result = srcFn.apply(self, args);
  if (_.isObject(result)) return result;
  return self;
}
function (obj) {
  return obj
  |>  # == null
    ? 0
    : #|> isArrayLike
    ? #|> #.length
    : #|> _.keys |> #.length;
  };
}

Smart pipelines make parallelism between all three clauses becomes clearer:
0 if it is nullish,
#|> #.length if it is array-like, and
#|> something |> #.length otherwise.
(Technically, #|> #.length could simply be #.length, but it is written in this redundant form in order to emphasis its parallelism with the other branch.)

This particular example becomes even clearer when paired with Additional Feature BP.

function (obj) {
  if (obj == null) return 0;
  return isArrayLike(obj)
    ? obj.length
    : _.keys(obj).length;
}

Lodash (Core Proposal only)

Lodash is a fork of Underscore.js that remains under rapid active development. Along with Underscore.js’ other utility functions, Lodash provides many other high-order functions that attempt to make functional programming more ergonomic. Like jQuery, Lodash is under the stewardship of the JS Foundation, a member organization of TC39, through which Lodash’s developers also have TC39 representation. And like jQuery and Underscore.js, Lodash’s API involves complex data processing that becomes more readable with smart pipelines.

With smart pipelines Status quo
function hashGet (key) {
  return this.__data__
  |> nativeCreate
    ? (#[key] === HASH_UNDEFINED
      ? undefined : #)
    : hashOwnProperty.call(#, key)
    ? #[key]
    : undefined;
}
function hashGet (key) {
  var data = this.__data__;
  if (nativeCreate) {
    var result = data[key];
    return result === HASH_UNDEFINED
      ? undefined : result;
  }
  return hasOwnProperty.call(data, key)
    ? data[key] : undefined;
}
function listCacheHas (key) {
  return this.__data__
  |> assocIndexOf(#, key)
  |> # > -1;
}
function listCacheHas (key) {
  return assocIndexOf(this.__data__, key)
    > -1;
}
function mapCacheDelete (key) {
  const result = key
  |> getMapData(this, #)
  |> #['delete']
  |> #(key);
  this.size -= result ? 1 : 0;
  return result;
}
function mapCacheDelete (key) {
  var result =
    getMapData(this, key)['delete'](key);
  this.size -= result ? 1 : 0;
  return result;
}
function castPath (value, object) {
  return value |>
    #|> isArray
    ? #
    : (#|> isKey(#, object))
    ? [#]
    : #|> toString |> stringToPath;
}
function castPath (value, object) {
  if (isArray(value)) {
    return value;
  }
  return isKey(value, object)
    ? [value]
    : stringToPath(toString(value));
}

Smart step syntax

Most pipeline steps will use topic references in their steps. This style of pipeline step is called topic style.

For three simple cases – unary functions, unary async functions, and unary constructors – you may omit the topic reference from the pipeline step. This is called bare style.

When a pipe is in bare style, we refer to the pipeline as a bare function call. (If Additional Feature BC or Additional Feature BA are used, then a bare-style pipeline step may instead be a bare awaited function call, or a bare constructor call, depending on the rules of bare style.) The step acts as just a simple reference to a function, such as with … |> capitalize or with … |> console.log. The step’s value would then be called as a unary function, without having to use the topic reference as an explicit argument.

The two bare-style productions require no parameters, because they can only be simple references, made up of identifiers and dots .. (If Additional Feature BC is used, then the simple reference may optionally be preceded by new: … |> new o.C. If Additional Feature BA is used, then the simple reference may optionally be preceded by await: … |> new o.af. Even with Additional Features BC or BA, new and await may not be used on their own without a simple reference: … |> o.C |> new 🚫 and … |> o.af |> await 🚫 are invalid pipelines. Instead, use either the bare style … |> new o.C and … |> await o.af, or use topic style: … |> af |> await #.

With the Core Proposal only:

Valid topic style Valid bare style Invalid pipeline
… |> f(#) … |> f … |> f() 🚫
″″ ″″ … |> (f) 🚫
″″ ″″ … |> (f()) 🚫
… |> o.f(#) … |> o.f … |> o.f() 🚫
… |> o.f(arg, #) const f = $ => o::f(arg, $); … |> f … |> o.f(arg) 🚫
… |> o.make()(#) const f = o.make(); … |> f … |> o.make() 🚫
… |> o[symbol](#) const f = o[symbol]; … |> f … |> o[symbol] 🚫

With Additional Feature BC:

Valid topic style Valid bare style Invalid pipeline
… |> new C(#) … |> new C … |> new C() 🚫
″″ ″″ … |> (new C) 🚫
″″ ″″ … |> (new C()) 🚫
″″ ″″ … |> new (C) 🚫
″″ ″″ … |> new (C()) 🚫
… |> new o.C(#) … |> new o.C … |> new o.f() 🚫
… |> new o.C(arg, #) const f = $ => new o::C(arg, $); … |> f … |> new o.C(arg) 🚫
… |> new o.make()(#) const C = o.make(); … |> new C … |> new o.make() 🚫
… |> new o[symbol](#) const f = new o[symbol]; … |> f … |> new o[symbol] 🚫
… |> await new o.make()(#) const af = new o.make(); … |> await af … |> new await o.make() 🚫

With Additional Feature BA:

Valid topic style Valid bare style Invalid pipeline
… |> await af(#) … |> await af … |> await af() 🚫
″″ ″″ … |> (await f) 🚫
″″ ″″ … |> (await f()) 🚫
″″ ″″ … |> await (f) 🚫
″″ ″″ … |> await (f()) 🚫
… |> af |> await # ″″ … |> af |> await 🚫
… |> await o.f(#) … |> await o.f … |> await o.f() 🚫
… |> await o.make()(#) const af = o.make(); … |> await af … |> await o.make() 🚫
… |> await new o.make()(#) const af = new o.make(); … |> await af … |> new await o.make() 🚫

Bare style

The bare style supports using simple identifiers, possibly with chains of simple property identifiers. If there are any operators, parentheses (including for method calls), brackets, or anything other than identifiers and dot punctuators, then it is in topic style, not in bare style.

If the pipeline step is a merely a simple reference, then that identifier is interpreted to be a bare function call. The pipeline’s value will be the result of evaluating the step as an identifier or member expression, then calling the result as a function, with the current topics as its arguments.

That is: if a pipeline is of the form
topic |> identifier
or topic |> identifier0.identifier1
or topic |> identifier0.identifier1.identifier2
or so forth,
then the pipeline is a bare function call.

With Additional Feature BC:
If a pipeline step starts with new, followed by a mere identifier, optionally with a chain of properties, and with no parentheses or brackets, then that identifier is interpreted to be a bare constructor.

That is: if a pipeline is of the form
topic |> new identifier
or topic |> new identifier0.identifier1
or topic |> new identifier0.identifier1.identifier2
or so forth,
then the pipeline is a bare constructor call.

With Additional Feature BA:
If a pipeline step starts with await, followed by a mere identifier, optionally with a chain of properties, and with no parentheses or brackets, then that identifier is interpreted to be a bare awaited function call.

That is: if a pipeline is of the form
topic |> await identifier
or topic |> await identifier0.identifier1
or topic |> await identifier0.identifier1.identifier2
or so forth,
then the pipeline is a bare async function call.

Topic style

The presence or absence of topic tokens (#, ##, ###) is not used in the grammar to distinguish topic style from bare style to fulfill the goal of syntactic locality. Instead, if a pipeline step of the form |> step does not match the bare style (that is, it is not a bare function call, bare async function call, or bare constructor call), then it must be in topic style. Topic style requires that there be a topic reference in the pipeline step; otherwise it is an early error.

A pipeline step that is not in bare style is usually a topic expression. This is any expression at the precedence level once tighter than pipeline-level expressions – that is, any conditional-level expression.

With Additional Feature BA:
A pipeline step that is a block { … } containing a list of statements is a topic block. The last statement in the block is used as the result of the whole pipeline, similarly to do expressions.

Practical consequences

Therefore, a pipeline step in bare style never contains parentheses (…) or brackets […]. Neither … |> object.method() nor … |> object.method(arg) nor … |> object[symbol] nor … |> object.createFunction() are in bare style (in fact, they all are Syntax Errors, due to their being in topic style without any topic references).

When a pipeline step needs parentheses or brackets, then don’t use bare style, and instead use a topic reference in the step (topic style)…or assign the step to a variable, then use that variable as a bare-style step.

Operator precedence and associativity

As a infix operation forming compound expressions, the operator precedence and associativity of pipelining must be determined, relative to other operations.

Precedence is tighter than arrow functions (=>), assignment (=, +=, …), generator yield and yield *, and sequence ,; and it is looser than every other type of expression. If the pipe operation were any tighter than this level, its steps would have to be parenthesized for many frequent types of expressions. However, the result of a pipeline is also expected to often serve as the body of an arrow function or a variable assignment, so it is tighter than both types of expressions.

All operation-precedence levels in JavaScript are listed here, from tightest to loosest. Each level may contain the parse types listed for that level – as well as any expression types from any precedence level that is listed above it.

Level Type Form Associativity / fixity
Primary This this Nullary
″″ Primary topic # ″″
″″ Secondary topic ## ″″
″″ Tertiary topic ### ″″
″″ Rest topic ... ″″
″″ Identifiers a ″″
″″ Null null ″″
″″ Booleans true false ″″
″″ Numerics 0 ″″
″″ Arrays […] Circumfix
″″ Object {…} ″″
″″ Function function (…) {…} ″″
″″ Classes class … {…} ″″
″″ Generators function * (…) {…} ″″
″″ Async functions async function (…) {…} ″″
″″ Regular expression /…/… ″″
″″ Templates …`…` ″″
″″ Parentheses (…) Circumfix
LHS Static properties ….… ″″
″″ Dynamic properties …[…] LTR infix with circumfix
″″ Tagged templates …`…` Unchainable infix with circumfix
″″ Super call op.s super(…) Unchainable prefix
″″ Super properties super.… ″″
″″ Meta properties meta.… ″″
″″ Object construction new … Prefix
″″ Function call …(…) LTR infix with circumfix
Postfix unary Postfix incrementing …++ Postfix
″″ Postfix decrementing …-- ″″
Prefix unary Prefix incrementing ++… RTL prefix
Prefix unary Prefix decrementing --… ″″
″″ Deletes delete … ″″
″″ Voids void … ″″
″″ Unary +/- +… ″″
″″ Bitwise NOT ~… ~… ″″
″″ Logical NOT !… !… ″″
″″ Awaiting await … ″″
Exponentiation Exponentiation … ** … RTL infix
Multiplicative Multiplication … * … LTR infix
″″ Division … / … ″″
″″ Modulus … % … ″″
Additive Addition … + … ″″
″″ Subtraction … - … ″″
Bitwise shift Left shift … << … ″″
″″ Right shift … >> … ″″
″″ Signed right shift … >> … ″″
Relational Greater than … < … ″″
″″ Less than … > … ″″
″″ Greater than / equal to … >= … ″″
″″ Less than / equal to … <= … ″″
″″ Containment … in … ″″
″″ Instance-of … instanceof … ″″
Equality Abstract equality … == … ″″
″″ Abstract inequality … != … ″″
″″ Strict equality … === … ″″
″″ Strict inequality … !== … ″″
Bitwise AND … & … ″″
Bitwise XOR … ^ … ″″
Bitwise OR … | … ″″
Logical AND … ^^ … ″″
Logical OR … || … ″″
Conditional … ? … : … RTL ternary infix
Pipeline Pipelines … |> … LTR infix
Assignment Pipeline functions +> … Prefix
″″ Async pipeline functions async +> … Prefix
″″ Arrow functions … => … RTL infix
″″ Async arrow functions async … => … RTL infix
″″ Assignment … = … ″″
″″ … += … ″″
″″ … -= … ″″
″″ … *= … ″″
″″ … %= … ″″
″″ … **= … ″″
″″ … <<= … ″″
″″ … >>= … ″″
″″ … >>>= … ″″
″″ … &= … ″″
″″ … |= … ″″
Yield Yielding yield … Prefix
″″ Flat yielding yield * … ″″
″″ Spreading ...… ″″
Comma level Comma …, … LTR infix
Base statements Expression statements …; Postfix with ASI
″″ Empty statements ; Nullary with ASI
″″ Debugger statements debugger; ″″
″″ Block statements {…} Circumfix
″″ Labelled statements …: … Prefix
″″ Continue statements continue …; Circumfix with ASI
″″ Break statements break …; ″″
″″ Return statements return …; ″″
″″ Throw statements throw …; ″″
″″ Variable statements var …; ″″
″″ Lexical declarations let …; ″″
″″ ″″ const …; ″″
″″ Hoistable declarations function … (…) {…} Circumfix with prefix
″″ ″″ async function … (…) {…} ″″
″″ ″″ function * … (…) {…} ″″
″″ ″″ async function * … (…) {…} ″″
″″ Class declarations class … {…} ″″
Compound statements If statements if (…) … else … Circumfix with prefix
″″ Switch statements switch (…) … ″″
″″ Iteration statements ″″
″″ With statements with (…) {…} ″″
″″ Try statements try {…} catch (…) {…} finally {…} ″″
Statement list Case clause case: … Unchainable prefix
Root Script Root
″″ Module ″″

Appendices

Additional Feature BC

See Additional Feature BC.

Additional Feature BA

See Additional Feature BA.

Additional Feature BP

See Additional Feature BP.

WHATWG Fetch Standard (Core Proposal + Additional Feature BP)

See Additional Feature BP.

jQuery (Core Proposal + Additional Feature BP)

See Additional Feature BP.

Lodash (Core Proposal + Additional Feature BP)

See Additional Feature BP.

Additional Feature TS

See Additional Feature TS.

Additional Feature PF

See Additional Feature PF.

Ramda (Core Proposal + Additional Feature BP+PF)

See Additional Feature PF.

WHATWG Streams Standard (Core Proposal + Additional Features BP+PP+PF)

See Additional Feature PF.

Additional Feature NP

See Additional Feature NP.

Lodash (Core Proposal + Additional Features BP+PP+PF+NP)

See Additional Feature NP.

Ramda (Core Proposal + Additional Features BP+PF+NP)

See Additional Feature NP.

WHATWG Streams Standard (Core Proposal + Additional Features BP+PP+PF+NP)

See Additional Feature NP.

Goals

See Goals.

“Don’t break my code.”

See “Don’t break my code”.

Backward compatibility

See Backward compatibility.

Zero runtime cost

See Zero runtime cost.

Forward compatibility

See Forward compatibility.

“Don’t shoot me in the foot.”

See “Don’t shoot me in the foot”.

Opt-in behavior

See Opt-in behavior.

Simple scoping

See Simple scoping.

Static analyzability

See Static analyzability.

“Don’t make me overthink.”

See “Don’t make me overthink”.

Syntactic locality

See Syntactic locality.

Semantic clarity

See Semantic clarity.

Expressive versatility

See Expressive versatility.

Cyclomatic simplicity

See Cyclomatic simplicity.

“Make my code easier to read.”

See “Make my code easier to read”.

Untangled flow

See Untangled flow.

Distinguishable punctuators

See Distinguishable punctuators.

Terse parentheses

See Terse parentheses.

Terse variables

See Terse variables.

Terse function calls

See Terse function calls.

Terse composition

See Terse composition.

Terse partial application

See Terse partial application.

Other Goals

See Other Goals.

Conceptual generality

See Conceptual generality.

Human writability

See Human writability.

Novice learnability

See Novice learnability.

Relations to other work

See Relations to other work.

Pipelines in other programming languages

See Relations to other work.

Topic references in other programming languages

See Topic references in other programming languages.

do expressions

See do expressions.

Function binding

See Function binding.

Function composition

See Function composition.

Partial function application

See Partial function application.

Optional catch binding

See Optional catch binding.

Pattern matching

See Pattern matching.

Block parameters

See Block parameters.

do expressions

See do expressions.

Explanation of nomenclature

See Nomenclature.

Term rewriting

See Term rewriting.