Add support for F#-style Pipeline Operator

[Pokute]

PR adding pipeline operator support to TypeScript. The proposal is currently a stage 1 proposal for EcmaScript. It "introduces a new operator |> similar to F#, OCaml, Elixir, Elm, Julia, Hack, and LiveScript, as well as UNIX pipes and Haskell's &. It's a backwards-compatible way of streamlining chained function calls in a readable, functional manner, and provides a practical alternative to extending built-in prototypes."

const square = (n: number) => Math.pow(n, 2);
4 |> square |> console.log; // 16

Currently this is a minimal implementation of pipeline operator, but I'm planning on making a F# variant support and after that a Topic/Smart pipeline variant PRs.

Partial application is a good pair for minimal/F# pipeline operator and has it's own PR at #37973. I'm planning on making a separate draft PR that has both features in them.

There is also a PR for Hack-style pipelines with # as a placeholder token.

Code status

This works minimally. The TS transformer generates valid JS and there's working type inference. There are many bugs, inconsistencies with specs, insufficient error messages currently. Most of failing tests are not included in this PR since Playground won't get generated with any failures and having faulty baselines is shady. I'm eagerly awaiting feedback, testing and even help with this PR. Even using and talking of these features will help advance them through TC39 proposal process!

await is currently not supported.

I used @rbuckton 's work on pipelineStage1 branch as a guide.

Testing

Playground link

To test it in your project, add following to your package.json:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/100059/artifacts?artifactName=tgz&fileId=D3836CF7DD5F6438481C5BCDBEE7A63AB21880164FEFD2BD75085B15EB974FCE02&fileName=/typescript-4.3.0-insiders.20210404.tgz"
    }
}

and then run npm install.

PR state

• There is an associated issue in the Backlog milestone (required)
• Code is up-to-date with the master branch
• You've successfully run gulp runtests locally
• There are new or updated unit tests validating the change

Why a draft PR for stage 1 proposal?

The TypeScript officially follows EcmaScript / stage 3 proposals. However, I'm opening this draft pull request despite it being only stage 1. The partial application and pipeline proposals now are in a stage where they want people to test the features - write it in their code, find problems, test existing codebases. The usual way these would be done is through a Babel plugin, but since a few years ago, just a plain text editor with syntax highlighting is no longer the default JS development environment.

TypeScript is no longer just a build step. It's used by a significant portion of JS developers for all their development, even when prototyping new code. If the tool chain of these developers requires TypeScript, then a simple Babel plugin won't help. This means that writing TypeScript with those features is impossible without first-class support for them in TypeScript.

But then could we ask them to write just plain JS instead? It turns out that TypeScript is more and more part of normal JavaScript development process with Language Server Protocol pointing out errors in code and providing typing information. Having people encounter red squigglies and disappearing type information as soon as they use the new language feature will not count favourably to the user experience. I believe having TypeScript support for new syntax proposals as important or even more important than having a Babel plugin.

I was considering creating and upkeeping a separate fork, but that would just mean duplication of work. There's already a great infrastructure for TypeScript with playgrounds, CI, etc. With a draft PR, new features can more easily get more eyes, testers and feedback. There will eventually be issues and questions on new features for TypeScript and github.com/microsoft/TypeScript is where anyone would first look up info on partial application.

Of course, I'm not expecting merging until the proposal reaches stage 3.

[orta]

@typescript-bot pack this

[typescript-bot]

Heya @orta, I've started to run the tarball bundle task on this PR at 19fd8b1. You can monitor the build here.

[typescript-bot]

Hey @orta, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/72713/artifacts?artifactName=tgz&fileId=5FDC6B0458B0184B85D790D1EB8AA47C0924EADA6FBC0575C4425A46F70973E002&fileName=/typescript-4.0.0-insiders.20200503.tgz"
    }
}

and then running npm install.

---

There is also a playground for this build.

[markusjohnsson]

Nice! I have wondered myself how pipeline operators would play with typescript, especially how the types would flow. You could have went the way of only parsing and type checking, and letting e.g. babel do translation, but since you have also done translation this is really easy to try out.

I did test it out and ran in to a type inference limitation which I hope can be fixed or it will be hard to use with complex types:

const ns = [1, 2, 3, 4, 5];

function map<T, R>(f: (t: T) => R) {
    return (ts: T[]): R[] => ts.map(f);
}

const result = ns |> map(x => x * x);

index.ts:8:16 - error TS2571: Object is of type 'unknown'.

8 ns |> map(x => x * x);

Here T must be inferred from the type of ns.

[rbuckton]

@Pokute: If it helps at all I have two old branches where I started to look into implementing pipeline as well:

• https://github.com/microsoft/TypeScript/tree/pipelineStage1 - Limited to only the pipeline operator
• https://github.com/microsoft/TypeScript/tree/rbuckton/pipeline - Pipeline, partial application, and a few other concepts. Older than pipelineStage1 and much more experimental

[Pokute]

Nice! I have wondered myself how pipeline operators would play with typescript, especially how the types would flow. You could have went the way of only parsing and type checking, and letting e.g. babel do translation, but since you have also done translation this is really easy to try out.

Writing the transformer was actually pretty simple as soon as I had the scanning & parsing in place.

I did test it out and ran in to a type inference limitation which I hope can be fixed or it will be hard to use with complex types:
Here T must be inferred from the type of ns.

I think that the form that TypeScript seems to internally handle it is map(x => x * x)(ns).
This is an interesting case where the type of a function would be resolved from it's usage and it's not directly either. I'm interested whether there are existing cases of this happening within TypeScript.

In the simplest cases, it feels like it should be possible to resolve the type. Beyond that is out of my expertise. :-)

Either way, thank you for an interesting test case!

[phaux]

Is there a way to try pipeline operator and partial application together?

[dustinlacewell]

When I found this PR I nearly cried. Thank you.

[lukewestby]

Hey folks, I stumbled upon this PR from https://github.com/tc39/proposal-pipeline-operator#build-tools. I'm not able to access the tarball mentioned here #38305 (comment) and I'm just wondering if it's been removed intentionally.

[Pokute]

Hey folks, I stumbled upon this PR from https://github.com/tc39/proposal-pipeline-operator#build-tools. I'm not able to access the tarball mentioned here #38305 (comment) and I'm just wondering if it's been removed intentionally.

At least I haven't removed the tarball. I wonder if those have limited lifetimes. Any ideas @orta?

[orta]

Yep, looks like they timeout - we're having a chat about what to do in the long term (normally this is not a problem for us because they are more active PRs), but for now I'll give it another 10-12 days.

@typescript-bot pack this

[typescript-bot]

Heya @orta, I've started to run the tarball bundle task on this PR at 19fd8b1. You can monitor the build here.

[typescript-bot]

Hey @orta, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/74014/artifacts?artifactName=tgz&fileId=53B4D99407A93B4AF6FAEF8437B02126A8C7FEC62F1C3307DAA098F75D2CE5D602&fileName=/typescript-4.0.0-insiders.20200515.tgz"
    }
}

and then running npm install.

---

There is also a playground for this build.

[mikearnaldi]

Nice! I have wondered myself how pipeline operators would play with typescript, especially how the types would flow. You could have went the way of only parsing and type checking, and letting e.g. babel do translation, but since you have also done translation this is really easy to try out.

I did test it out and ran in to a type inference limitation which I hope can be fixed or it will be hard to use with complex types:

const ns = [1, 2, 3, 4, 5];

function map<T, R>(f: (t: T) => R) {
    return (ts: T[]): R[] => ts.map(f);
}

const result = ns |> map(x => x * x);

index.ts:8:16 - error TS2571: Object is of type 'unknown'.

8 ns |> map(x => x * x);

Here T must be inferred from the type of ns.

Running in the same issue, without this sort of inference the usage of the operator is very limited, not sure if this can help but currently the only place I sow proper inference with a pipe implementation is this (not the operator a simple pipe function):
https://github.com/gcanti/fp-ts/blob/master/src/pipeable.ts#L97

[xixixao]

This is great, I might take a stab at adding the Hack-style proposal (see tc39/proposal-pipeline-operator#167 (comment) - it's the Smart proposal without the minimal proposal) to Flow.

[ken-okabe]

One of the greatest updates.
Just in case, here's the working:

 "devDependencies": {
    "@types/node": "*",
    "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/74014/artifacts?artifactName=tgz&fileId=53B4D99407A93B4AF6FAEF8437B02126A8C7FEC62F1C3307DAA098F75D2CE5D602&fileName=/typescript-4.0.0-insiders.20200515.tgz"
  }

[sgioia9]

YESSSS

[chenglou]

Funny that this is coming back full circle. I've made a comment a while ago regarding the type inference/tooling problem with the pipe, and how to solve it: tc39/proposal-pipeline-operator#143 (comment)

[Jopie64]

About that backward type inference problem:

In RxJS, TypeScript works just fine for a situation like this:

const int$ = of('hello').pipe(map(x => x.length));
// type of int$ is inferred to be Observable<number>

This would be equivalent to:

const int$ = of('hello') |> map(x => x.length);

Why would it be impossible to infer int$ in the latter case while it is inferrable in the former? In both cases map is something of type:

type Map = <T, U>(f: T => U) => Observable<T> => Observable<U>

And hence the receiver is the last curried argument. So it feels like it should be inferrable in both cases..?

[benlesh]

FYI: It appears the type inference isn't working that great for more advanced cases like RxJS see this playground example

Copied here for convenience:

interface Subscription {
  unsubscribe(): void;
}

interface Observer<T> {
  next(value: T): void;
  error(err: any): void;
  complete(): void;
}

interface Observable<T> {
  subscribe(observer: Partial<Observer<T>>): Subscription;
}

type Op<T, R> = (source: Observable<T>) => Observable<R>;

declare function map<T, R>(fn: (value: T, index: number) => R): Op<T, R>;

declare function filter<T>(fn: (value: T, index: number) => boolean): Op<T, T>;

declare const source: Observable<number>;

const result = source
  |> map(x => x + x) // <-- x is `unknown`
  |> map(x => x + '!!!') // <-- x is `unknown`
  |> filter(x => x.length < 20); // <-- x is `unknown`

This is a real shame, because it would be a killer language feature for libraries like RxJS.

[lozandier]

@benlesh @markusjohnsson Seems to be a pattern of type inference needing to be added for its use w/ generics; generic use cases don't seem to be covered yet in cases/conformance nor baselines/reference yet by @Pokute.

[ken-okabe]

@typescript-bot pack this

[orta]

@typescript-bot pack this

[typescript-bot]

Heya @orta, I've started to run the tarball bundle task on this PR at 19fd8b1. You can monitor the build here.

[kiprasmel]

Testing features like this out in a project and having to manually update the packed bundle because it expires in about 10 days is very annoying - could we get a feature to tell typescript bot to pack things indefinitely? Thanks!

/cc @orta

[orta]

@typescript-bot pack this

[typescript-bot]

Heya @orta, I've started to run the tarball bundle task on this PR at 50ce993. You can monitor the build here.

[typescript-bot]

Hey @orta, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/101226/artifacts?artifactName=tgz&fileId=52B72C8EFE9367109126FF631C5367AC4E9A15482B3AD34AB472EEEEED73464F02&fileName=/typescript-4.3.0-insiders.20210418.tgz"
    }
}

and then running npm install.

---

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@4.3.0-pr-38305-57".;

[Lonli-Lokli]

Will it be possible to use pipeline with iterators, sync and async?

[lozandier]

@Lonli-Lokli Yes, see the following examples from the proposal's github repo

The simple pipeline handles the former 2 trivially by design; F#, Smart, & Hack-style proposals build on top of the simple proposal to accommodate async functions / promises / iterables a variety of ways w/ pros & cons.

[rjhilgefort]

Is anyone aware of a working version of this PR? I tried the playground link posted by the bot, but the playground doesn't appear to be working with this branch?

https://www.staging-typescript.org/play?ts=4.3.0-pr-38305-57

[Pokute]

Is anyone aware of a working version of this PR? I tried the playground link posted by the bot, but the playground doesn't appear to be working with this branch?

There might be a problem with operator precedence. You need to wrap the inline arrow function within parenthesis: |> ((x: string) => x.toUpperCase()) |>

Thank you for testing this out, @rjhilgefort.

[rjhilgefort]

@Pokute ah, thanks, I didn't realize you had to wrap the lamda in parens. That seems to work, mostly. The compiler accepts the expression, but still has an error.

It doesn't seem to properly infer that x is a string and I would have to tell the compiler that it's a string explicitly.

Is that expected for this build? Is the plan to have that inference work?

[ken-okabe]

@Pokute ah, thanks, I didn't realize you had to wrap the lamda in parens. That seems to work, mostly. The compiler accepts the expression, but still has an error.

It doesn't seem to properly infer that x is a string and I would have to tell the compiler that it's a string explicitly.

Is that expected for this build? Is the plan to have that inference work?

I think it's natural that the pipeline operator has a high precedence.

and the error is common in typescript, nothing to do with the pipeline operator.

[Pokute]

The compiler [...] still has an error.
It doesn't seem to properly infer that x is a string and I would have to tell the compiler that it's a string explicitly.

Is that expected for this build? Is the plan to have that inference work?

and the error is common in typescript, nothing to do with with to the pipeline operator.

This should be inferable as shown in a above screenshot. It does require more work though, but I'm not sure when I can work on it.

I'm not sure about the precedence either. I'll need to check it out.

[timbuckley]

TC39 is moving forward with the Hack-proposal version (stage 1 -> 2).

Therefore, this PR should be closed in favor of the Hack-style approach #43617

[ken-okabe]

TC39 is moving forward with the Hack-proposal version (stage 1 -> 2).

Therefore, this PR should be closed in favor of the Hack-style approach #43617

I firstly thought it's good news, but investigating "hack-style" that for some reason I didn't notice (I only knew miminal/F#/Smart).

I think if a programmer knows mathematics or functional programming, the one will notice it's just a mess and night-mare.

I wrote this:

https://gist.github.com/tabatkins/1261b108b9e6cdab5ad5df4b8021bcb5#gistcomment-3892109

[ken-okabe]

I made an issue:
Inconsistency of type inference of Hack-style pipeline-operator proposal (Stage2) #46101
also an article:
TC39/proposal-pipeline-operator Hack-style |> hijacks Grouping operator ( )
https://dev.to/stken2050/the-current-tc39-proposal-pipeline-operator-hack-style-hijacks-grouping-operator-1dam

[andrewbranch]

@stken2050 please do not post any comments here that are unrelated to the specific implementation of this PR. General discussion of the pipeline proposal should be done on the pipeline proposal’s repository. Seeing as you’re already participating there, one backlink for visibility was ok, but at this point you’re bordering on spam.

[ken-okabe]

@andrewbranch Thanks for your comments,

Seeing as you’re already participating there, one backlink for visibility was ok

however, this is not exactly true, unfortunately, they are deleting comments.

[sandersn]

This experiment is pretty old, so I'm going to close it to reduce the number of open PRs.