Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin Support for Custom Transformers #14419

Open
MichaReiser opened this issue Mar 2, 2017 · 100 comments
Open

Plugin Support for Custom Transformers #14419

MichaReiser opened this issue Mar 2, 2017 · 100 comments
Assignees
Labels

Comments

@MichaReiser
Copy link

@MichaReiser MichaReiser commented Mar 2, 2017

Since #13764 has landed, it's easy to write custom transformers. However, if I understand the API correctly, it is needed to duplicate the whole tsc command line just to add a single transformer. This leads to incompatibilities since these, on typescript depending, applications do not support the same features as the tsc command line tool.

It, therefore, would be favored to have a plugin system that allows loading Custom Transformers from third party node modules as this is the case for the language service proxies #12231. These transformers then easily integrate into existing build tools / build workflows.

If someone experienced has inputs on how to implement the changes, I'm willing to create a PR as it would simplify my project tremendously.

@mhegazy
Copy link
Contributor

@mhegazy mhegazy commented Mar 2, 2017

We do not plan on exposing compiler plugin system in the short term. The transforms are exposed as part of the public API as you noted, and we are in the process of writing documentation and samples for these.

@MichaReiser
Copy link
Author

@MichaReiser MichaReiser commented Mar 2, 2017

I don't want to have an exposed plugin API. Instead, I would prefere way to register my custom transforms by just specifying them in the tsconfig.json instead of having to use the TS-Compiler API. Because using the TS-Compiler API implies that I can no longer use the tsc command line tool which further can imply that it does no longer integrate nicely into existing build tools and workflows.

@WanderWang
Copy link

@WanderWang WanderWang commented Apr 17, 2017

anything update about documentation and samples ? @mhegazy

@longlho
Copy link
Contributor

@longlho longlho commented May 3, 2017

@MichaReiser you can write a simple compiler based on the API (sample here). We actually use TS very heavily within Yahoo Finance and the transformer public API have been awesome.

Those are some of the transformers we wrote after it became public:
https://github.com/longlho/ts-transform-system-import
https://github.com/longlho/ts-transform-img
https://github.com/longlho/ts-transform-react-intl
https://github.com/longlho/ts-transform-css-modules-transform

@mhegazy lmk if you need help documenting/gathering samples

@MichaReiser
Copy link
Author

@MichaReiser MichaReiser commented May 3, 2017

@longlho
Thanks for your sample.

That is what I'm currently doing. However, it makes it impossible to use other build tools created for typescript, e.g. web pack loaders, jest loaders. Besides, how can I use one of your plugins together with one of mine when each of us uses a different frontend?

Therefore, I do believe that the current approach is a good start but not sufficient for a plugin ecosystem. Because using a plugin is too much effort for the user and requires more than just writing the transformation code for the author.

I believe a plugin system like the one of Babel is essential for typescript if the goal is to encourage the community to create and use custom transformers.

@longlho
Copy link
Contributor

@longlho longlho commented May 3, 2017

@MichaReiser yup. I'm not saying it's sufficient for the ecosystem, just good enough for the 1st step towards it.

@ivogabe
Copy link
Contributor

@ivogabe ivogabe commented May 9, 2017

I'd like to add support for transforms in gulp-typescript, but I want to prevent that all TypeScript plugins (for gulp, webpack etc) propose a different API. And TypeScript might add even a different way for configurating this later on. So do you currently have plans to add this in the near or far future?

@RubaXa
Copy link

@RubaXa RubaXa commented Jun 6, 2017

Hi all, I try to write a preprocessor, but nothing comes out :[

In fact, I have two questions:

  • How to create AST-fragment from string?
  • How to add import?
// 1. Input
class Foo {
    templateString = 'some value';
}

// 2. After transformation
import __LIB__ from '@external/lib';

class Foo {
    templateString = (function compiledTemplate(deps) {
        // ...
        return result;
    })({lib: __LIB__});
}

// 3. Expected result
var lib_1 = require("@external/lib");
var Foo = (function () {
    function Foo() {
        this.templateString = (function compiledTemplate(deps) {
            // ...
            return result;
        })({ lib: lib_1 }); 
    }
    return Foo;
}());

A Simplified Example: https://github.com/RubaXa/typescript-api-questions/tree/master/import-add

@RubaXa
Copy link

@RubaXa RubaXa commented Jun 8, 2017

⬆️ ⬆️ ⬆️
@longlho, @mhegazy Can you give any hint?

@rbuckton
Copy link
Member

@rbuckton rbuckton commented Jun 8, 2017

@RubaXa Since you added the import declaration in a transform, it isn't bound or type checked. Since it wasn't bound or type checked, we cannot resolve __LIB__ in your expression to the __LIB__ in the declaration.

One option is to use a namepace import instead of a default import, so that your emit is something like:

import * as __LIB__ from "@external/lib"

As there is no aliasing that can occur.

The other hold onto a generated identifier for the import declaration, as per the attached zip

@rbuckton
Copy link
Member

@rbuckton rbuckton commented Jun 8, 2017

@RubaXa if your goal is to pass in the entire module object, you probably want to use the import * as __LIB__ from "@external/lib" syntax (which does not require aliasing) and not import __LIB__ from "@external/lib" as the latter imports the default export rather than the entire module object.

@longlho
Copy link
Contributor

@longlho longlho commented Jun 9, 2017

yeah we're doing import * as foo in our CSS modules transformer as well to prevent aliasing. Although it'd be nice to tap into that to inline certain named exports

@RubaXa
Copy link

@RubaXa RubaXa commented Jun 9, 2017

@rbuckton O, thanks a lot!
Still interested in how to create an arbitrary AST-fragment from string?

function createFragmentFromString(code: string) {
  // ????
}

function visitPropertyDeclaration(node) {
	if (ts.isIdentifier(node.name) && node.name.text === "templateString") {
		// ...
		return ts.updateProperty(
			node,
			ts.visitNodes(node.decorators, visitor),
			ts.visitNodes(node.modifiers, visitor),
			ts.visitNode(node.name, visitor),
			ts.visitNode(node.type, visitor),
			createFragmentFromString('(function compiledTemplate() { /* ... */ })()')
		);
	}
	return node;
}
@rightisleft
Copy link

@rightisleft rightisleft commented Oct 21, 2017

This is what i was hoping the transformer support and workflow would behave like in TypeScript.

https://www.dartlang.org/tools/pub/transformers

@chandu0101
Copy link

@chandu0101 chandu0101 commented Dec 11, 2017

I am also very much interested in this feature.

as @MichaReiser mentioned
If someone experienced has inputs on how to implement the changes, I'm willing to create a PR as it would simplify my project tremendously.

love to see inputs from experts/typescript compiler DEV .

@MeirionHughes
Copy link

@MeirionHughes MeirionHughes commented Jan 30, 2018

Looks like someone has wrapped typescript to expose this functionality. https://github.com/cevek/ttypescript

Also looks like ts-loader has it: https://www.npmjs.com/package/ts-loader#getcustomtransformers-----before-transformerfactory--after-transformerfactory----

@jcimoch
Copy link

@jcimoch jcimoch commented Feb 1, 2018

I think this is something that could land in typescript 2.8 release or at least in roadmap at some point, @ahejlsberg @mhegazy @DanielRosenwasser what do you think? In this way custom transformers might be more popular and therefore more powerfull. Having option to plugin in transfromer from tsconfig.json perspective would simplify life a lot.

@mhegazy
Copy link
Contributor

@mhegazy mhegazy commented Feb 1, 2018

We have no plans to expose plugins in the short term, as noted earlier.

@raveclassic
Copy link

@raveclassic raveclassic commented Feb 1, 2018

@mhegazy Is it a considered decision or is it out of scope just because of low community interest?

@jantimon
Copy link

@jantimon jantimon commented Mar 17, 2020

There are also transforms which don't change the language but provide additional information for easier debugging.

For example the babel-plugin-transform-react-jsx-source is part of the default react transpiling preset (@babel/preset-react).
It transforms

<sometag />

Into

<sometag __source={ { fileName: 'this/file.js', lineNumber: 10 } } />

This information allows react to provide better error stack traces during development.

For typescript there is no standard way to achieve the same although there is an open source transform: https://github.com/dropbox/ts-transform-react-jsx-source

@nojvek
Copy link
Contributor

@nojvek nojvek commented Apr 18, 2020

I understand @mhegazy mentioned twice that TS team has no plan of including “plugins” as part of tsconfig. What is the reasoning behind this?

Would you be open to a PR? The way ttypescript handles the transformers via tsconfig is pretty nice.

Usecases:

  1. generating interface types to schemas so they can be runtime checked.
  2. importing json with strings as literals so typescript doesn’t barf when assigned to a type with discriminated unions.
  3. importing css/yaml and other object like syntax
  4. making graphql queries that dynamically create the right types
  5. compiling jsx to html strings so they can be innerHTML injected
  6. rewriting absolute imports to relative imports based on tsconfig paths.

The list goes on and on. The plugin api is nice. Thanks for landing it. Having “plugins” part of “tsconfig” makes tsc really powerful. Is there something big we’re missing ?

@madou
Copy link

@madou madou commented Apr 18, 2020

you can also do a lot of performance optimizations. i've been writing https://github.com/atlassian-labs/compiled-css-in-js with typescript transformers 🤙 would be cool if consumers didn't need to jump through hoops to use it though.

@pldilley
Copy link

@pldilley pldilley commented Apr 20, 2020

Dear Typescript team, please stop dragging your heels on this <_<

@imrickywong
Copy link

@imrickywong imrickywong commented Jun 22, 2020

What's the status on this?

@NSExceptional
Copy link

@NSExceptional NSExceptional commented Jul 20, 2020

Status?

@ilya-buligin
Copy link

@ilya-buligin ilya-buligin commented Nov 14, 2020

Friends what news?

@jantimon
Copy link

@jantimon jantimon commented Nov 15, 2020

The typescript team has decided against this idea - if you need flexibility or better dev tooling using typescript is not an option without hacks - go with babel instead and use typescript only for typechecking

@ilya-buligin
Copy link

@ilya-buligin ilya-buligin commented Nov 15, 2020

The typescript team has decided against this idea - if you need flexibility or better dev tooling using typescript is not an option without hacks - go with babel instead and use typescript only for typechecking

But what if I want some hacks for type-checking? I write a little transformer that makes some magic:

type Human = {
    name: string,
    age: number
}

const isValid = check<Human>({ name: 'Carl', age: 16 }) // => true
const isValid = check<Human>({ name: 'Carl' }) // => false

Function check transformed into specific type-guard function! How I can make it with Babel? Currently, I should use ttypescript...

@nonara
Copy link

@nonara nonara commented Nov 15, 2020

The typescript team has decided against this idea - if you need flexibility or better dev tooling using typescript is not an option without hacks - go with babel instead and use typescript only for typechecking

There is actually some talk of this becoming supported in the future. That said, it really is already supported. The TypeScript compiler API supports transformers, and it's really simple to write a custom compiler that uses transformers in only a few lines of code.

But in order to make it easier, you can use ttypescript or ts-patch.

These libraries aren't really a "hacks" in the sense that they are not augmenting the compiler to add transformer ability. Instead, they simply expose the existing functionality of the compiler API to tsc, making it usable via tsconfig.

@just-boris
Copy link

@just-boris just-boris commented Nov 15, 2020

@ilya-buligin if I understand your use-case correctly, you can declare an ambient constant

declare const check: <T>(value: unknown) => value is T;

const isValid = check<Human>({ name: 'Carl', age: 16 }) // => true

Typescript will compile the code relying on check type definition and then you will replace it with a generated code using Babel.

@ilya-buligin
Copy link

@ilya-buligin ilya-buligin commented Nov 19, 2020

@just-boris how I can get access to generic type <T> in Babel?

@MartinJohns
Copy link
Contributor

@MartinJohns MartinJohns commented Nov 19, 2020

This thread is getting fairly repetitive and off-topic. @RyanCavanaugh Perhaps this thread should be locked, until you have any updates on the subject.

@Eddie-CooRo
Copy link

@Eddie-CooRo Eddie-CooRo commented Jan 5, 2021

I just wanted to hop in and add another use-case.
I'm working on an application that leverages the webpack extensions config to prioritize some custom extensions over the others. So, for example, if a Hello.android.ts file exists, it will pick that up automatically when you do import Hello from "./Hello".
Right now, obviously, typescript is not very happy with it, and we have to patch our way around it by adding a Hello.ts file which exports a null value as the type of the Hello.android.ts But we really like to eliminate this part as it has become kind of a headache to write and maintain these files.
I was researching for a solution when I came up with the typescript transformers, and they just seemed like the ultimate solution for our issue.

@sosoba
Copy link

@sosoba sosoba commented Jan 25, 2021

What do you think. Is this an announcement of the end of API for plugins?

  • We are 100% committed to not rewriting JavaScript code as part of non-downleveling compilation. This is how everything else in TS works

Originally posted by @RyanCavanaugh in #16577 (comment)

@timocov
Copy link
Contributor

@timocov timocov commented Jan 31, 2021

Is this an announcement of the end of API for plugins?

I just read the original comment and I do believe that this is related to "rewriting JavaScript code" inside the compiler as a part of TypeScript compiler itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet