New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow "Compiler Plugins" #16607

Open
mohsen1 opened this Issue Jun 18, 2017 · 19 comments

Comments

Projects
None yet
@mohsen1
Copy link
Contributor

mohsen1 commented Jun 18, 2017

From wiki:

TypeScript Language Service Plugins ("plugins") are for changing the editing experience only

TypeScript plugins are very limited. Plugins should be able to:

  • Apply Transformers
  • Provide type definitions
  • Override module resolver (LanguageServiceHost#resolveModuleNames)
  • Emit code beside TypeScript standard output
@DanielRosenwasser

This comment has been minimized.

Copy link
Member

DanielRosenwasser commented Jun 19, 2017

@rbuckton and I have some offhand thoughts about this

  1. The scope of this is fairly broad - does this include generating new nodes for type-checking? If so, that makes this a much larger item.
  2. When it comes to module resolution, there are quite a few subtleties.
    1. Custom module resolution also means providing some special module resolution "host" which can provide this behavior in the editing scenarios.
    2. It's not clear how custom module resolution works between dependencies & dependents.

In general this isn't simple but we're open to at least hearing ideas.

@mohsen1

This comment has been minimized.

Copy link
Contributor

mohsen1 commented Jun 20, 2017

I'm not familiar with TypeScript well enough to write a proposal. Instead I can list a few plugins that can be useful and exist in the wild in other forms (Webpack plugin, Babel transforms) to make a case for having such extensibility:

Custom module resolution

Code transformers

Emitting other code

  • A Swagger code generator embedded in TypeScript plugin
  • Functionality similar to Webpack: emitting image and other asset files

Providing types

  • Typed GraphQL query responses
    • This applies to pretty much any database query language. Result of a query can be typed based on query

There are so many other use-cases for compiler plugins that I'm not aware of but I'm sure compiler plugins will make TypeScript ecosystem thrive.

@phra

This comment has been minimized.

Copy link

phra commented Jul 3, 2017

i think that this can be a very powerful feature.
in particular i'm interested in points 1, 2 and 4 at the moment.

@ryanelian

This comment has been minimized.

Copy link

ryanelian commented Jul 5, 2017

Big yes to this feature being supported by TypeScript.

I propose that the plugin system should be implemented using streaming pattern.

// NodeJS stuffs
import * as stream from 'stream';
import * as path from 'path';

export enum TypeScriptModuleEnum {
    CommonJS, AMD, System, UMD, ES6
}

export interface ITypeScriptTransform {
    (filename: string, module: TypeScriptModuleEnum): stream.Transform
}

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// As sample plugin to transform file content into string if matches extensions. Handy for templates.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

// Plugin creators can use this to extend TypeScript!
import * as through2 from 'through2';

export interface IStringifyOptions {
    extensions: string[]
}

function doStringify(filename: string, extensions: string[]) {
    return extensions.includes(path.extname(filename));
}

export function StringTransform(options: IStringifyOptions): ITypeScriptTransform {
    return function (filename, module) {
        return through2(function (file, encoding, next) {

            // Determines whether we should stringify the file.
            // For example, the file name = 'test.txt' and extensions list = ['.txt', '.html']
            if (!doStringify(filename, options.extensions)) {
                return next(null, file);
            }

            let s = JSON.stringify(file);

            if (module === TypeScriptModuleEnum.CommonJS) {
                s = 'module.exports = ' + s + ';\n';
                return next(null, s);
            } else {
                return next({
                    message: 'Module not supported!'
                });
            }
        });
    };
}

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Later...
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

import * as ts from 'typescript';
import { StringTransform } from 'StringTransform';

let tsString = StringTransform({
    extensions: ['.html', '.txt']
});

ts.useTransforms([tsString]);

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Later in application source code...
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

import s = require('./test.txt');

This system has the following benefits:

  • The vast majority of browserify and gulp plugins (which are mostly written using through2) can be ported to TypeScript easily!
  • Transforms can be executed sequentially by TypeScript during compilation, in-memory, using the order defined in the passed transform array.
  • Allows file manipulation freedom by plugin developers.
@cameron-martin

This comment has been minimized.

Copy link

cameron-martin commented Aug 8, 2017

Something similar to browserify transforms or webpack loaders would be very powerful and cover most of these use cases.

@phra

This comment has been minimized.

Copy link

phra commented Aug 8, 2017

looking forward to see this feature implemented! 🙌

@data-ux

This comment has been minimized.

Copy link

data-ux commented Aug 21, 2017

Currently program.emit() already has customTransformers as possible parameter, but it isn't exposed to consumers of the tsc command line program. It would be great to be able to give transformers in compilerOptions as proposed in #14419. Currently to use customTransformers you have to use the Compiler API and re-implement all the functionality in tsc like watching files etc.

@mohsen1

This comment has been minimized.

Copy link
Contributor

mohsen1 commented Aug 23, 2017

Transformers do not allow custom module resolution or extra file emit

@data-ux

This comment has been minimized.

Copy link

data-ux commented Aug 25, 2017

@mohsen1 Yes, you're right. I was suggesting it as an approach for your first point "Apply transformers". For a plugin to do all the things suggested in your issue description is too broad of a scope, as @DanielRosenwasser noted. I think exposing custom transformers is the highest value feature of the suggested and it's also the most simple to implement taking into account the way the TS compiler currently works.

@WiseBird

This comment has been minimized.

Copy link

WiseBird commented Sep 27, 2017

@Jack-Works Isn't Language Service doing it already?

@geovanisouza92

This comment has been minimized.

Copy link

geovanisouza92 commented Dec 22, 2017

Any news on this? I'm writing a plugin, and would like to just plug it on my current setup (tsc & webpack + awesome-typescript-loader).

@fis-cz

This comment has been minimized.

Copy link

fis-cz commented May 31, 2018

I am sorry. I didn't know it is possible to write language service plugins... I should read before I write. So, for the compiler it can be implemented in the similar way as for ls.

Regarding this topic I have some tips.

It would be great if loading of plugins is configurable through the tsconfig.json file. Thats because i.e. VS code syntax highlighter / lens / intellisense will use the same plugins as the compiler during the regular build. Plugin can be standard Node module and can be resolved in the standard CommonJS way.

When the plugin for the typescript compiler will be defined in the tsconfig.json file it should be loaded during the tsc startup and tsc should provide access to all currently available tsc APIs (it would be also great if extended tsconfig can be read through API too as when I was playing around the API about year ago i had to write custom config reader / extender, what is not good as with next release of ts you can remove or add some options and its hard to maintain the code afterwards ;).

During the init phase, the plugin can replace various stages of the compiler API with custom implementations (such as file reader) or bind event listeners to events occurring during the compilation process (file load, resolve, parse, transform, emit...). Events with "pre" and "post" would be also great in order to pre-process or post-process the stage data while original components are still in use. I.e. preParse is great time to run text preprocessor which can implement #ifdefs and replace them with empty lines to keep it possible to generate source maps properly, or postParse when AST can be searched for dead code and the dead code can be removed from furthermore processing.

From my perspective, it would be much easier to implement call puigin.init(...) with references to all available tsc components and let plugin developer to choose if he will replace it or not what would be specified in the return object.
I'll update this later once I'll check tsc sources.

If this would be possible we can simply use various plugins for code preprocessing, death code elimination, output minification or whatever else we can imagine directly under the hood of the compiler "executable" but without touching the compiler code itself. Currently, we have to write everything as a new compiler using the tsc API. Unfortunately, this later means we have to implement the "new" compiler to our development tools (such as VS code or full VS, what is almost impossible ;).

@pcan

This comment has been minimized.

Copy link
Contributor

pcan commented May 31, 2018

Events with "pre" and "post" would be also great in order to pre-process or post-process stage data (i.e. preParse is great time to run text preprocessor which can implement #ifdefs and replace them with empty lines to keep it possible to generate source maps properly, or postParse when AST can be searched for dead code and remove it from the furthermore processing.)

I would really need this one. The ideal spot for synthetic code injection is after the parsing phase: here you have the AST ready, you can do some enhancements, and they are already available to language service!

@fis-cz

This comment has been minimized.

Copy link

fis-cz commented May 31, 2018

I would really need this one. The ideal spot for synthetic code injection is after the parsing phase: here you have the AST ready, you can do some enhancements, and they are already available to language service!

I think it is enough if you can replace the parser with custom one and inside of it you would do your pre, call original parser with modified input and your post where you would modify the AST.

@agDmitry

This comment has been minimized.

Copy link

agDmitry commented Oct 26, 2018

Please make it happen.
There is just too much need in this.
Whenever i find myself in need of extending the typescript compiler with some feature (typed css modules, #10866) i always think "This is definetly can/must be done with a plugin". Babel has a plugin system, and people have an opportunity to do anything they need to do to get a work done, without bothering the core team with requests of new features. Besides, all burden of support lies on the plugin author. I mean that the community can add a ton of requested features to the typescript without bloating the core codebase (which will make it hard to support).

@xtuc

This comment has been minimized.

Copy link

xtuc commented Oct 27, 2018

Now that Babel 7 has the support for TypeScript, would it be possible to achieve that trough Babel?

@krzkaczor

This comment has been minimized.

Copy link

krzkaczor commented Oct 27, 2018

@xtuc i think this would break all existing tooling etc.

Personally i belive that type providers should be a way to extend typescript (only type system).

@mrmckeb

This comment has been minimized.

Copy link

mrmckeb commented Nov 22, 2018

I've been working on this plugin, and I would love that it would work during build - and not just when working with files in the IDE.

https://github.com/mrmckeb/typescript-plugin-css-modules

@FredyC

This comment has been minimized.

Copy link

FredyC commented Dec 9, 2018

I would like to invite everyone to the discussion I've started today that tries to find a way how to make a universal plugin/transform that would work in a similar way as babel-plugin-macros, ideally alongside it to avoid the need for multiple plugins that are hard to configure and use.

Please join and bring your ideas and experience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment