support custom transformers for ts 2.3 #535

longlho · 2017-05-03T19:33:26Z

TS 2.3.1 support custom transformer so I think ts-loader should support that too

johnnyreilly · 2017-05-25T15:07:04Z

Sorry for the delay in responding - I've been pretty occupied with other things. Would you mind giving a few more details about this please? I'm guessing that your change is related to: microsoft/TypeScript#14419 ?

Having had a quick read I'm not totally clear as to the purpose of custom transformers - could you elaborate a little?

Also, it looks like you might need to resupply the PR as the changes for happypack compatibility collide with your changes. Finally, please could also provide a test? Thanks!

johnnyreilly · 2017-06-06T12:32:08Z

Ping @longlho 😄

longlho · 2017-06-06T12:36:38Z

@johnnyreilly sry this slipped thru my attention. I believe the purpose of custom transformers is like babel-plugin: You have more control over what's being transformed as TSC walks thru the AST. 1 example is loading external asset (https://github.com/longlho/ts-transform-img).

This allows you to import * as img from 'foo.png' and output would be const img = 'base64_encoded_img' inline. Right now I have to pipe ts-loader into babel-loader and would love to get rid of babel-loader.

I'll rebase the PR :)

Igorbek · 2017-06-19T18:39:14Z

Hey @johnnyreilly @longlho here's an example of a TS transformer for styled-components: https://github.com/Igorbek/typescript-plugin-styled-components. Since ts-loader doesn't support them, I have only section for integrating with awesome-typescript-loader.

johnnyreilly · 2017-06-19T18:45:13Z

Thanks for sharing @Igorbek - I'm hoping @longlho is going come back and finish the PR soon. He said he would so I have high hopes!

longlho · 2017-06-19T18:46:05Z

yeah sorry I've been tied up. Will try to wrap this up this week

johnnyreilly · 2017-06-19T18:56:03Z

Great - thanks @longlho! Could we use the same API as awesome-typescript-loader please?

getCustomTransformers?(): ts.CustomTransformers

We've a vague plan that atl and ts-loader may come together at some point. Either way, we want it to be easy for people to move from one loader to the other as required.

longlho · 2017-06-19T20:27:33Z

@johnnyreilly I fixed the PR. I'm trying to find a similar test to mimic but couldn't. Can you point me to something?

johnnyreilly · 2017-06-19T20:46:03Z

Since there's no way for this to be tested at runtime (I think?) then it makes logical sense to create a comparison test rather than an execution test. I'd be tempted to use this one as a basis:

https://github.com/TypeStrong/ts-loader/tree/master/test/comparison-tests/basic

Remember you only need generate the 2.3 expected output as we only run comparison tests for the latest released build.

Do set me straight if you think this can be covered by an execution test but since it's about specifically using transformed output I'm guessing that doesn't make sense.

johnnyreilly · 2017-06-19T20:47:47Z

Build failure BTW:

src/index.ts(189,9): error TS2345: Argument of type '{ compilerOptions: CompilerOptions; transformers: CustomTransformers; reportDiagnostics: true; fi...' is not assignable to parameter of type 'TranspileOptions'.
  Object literal may only specify known properties, and 'transformers' does not exist in type 'TranspileOptions'.
src/instances.ts(106,37): error TS2339: Property 'transformers' does not exist on type 'LoaderOptions'.

Igorbek · 2017-06-19T21:18:12Z

As a test input, can be passed a simple transformer. I can write one if you want.

longlho · 2017-06-19T22:07:12Z

I think build might not be passing on 2.4.0-rc :-/ I tried changing the sub version and condition to support 2.4 but a bunch failed.

johnnyreilly · 2017-06-20T05:00:33Z

Don't worry about 2.4 - that's a possible regression in TypeScript 2.4. see here: microsoft/TypeScript#16609 (comment)

johnnyreilly · 2017-06-20T05:01:43Z

@Igorbek that'd be very helpful - thanks!

johnnyreilly · 2017-06-20T05:02:32Z

(I haven't had time to fully understand custom transformers as yet)

johnnyreilly · 2017-06-20T05:29:43Z

Yeah - looking at the build this all seems good. The failure seems to be TypeScript@next related and I don't think that's something that need concern us as I'm expecting that to be resolved in the compiler itself. All we need is a test and I think we're good to go! @Igorbek - if you get the chance it might be good to see if @longlho's fork works with https://github.com/Igorbek/typescript-plugin-styled-components as you would hope

Igorbek · 2017-06-20T17:47:13Z

Here's a simple transformer that makes all string literals uppercase:

import * as ts from 'typescript';

const transformer: ts.TransformerFactory<ts.SourceFile> = (context) => {
    const visitor: ts.Visitor = (node) => {
        if (node.kind === ts.SyntaxKind.StringLiteral) {
            const text = (node as ts.StringLiteral).text;
            if (text !== text.toUpperCase()) {
                return ts.createLiteral(text.toUpperCase());
            }
        }

        return ts.visitEachChild(node, visitor, context);
    }

    return (node) => ts.visitNode(node, visitor);
};

export default transformer;

johnnyreilly · 2017-06-20T18:02:53Z

Great! How would you expect that to be plugged into ts-loader's config?

johnnyreilly · 2017-06-20T18:10:30Z

Also; how would the output of that be expected to look? Am I right when I guess that this input:

const greeting = 'hi';

would become transformed into:

const greeting = 'HI';

?

longlho · 2017-06-20T18:16:40Z

yeah I think that's the output. The way custom transformer would be passed in is something like:

{
  loader: 'ts-loader',
  query: {
    getCustomTransformers: () => {
      before: [require('custom-transformer')],
      after: [require('some-other-transformers')]
    }                 
  }
}

Igorbek · 2017-06-20T18:21:57Z

@johnnyreilly that's right, expected output is correct
@longlho yes, getCustomerTransformer should return { before?: Array<ts.TransformerFactory<ts.SorceFile>>; after?: Array<ts.TransformerFactory<ts.SorceFile>>; }

getCustomTransformers: () => ({ // note parens
      before: [transformer]
})

johnnyreilly · 2017-06-20T19:52:43Z

Okay - I've had a first go at a comparison test based on @Igorbek's sample. Unfortunately it doesn't seem to work - I'm not too sure why though... BTW I built ts-loader with TypeScript@2.4.0 but I was using TypeScript 2.3.1 to generate the test data as that's what the test pack runs against. (So post build of ts-loader and pre-test I npm install typescript@2.3.1)

Igorbek · 2017-06-20T20:22:31Z

test/comparison-tests/customTransformer/webpack.config.js

+// 2. create a transformer;
+// the factory additionally accepts an options object which described below
+var styledComponentsTransformer = createStyledComponentsTransformer();
+


The sample transformer exports the transformer itself, and not a factory function that typescript-plugin-styled-components does. So you don't need to call the function:

var uppercaseStringLiteralTransformer = require('./uppercaseStringLiteralTransformer').default;

Igorbek · 2017-06-20T20:22:49Z

test/comparison-tests/customTransformer/webpack.config.js

+                loader: 'ts-loader',
+                options: {
+                    getCustomTransformers: () => ({ // note parens
+                        before: [styledComponentsTransformer]


before: [uppercaseStringLiteralTransformer]

johnnyreilly · 2017-06-21T04:38:29Z

Thanks @Igorbek - just made those changes. Looks promising!

Igorbek

Looks very nice!

Igorbek · 2017-06-21T05:13:36Z

test/comparison-tests/customTransformer/expectedOutput-2.3/output.txt

+    Asset     Size  Chunks             Chunk Names
+bundle.js  2.92 kB       0  [emitted]  main
+chunk    {0} bundle.js (main) 193 bytes [entry] [rendered]
+    [0] ./.test/customTransformer/SUBMODULE/SUBMODULE.ts 78 bytes {0} [built]


transformer uppercased the module import as well :) it was unintentional, but ok ))

In actual fact that is the reason the test is failing on Travis right now I think. Case insensitive Windows Vs case sensitive Linux 😄 I'll change the test so it is 1 file only (no imports) tonight and then we should have a reliable test that works cross platform. Then I think we're ready to merge!

johnnyreilly · 2017-06-21T17:52:18Z

Okay - so we've now got a cross platform test! However, it looks like customTransfrmers aren't applied in transpileOnly mode. Is that intentional?

longlho · 2017-06-21T17:53:48Z

hmm nope those 2 configs should be independent

johnnyreilly · 2017-06-21T18:01:36Z

If you look at TranspileOptions it doesn't appear to a valid option:

    interface TranspileOptions {
        compilerOptions?: CompilerOptions;
        fileName?: string;
        reportDiagnostics?: boolean;
        moduleName?: string;
        renamedDependencies?: MapLike<string>;
    }

johnnyreilly · 2017-06-21T18:01:59Z

So maybe this isn't supported for transpileOnly?

longlho · 2017-06-21T18:03:05Z

Maybe in 2.4 they're supporting it? (https://github.com/Microsoft/TypeScript/blob/4b3e661aaa69e53b6a5992c76f2232c9f5dace10/src/services/transpile.ts#L2-L9)

Igorbek · 2017-06-21T18:05:37Z

yes, 2.4rc supports it.

johnnyreilly · 2017-06-21T18:10:31Z

Ok - so the test was generated with 2.3 so if it was added with 2.4 that makes sense. If you use it with 2.4 it'll just work?

johnnyreilly · 2017-06-21T18:22:42Z

I'm happy - let's merge!

johnnyreilly · 2017-06-21T18:40:46Z

I'm cutting ts-loader 2.2 right now - should be released within the hour. Thanks for the contribution!

TomMarius · 2017-07-06T17:48:51Z

Hey guys, how do I get to the type checker (ts.TypeChecker)? The method getTypeChecker is available only in the ts.Program interface and I don't see it being passed down from the loader nor can I find any examples that would use the type checker. I'm probably just missing something, but I've been scratching my head for a few hours now.

johnnyreilly · 2017-07-06T18:40:42Z

Do you mean fork-ts-checker-webpack-plugin? It's a separate npm package: https://github.com/Realytics/fork-ts-checker-webpack-plugin

johnnyreilly · 2017-07-06T18:42:08Z

See here: https://github.com/TypeStrong/ts-loader/tree/master/examples/fork-ts-checker

TomMarius · 2017-07-06T18:46:56Z

Nonono, I meant how to get to the TypeScript's internal instance of TypeChecker that I use in my custom transformers. Thank you for your reply though.

TomMarius · 2017-07-06T18:50:47Z

See this custom transformer for example: https://github.com/kimamula/ts-transformer-enumerate/blob/master/transformer.ts

He uses argument program of function visitNode to pass down the compiler's instance of the Program interface. He then uses the getTypeChecker method and then uses the TypeChecker instance to resolve types in the transformer. I need to get either the TypeChecker itself or an instance of the Program interface down from ts-loader.

johnnyreilly · 2017-07-06T18:56:03Z

Ah I see. To be honest I've not really done anything with transformers and so can't help. Maybe @longlho or @Igorbek can...

longlho · 2017-07-06T19:05:58Z

@TomMarius you can't get access to the program currently since it's out of scope. You will need to write a custom compiler to hoist that or PR'ed to TS have that passed down as part of the TransformationContext. I guess ts-loader can pass that down as a param, althou I'm not sure how future-proof that will be.

TomMarius · 2017-07-06T19:12:36Z

@longlho Okay, I'll probably create a PR in the following days.

support custom transformers for ts 2.3

976ddf6

First go at comparison test

e4dfded

Igorbek suggested changes Jun 20, 2017

View reviewed changes

Suggested tweaks by @Igorbek

45f7af2

Igorbek approved these changes Jun 21, 2017

View reviewed changes

Made cross platform test

3704c0b

johnnyreilly merged commit 6594328 into TypeStrong:master Jun 21, 2017

plievone mentioned this pull request Jun 22, 2017

babel-plugin-inferno equivalent as a TypeScript custom transformer infernojs/inferno#1154

Closed

trotyl mentioned this pull request Aug 12, 2017

Proposal: ngc for JIT angular/angular#18664

Closed

yihongang mentioned this pull request Sep 12, 2017

Typescript 2.3+ custom transformers fuse-box/fuse-box#792

Closed

deamme mentioned this pull request Jan 29, 2018

Support for Custom Typescript transformers parcel-bundler/parcel#699

Closed

longlho mentioned this pull request Dec 10, 2018

Let's discuss TypeScript support. kentcdodds/babel-plugin-macros#94

Closed

support custom transformers for ts 2.3 #535

support custom transformers for ts 2.3 #535

Conversation

longlho commented May 3, 2017

johnnyreilly commented May 25, 2017

johnnyreilly commented Jun 6, 2017

longlho commented Jun 6, 2017

Igorbek commented Jun 19, 2017

johnnyreilly commented Jun 19, 2017

longlho commented Jun 19, 2017

johnnyreilly commented Jun 19, 2017

longlho commented Jun 19, 2017

johnnyreilly commented Jun 19, 2017

johnnyreilly commented Jun 19, 2017

Igorbek commented Jun 19, 2017

longlho commented Jun 19, 2017

johnnyreilly commented Jun 20, 2017

johnnyreilly commented Jun 20, 2017

johnnyreilly commented Jun 20, 2017

johnnyreilly commented Jun 20, 2017

Igorbek commented Jun 20, 2017

johnnyreilly commented Jun 20, 2017

johnnyreilly commented Jun 20, 2017 • edited Loading

longlho commented Jun 20, 2017 • edited Loading

Igorbek commented Jun 20, 2017

johnnyreilly commented Jun 20, 2017

Igorbek Jun 20, 2017

Choose a reason for hiding this comment

Igorbek Jun 20, 2017

Choose a reason for hiding this comment

johnnyreilly commented Jun 21, 2017

Igorbek left a comment

Choose a reason for hiding this comment

Igorbek Jun 21, 2017

Choose a reason for hiding this comment

johnnyreilly Jun 21, 2017

Choose a reason for hiding this comment

johnnyreilly commented Jun 21, 2017

longlho commented Jun 21, 2017

johnnyreilly commented Jun 21, 2017

johnnyreilly commented Jun 21, 2017

longlho commented Jun 21, 2017

Igorbek commented Jun 21, 2017

johnnyreilly commented Jun 21, 2017

johnnyreilly commented Jun 21, 2017

johnnyreilly commented Jun 21, 2017

TomMarius commented Jul 6, 2017 • edited Loading

johnnyreilly commented Jul 6, 2017

johnnyreilly commented Jul 6, 2017

TomMarius commented Jul 6, 2017

TomMarius commented Jul 6, 2017 • edited Loading

johnnyreilly commented Jul 6, 2017

longlho commented Jul 6, 2017

TomMarius commented Jul 6, 2017

johnnyreilly commented Jun 20, 2017 •

edited

Loading

longlho commented Jun 20, 2017 •

edited

Loading

TomMarius commented Jul 6, 2017 •

edited

Loading

TomMarius commented Jul 6, 2017 •

edited

Loading