Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Replace emitter with syntax tree transformations and a simplified node emitter. #5595

Closed
rbuckton opened this issue Nov 10, 2015 · 21 comments

Comments

@rbuckton
Copy link
Member

commented Nov 10, 2015

In an effort to better support down-level transformations for complex emit like generators and Async Functions, and to be able to add new down-level transformations for the evolving ECMA-262 specification, we propose to replace the current emitter logic for the TypeScript compiler with an implementation that relies on transformations of the source tree.

Our current emitter is not built to handle the requirements for some of the more complex rewriting needed to support features such as generators (ES6/ES2015) and Async Functions (ES2016). In addition, as time goes on we will need to add more and more branching logic to handle down-level emit of new language features due to the yearly cadence that TC39 is adopting for the ECMAScript specification.

Syntax tree transformations will give us the ability to transform our source tree in an iterative fashion, allowing us to inject new transformations at the head of a transformation chain so that few (if any) changes need to be made to existing transformations in later iterations. As a result, there would be minimal maintenance for the complex transformations needed for down-level generators, as long as new features are already transformed into a syntactically valid ES6 tree.

Generally this will also help to reduce the branching logic of our current emitter, and keep syntactic transformations isolated to an individual file for a language version or feature. As a result, the emitter itself can be simplified drastically and focus specifically on emitting the given syntax tree with almost no branching logic.

Requirements:

  • No transformation should directly modify the original source tree.
  • Transformations should be isolated to a specific language version or, when necessary, a language feature (i.e. TypeScript to ES6, ES6 to ES5, AMD module transformations, etc.).
  • Transformations should reuse existing syntax tree nodes when possible.
  • Transformations should employ a mechanism to quickly identify whether a node or subtree requires transformation to avoid a full walk of a source tree for each iteration.
  • Transformations should preserve source locations for use with source maps.
  • Transformations should not significantly impact the performance of the compiler, and ideally should improve compile time.

Goals:

  • Transform a TypeScript syntax tree into a compatible JavaScript syntax tree based on compiler settings.
  • Source Map emit should behave as it does in the current compiler.
  • Comment preservation should behave as it does in the current compiler.

Non-Goals:

  • We will not be replacing our declaration emitter with transformations at this time, though we may consider this in a future proposal.
  • We will not support end-user extensibility of these transformations at this time, though we may consider this in a future proposal.

Remaining Tasks:

  • Update to support block-scoped declarations captured in loops.
  • Verify comment preservation
  • Investigate comment preservation performance issues
  • Verify/adjust source map output
  • Fix failing tests
  • Update transformations to match recent changes to the emitter.
  • Investigate/improve performance of tree-transforming emitter.
@falsandtru

This comment has been minimized.

Copy link
Contributor

commented Dec 3, 2015

+1 TypeScript should has more modularbility for testability and maintainability for implementations of new ECMA-262 specification. If possible, I want the plugable design and hook points for transforms like the babel. TypeScript team will be able to get feedbacks from user implemented plugins.

@tobich

This comment has been minimized.

Copy link
Contributor

commented Dec 3, 2015

+1

@jjrv

This comment has been minimized.

Copy link

commented Dec 8, 2015

It would be very nice to allow extending the emitter to output JSDoc comments with types, so transpiled ES5 code could then be fed into Closure Compiler with advanced optimizations turned on.

@ctaggart

This comment has been minimized.

Copy link
Contributor

commented Dec 10, 2015

Will you be able to create the Syntax Tree from scratch?
Will be be to write the transformed Syntax Tree to a file (a .ts file)?

@mhegazy

This comment has been minimized.

Copy link

commented Dec 10, 2015

that is the plan :)

@D1no

This comment has been minimized.

Copy link

commented Mar 20, 2016

It would be great if this would be compatible to the Babel-plugin style. Great way to open the door to implement typescript "native" runtime type checking

@dasa

This comment has been minimized.

Copy link

commented Jul 8, 2016

https://github.com/Microsoft/TypeScript/wiki/Roadmap hasn't been updated to match.

@whitecolor

This comment has been minimized.

Copy link

commented Sep 4, 2016

Hope to see it soon! another big step to make babel obsolete.

@rbuckton

This comment has been minimized.

Copy link
Member Author

commented Sep 7, 2016

Completed as of 4685646.

@rbuckton rbuckton closed this Sep 7, 2016

@whitecolor

This comment has been minimized.

Copy link

commented Sep 7, 2016

@rbuckton where it is possible to read how to use committed changes?

@phiresky

This comment has been minimized.

Copy link

commented Sep 7, 2016

@whitecolor see #1564 (comment) (use typescript@next)

@ctaggart

This comment has been minimized.

Copy link
Contributor

commented Sep 7, 2016

Would it be possible to provide an example that creates a syntax tree from scratch and emits the TypeScript? I've been wanting to be able to do this for a couple of years now. See How do you do a straight TS emit?

@MastroLindus

This comment has been minimized.

Copy link

commented Oct 12, 2016

One thing I would like to do is to manipulate the emitted values for string literals. For example I would love to be able to specify strings to be translated with some kind of special syntax in string literals, like "@foo", and have the emitter translate all strings literals that match the given syntax to a specific function call.

In the non-goal there is : "We will not support end-user extensibility of these transformations at this time, though we may consider this in a future proposal."

So should I assume that doing something like that is not currently possible?

@mhegazy

This comment has been minimized.

Copy link

commented Oct 14, 2016

So should I assume that doing something like that is not currently possible?

yes. but we intend to expose all the transformation and factory APIs so it should be possible in the near future.

@MastroLindus

This comment has been minimized.

Copy link

commented Oct 14, 2016

Thanks for the answer, looking forward to that

@ToddThomson

This comment has been minimized.

Copy link
Contributor

commented Nov 10, 2016

In my Typescript packaging compiler I create a single module bundle (via import dependency analysis) and then directly minify the bundle. To minify the ts bundle I must first perform identifier shortening, compile the bundle and then perform whitespace removal on the js output ( a multistep process as I currently don't have access to the emitter ).

With the new transformation and factory APIs I am hoping to do this all in 1 step.

My question is how to provide directives to a transform within the ts source file. Since I have total control of how I package my bundle I can essentially make the whole package internal and thus shorten the majority of identifiers. However, when using Angular 2 components, mangling identifiers which are used within component html templates is a problem!. If I had a mechanism to provide a directive to the minification transform I could then selectively bypass identifier shortening for those used within the template.

As an alternative to transform directives would using /* @transformDirective */, like your use of /* @internal */ be the way forward for now?

Thanks for any help!

@mischkl

This comment has been minimized.

Copy link

commented Nov 27, 2016

Out of curiosity, might this change be of benefit to transforming ES6 Maps and Sets to ES5 (#4031, #11209)?

@trusktr

This comment has been minimized.

Copy link

commented Mar 9, 2017

@mhegazy

yes. but we intend to expose all the transformation and factory APIs so it should be possible in the near future.

How near is this future?

@trusktr

This comment has been minimized.

Copy link

commented Mar 9, 2017

@mischkl

Out of curiosity, might this change be of benefit to transforming ES6 Maps and Sets to ES5 (#4031, #11209)?

I'm not entirely sure, but I don't think it is a goal of TypeScript to transform non-language features.Map and Set are just references to constructor functions that are implemented by the JS engine, they are not language features really (i.e. they are not syntax features of the language, they are just runtime tools provided to us).

Is there any plan to compile such things in TypeScript?

@RyanCavanaugh

This comment has been minimized.

Copy link
Member

commented Mar 9, 2017

Is there any plan to compile such things in TypeScript?

No

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
You can’t perform that action at this time.