Skip to content
This repository has been archived by the owner on May 19, 2018. It is now read-only.

TypeScript parser #523

Merged
merged 74 commits into from Jun 28, 2017
Merged

TypeScript parser #523

merged 74 commits into from Jun 28, 2017

Conversation

ghost
Copy link

@ghost ghost commented May 12, 2017

Q A
Bug fix? no
Breaking change? no
New feature? yes
Deprecations? no
Spec compliancy? no
Tests added/pass? Tests added, and pass
Fixed tickets #320
License MIT

This is a WIP, so please don't merge yet. The AST spec is likely to change. I am just looking to get comments on the implementation.
For those interested in just the AST spec, just look at the changes to src/types.js. I am tracking differences with typescript-eslint-parser here.

CC @stephen @JamesHenry @DanielRosenwasser

Notes:

  • It shares a lot in common with the flow plugin, but for now I am just using identical code (and marking it as such). In the future they should probably share.

  • Test files should really have a .ts extension, but that would require editing babel-helper-fixtures. Shall I make a pull request there?

@codecov
Copy link

codecov bot commented May 12, 2017

Codecov Report

Merging #523 into master will decrease coverage by 2.29%.
The diff coverage is 89.05%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #523     +/-   ##
=========================================
- Coverage   98.27%   95.97%   -2.3%     
=========================================
  Files          22       23      +1     
  Lines        3530     4477    +947     
  Branches      981     1240    +259     
=========================================
+ Hits         3469     4297    +828     
- Misses         22       86     +64     
- Partials       39       94     +55
Flag Coverage Δ
#babel 65.04% <13.98%> (-16.01%) ⬇️
#babylon 95.06% <89.05%> (-2.05%) ⬇️
Impacted Files Coverage Δ
src/tokenizer/state.js 100% <ø> (ø) ⬆️
src/tokenizer/types.js 100% <ø> (ø) ⬆️
src/tokenizer/context.js 100% <ø> (ø) ⬆️
src/tokenizer/index.js 98.36% <100%> (ø) ⬆️
src/plugins/estree.js 99.16% <100%> (ø) ⬆️
src/parser/lval.js 98.12% <100%> (+0.11%) ⬆️
src/parser/util.js 92.3% <100%> (+2.3%) ⬆️
src/plugins/flow.js 98.37% <100%> (-0.01%) ⬇️
src/index.js 94.44% <75%> (-5.56%) ⬇️
src/plugins/typescript.js 86.13% <86.13%> (ø)
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a738870...fc81c23. Read the comment docs.

@codecov
Copy link

codecov bot commented May 12, 2017

Codecov Report

Merging #523 into master will decrease coverage by 2.32%.
The diff coverage is 88.36%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #523      +/-   ##
==========================================
- Coverage   98.14%   95.82%   -2.33%     
==========================================
  Files          22       23       +1     
  Lines        3674     4695    +1021     
  Branches     1024     1287     +263     
==========================================
+ Hits         3606     4499     +893     
- Misses         25      107      +82     
- Partials       43       89      +46
Flag Coverage Δ
#babel 64.21% <10.56%> (-16.6%) ⬇️
#babylon 94.78% <88.36%> (-2.04%) ⬇️
Impacted Files Coverage Δ
src/tokenizer/types.js 100% <ø> (ø) ⬆️
src/tokenizer/state.js 100% <ø> (ø) ⬆️
src/tokenizer/context.js 100% <ø> (ø) ⬆️
src/parser/node.js 96.87% <100%> (+0.2%) ⬆️
src/parser/util.js 91.89% <100%> (+1.89%) ⬆️
src/tokenizer/index.js 98.44% <100%> (ø) ⬆️
src/plugins/estree.js 99.18% <100%> (-0.01%) ⬇️
src/index.js 94.44% <75%> (-5.56%) ⬇️
src/parser/lval.js 96.83% <83.33%> (-1.21%) ⬇️
src/plugins/flow.js 98.17% <84.61%> (-0.23%) ⬇️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b4fba4...5b56e59. Read the comment docs.

@hzoo
Copy link
Member

hzoo commented May 14, 2017

Test files should really have a .ts extension, but that would require editing babel-helper-fixtures. Shall I make a pull request there?

I think that should be covered by babel/babel#5511 but I didn't think we really wanted it earlier?

@hzoo
Copy link
Member

hzoo commented May 16, 2017

I know it's easier to modify the code in src/expression etc, but is there a way for us to make the parser plugin like in estree/flow/jsx? https://github.com/babel/babylon/tree/master/src/plugins.

The concern is how we are going to do versioning? In flow I guess everything is a minor/patch though. Maybe we can discuss is at the meeting (not typescript specific issue though)

Andy Hanson added 20 commits May 17, 2017 10:05
Change TSIndexSignatureDeclaration to TSIndexSignature
Change TSModuleBlock `.statements` to `.body`
Use TypeParameterDeclaration and TypeParameter nodes instead of a TsTypeParameterDeclaration[]
Use TypeAnnotation node for type annotations
…ts may never have "name", others are TSNamedTypeElementBase
Add TSInterfaceBody intermediate node
Change "name" fields to "id" or "key"
Introduce TSDeclareFunction and TSDeclareMethod node types
node.generator = !!isGenerator;
this.parseFunctionBody(node);
const missingBody = this.maybeParseFunctionBody(node, /* allowExpression */ false, /* allowMissingBody */ type === "ClassMethod");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could pass an object here, like:

maybeParseFunctionBody({ allowExpression: false, allowMissingBody: type === "ClassMethod"});

Looks better than comments to me.

// Strict mode words may be allowed as in `declare namespace N { const static: number; }`.
// And we have a type checker anyway, so don't bother having the parser do it.
return;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about a better integration with TypeScript here? Early grammar errors are great for online editor (like Babel's REPL). Babylon has that kind of logic for JavaScript.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would have to know of whether we're in a declaration context or not. And knowing that requires knowing the current file extension. export let package: number; is a perfectly legal .d.ts file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And knowing that requires knowing the current file extension.

We could also add a "sourceType": "declaration" which treats files as .d.ts

@@ -786,6 +795,8 @@ export default class ExpressionParser extends LValParser {
node.callee = this.parseNoCallExpr();
const optional = this.eat(tt.questionDot);

this.parseNewTypeArguments(node);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't notice this before: is it possible to implement new type arguments only in the typescript plugin?

Something like

parseNew() {
    const node = super.parseNew();
    if (/* node is NewExpression */ && !/* node has args parentheses*/ && this.match("<")) {
       // parse type arguments
       // maybe parse arguments
    }
}

Anyway, apart from this detail this pr looks good to me! (well, I have to admit I haven't reviwed all of it because I don't know typescript very well 😅 )

Copy link
Author

@ghost ghost Jun 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the AST gives us a way to distinguish between new X and new X(). But I found a different way that works just as well, see the latest commit 102be0e.

@nicolo-ribaudo
Copy link
Member

I have a concern about versioning: https://github.com/Microsoft/TypeScript/wiki/Breaking-Changes makes me think that typescript can have breaking changes. Does this PR make babylon potentially need a major version bump every time a new ts version is released?

@mhegazy
Copy link

mhegazy commented Jun 25, 2017

Typescript never had syntactic breaking changes. Meaning valid syntax that is supported by one version and disallowed by a newer version. All breaking changes are semantic in nature; either new error checks or new behaviors for the type system. All these are outside the scope of Babylon. There are some changes in the resulting code from tsc but these are not relevant here either.
So from parsing perspective we have a guarantee for no breaking changes movinging forward.
Hope that addresses your concern.

@nicolo-ribaudo
Copy link
Member

Thank you 👍

// 99% certain this is `new C<T>();`. But may be `new C < T;`, which is also legal.
const typeParameters = this.tsTryParseAndCatch(this.tsParseTypeArguments.bind(this));
if (typeParameters) {
node.typeParameters = typeParameters;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should check that this.match(tt.parenL), otherwise new Foo<T> will be allowed

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@hzoo hzoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super job @andy-ms, I agree with landing and iterating. Sorry for the delays

@hzoo hzoo changed the title WIP: TypeScript parser TypeScript parser Jun 28, 2017
@hzoo hzoo merged commit 97c2346 into babel:master Jun 28, 2017
@JamesHenry
Copy link
Member

Congrats, @andy-ms! Awesome work 😄

We can keep fine tuning the AST together but it's fantastic to have this merged!

@ghost
Copy link
Author

ghost commented Jun 28, 2017

@hzoo Please notify me when this is published.

@hzoo
Copy link
Member

hzoo commented Jun 28, 2017

yep doing it now!

@hzoo
Copy link
Member

hzoo commented Jun 28, 2017

https://github.com/babel/babylon/releases/tag/v7.0.0-beta.16

@yahiko00
Copy link

Oh. My. God. <3

@hzoo hzoo mentioned this pull request Jun 29, 2017
5 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants