Introduce scope tracking in the parser

[danez]

Q	A
Fixed Issues?	Fixes #6722 (Fixes maybe #6283) Fixes #9316 Fixes #4331
Patch: Bug Fix?	y
Major: Breaking Change?	n
Minor: New Feature?	n
Tests Added + Pass?	Yes
Documentation PR Link
Any Dependency Changes?
License	MIT (acorn MIT)

This introduces scope tracking in the parser. The scope tracking replaces some of the Flags we currently have (inGenerator, inFunction, ...).

Spec changes:

• We now report an syntax error if super is used even though there is no superClass
• Duplicate bindings will throw according to spec

The idea here is to enter and exit scopes, during parsing so we always can check the current scope. A scope consists of flags (which adds information we need in several places like SCOPE_FUNCTION, SCOPE_ASYNC, ...) and 3 collections of declared bindings in this scope (var, lexical(let, const, classes) and functions).

TODO:

• Add tests for duplicate variable bindings.
• Migrate some traverse tests (duplicate declarations) to not use the parser

Big thanks to acorn, as most of this here is ported from acorn. :) Same goes for copyright on most of it.

[babel-bot]

Build successful! You can test your changes in the REPL here: https://babeljs.io/repl/build/10229/

[danez]

We have some tests that test the duplicate declaration checks of babel-traverse, but as the parser now throws first we are not getting to the point of testing duplicate declarations in traverse.

I see 3 options:

1. Leave the tests as is and don't care that we are not testing traverse.
2. Add an option to the parser to disable the duplicate check.
3. Move the tests to the parser (as thats what they are testing now) but also recreate new tests for traverse which do not use the parser but node builders instead to create the AST.

1 and 2 are not really good options for me. So I guess I will go for 3 which is the most work but the best I feel? Any other ideas on this?

[nicolo-ribaudo]

👍 for 3. We could ask help to the community to rewrite the @babel/traverse tests.

[danez]

👍 for 3. We could ask help to the community to rewrite the @babel/traverse tests.

Turns out it was only 4 tests. Probably took longer to write the message above, than it will take to fix them :D

Whoops, wrong PR. I meant to approve #9522

[danez]

Not sure if i should leave that or not. If this check would be in traverse I would leave it, but in the transform, the only valid usecase now would be someone creating in a plugin invalid code and then running the transform.

[nicolo-ribaudo]

I don't think that we need it: currently all the other early errors (except for the duplicated bindings errors) are only in @babel/parser.

[nicolo-ribaudo]

I think that we should also add binding types for Flow and TS types, and check if they are duplicated.

In flow types and vars share the same namespace (type A = string; var A is an error), while in TypeScript they are in different namespaces. type A = string; type A = number; is an error in both languages.

[nicolo-ribaudo]

Since ScopeParser is pretty much its own thing, maybe we could use it like this.scope.* instead of adding it to the prototype chain, like we do for State?

[danez]

I changed it to be a separate class, which works and might be nice to have like this if we would want to reuse this parts in traverse for example.

But when looking into flow I noticed another problem: extending the scope tracking is not as easy possible with the external class. So for typescript we probably would want to add another binding group for types. Flow has some custom rules too when it comes to declare class|var|function. I'm not sure if we actually want to support extending the scope tracking but it might be nice.

[danez]

Okay so I added fixes for flow:

• declare module {} has it's own scope now
• type and opaque type are now registered as lexical declaration, so the are handled the same as let or const

I also added tests for TS to ensure types do not count as duplicate declarations of other lexical declarations.

Any further additions for flow and typescript need changes to the logic in how duplicate declarations are checked. But the problem is that there is no easy way to extend the scope handler, as any addition (for example a new scope group for typescript type declarations) will need adjustments to all other parts of the scope tracking logic, so that lexical and function declarations are checked correctly against for example flow interfaces.
The only way to achieve correct scope tracking for flow and TS is to check on initialization which language we are parsing and depending on that choose a corresponding ScopeHandler. This ScopeHandlers for JS, TS and flow ultimately will need to be copy and pasted and kept in sync for bug fixes as I do not see a way to easy overwrite a subset of the logic of the duplicate detection algorithm for above reasons.

For now this PR does not introduce any duplicate checks for flow an ts, besides the ones mentions above, and we can follow up in a separate PR.

[nicolo-ribaudo]

This PR looks good, I left some last minor comments.

• Could you add a test that the first code throws while the second doesn't, in script scope? (I couldn't find it, but maybe it already exists? I searched for var in the diff)

  {
    function f() {}
    var f;
  }

  function f() {}
  var f;