Crash when there's no space after generic type parameters

[noppa]

Input

type A<B>= {}

causes a crash with both TypeScript and Flow transforms.

Error for the TS transform:

SyntaxError: Error transforming testdir/test.ts: Unexpectedly reached the end of input. (4:1)
    at unexpected (node_modules/sucrase/dist/parser/traverser/util.js:83:15)
    at nextToken (node_modules/sucrase/dist/parser/tokenizer/index.js:251:24)
    at next (node_modules/sucrase/dist/parser/tokenizer/index.js:151:3)
    at parseIdentifier (node_modules/sucrase/dist/parser/traverser/expression.js:9

Adding a space between the > and = fixes the issue.

[alangpierce]

Sorry for the delay! Agreed that this is a bug, though I looked into it a bit and it may end up being a tricky fix. Here's what I've found so far:

The underlying issue is that the tokenizer is recognizing >= as a greater-than-or-equal operator. Generally Sucrase will need to pass more context from the parser (which knows that we're parsing a type declaration) to the tokenizer, though simply doing an isType check in the tokenizer's readToken_lt_gt (like we do to handle >>) seems to break other things.

Babel has the same bug (Babel repl example), though it looks like it doesn't have the same problem with Flow, so there might be a reasonable fix by tracing that code path.

TypeScript fixes it by always parsing the token as > and then sometimes calling reScanGreaterToken to turn it into a >= if it's in the right context. It may be possible to translate the same strategy into Sucrase, though it could end up being a fairly big and fragile change.

Another possible approach is to add a special case to tsParseTypeParameters where we accept tt.relationalOrEqual and then correct the tokens in that case. That approach certainly feels like a hack, but it might still be the best fix here.

[noppa]

Babel has the same bug (Babel repl example), though it looks like it doesn't have the same problem with Flow

Oh that's very interesting, we use Flow so I didn't realize that Babel could also be having trouble with this. Is this perhaps related to the fact that Flow doesn't have <Foo> foo casting syntax like TS does?

Anyway, this probably isn't a high priority blocker or anything. Writing type A<B>= is probably against most style linting rules anyway. We had grand total of one of these instances in our codebase so I just ended up adding a space there.

[alangpierce]

I was taking a look at old bugs and noticed that this one seems to be working now! From some testing, it looks like it was fixed in version 3.24.0, most likely from the tokenizer changes in #717 .