Skip to content

Commit

Permalink
Add support for TS 4.5 import type modifiers (#713)
Browse files Browse the repository at this point in the history 
Fixes #668

Details on the syntax are described here:
https://devblogs.microsoft.com/typescript/announcing-typescript-4-5/#type-on-import-names

This was implemented in Babel here: babel/babel#13802

In terms of behavior, the type syntax is *almost* a no-op in valid code; we
already scan the file to see which identifiers have been used in a value or type
context and remove type imports automatically, and this PR doesn't change that.
However, there are a few subtle differences:
* When targeting commonjs, if an imported name has a `type` modifier and is used
  in a value context, we do *not* transform the syntax to access the module like
  we did before. For example, this comes up if accessing a global value with the
  same name as an imported type.
* In both CJS and ESM targets, code that imports and then re-exports a type
  needs to have that export elided. Previously, this was a situation where TS
  doesn't know whether the name is a type or not, so it included the export as a
  value.

Right now, there is no equivalent to `--preserveValueImports`, but this could be
added in the future. It *almost* would turn the TS transform into a purely
syntax-level transform without the need for scope analysis and variable name
tracking (thus simplifying and speeding up Sucrase if it became the standard
approach), except that even with that flag set, type-only exports are still
elided.

In terms of implementation details, the Babel parser details were a bit
AST-focused and didn't need to deal with `IdentifierRole`s, so I re-implemented
parsing using a simpler approach, which was also useful in the transform step.

It turns out that in both Flow and TS, you can parse an import or export
specifier by accepting anywhere from 1 to 4 identifiers, and the number of
identifiers is enough to exactly determine the syntax. This is more reliable
than matching on the names `type` (or `typeof`) and `as` because those are
contextual keywords that might be valid identifiers.

In the transform step, there were 5 places where we were manually walking
import/export specifiers, so it seemed best to unify that parsing into a shared
utility. This potentially has some performance cost due to the object
allocations, but it's likely to be very minor and could potentially be improved
later.
  • Loading branch information
@alangpierce
alangpierce committed Jul 1, 2022
1 parent ee5879e commit b9bce13
Show file tree
Hide file tree
Showing 8 changed files with 354 additions and 114 deletions.
31 changes: 8 additions & 23 deletions src/CJSImportProcessor.ts
View file
Expand Up @@ -5,6 +5,7 @@ import {isDeclaration} from "./parser/tokenizer";
import {ContextualKeyword} from "./parser/tokenizer/keywords";
import {TokenType as tt} from "./parser/tokenizer/types";
import type TokenProcessor from "./TokenProcessor";
import getImportExportSpecifierInfo from "./util/getImportExportSpecifierInfo";
import {getNonTypeIdentifiers} from "./util/getNonTypeIdentifiers";

interface NamedImport {
Expand Down Expand Up @@ -359,31 +360,15 @@ export default class CJSImportProcessor {
break;
}

// Flow type imports should just be ignored.
let isTypeImport = false;
if (
(this.tokens.matchesContextualAtIndex(index, ContextualKeyword._type) ||
this.tokens.matches1AtIndex(index, tt._typeof)) &&
this.tokens.matches1AtIndex(index + 1, tt.name) &&
!this.tokens.matchesContextualAtIndex(index + 1, ContextualKeyword._as)
) {
isTypeImport = true;
index++;
const specifierInfo = getImportExportSpecifierInfo(this.tokens, index);
index = specifierInfo.endIndex;
if (!specifierInfo.isType) {
namedImports.push({
importedName: specifierInfo.leftName,
localName: specifierInfo.rightName,
});
}

const importedName = this.tokens.identifierNameAtIndex(index);
let localName;
index++;
if (this.tokens.matchesContextualAtIndex(index, ContextualKeyword._as)) {
index++;
localName = this.tokens.identifierNameAtIndex(index);
index++;
} else {
localName = importedName;
}
if (!isTypeImport) {
namedImports.push({importedName, localName});
}
if (this.tokens.matches2AtIndex(index, tt.comma, tt.braceR)) {
index += 2;
break;
Expand Down
77 changes: 77 additions & 0 deletions src/parser/plugins/typescript.ts
View file
@@ -1,5 +1,6 @@
import {
eat,
IdentifierRole,
lookaheadType,
lookaheadTypeAndKeyword,
match,
Expand Down Expand Up @@ -1256,6 +1257,82 @@ export function tsTryParseExport(): boolean {
}
}

/**
* Parse a TS import specifier, which may be prefixed with "type" and may be of
* the form `foo as bar`.
*
* The number of identifier-like tokens we see happens to be enough to uniquely
* identify the form, so simply count the number of identifiers rather than
* matching the words `type` or `as`. This is particularly important because
* `type` and `as` could each actually be plain identifiers rather than
* keywords.
*/
export function tsParseImportSpecifier(): void {
parseIdentifier();
if (match(tt.comma) || match(tt.braceR)) {
// import {foo}
state.tokens[state.tokens.length - 1].identifierRole = IdentifierRole.ImportDeclaration;
return;
}
parseIdentifier();
if (match(tt.comma) || match(tt.braceR)) {
// import {type foo}
state.tokens[state.tokens.length - 1].identifierRole = IdentifierRole.ImportDeclaration;
state.tokens[state.tokens.length - 2].isType = true;
state.tokens[state.tokens.length - 1].isType = true;
return;
}
parseIdentifier();
if (match(tt.comma) || match(tt.braceR)) {
// import {foo as bar}
state.tokens[state.tokens.length - 3].identifierRole = IdentifierRole.ImportAccess;
state.tokens[state.tokens.length - 1].identifierRole = IdentifierRole.ImportDeclaration;
return;
}
parseIdentifier();
// import {type foo as bar}
state.tokens[state.tokens.length - 3].identifierRole = IdentifierRole.ImportAccess;
state.tokens[state.tokens.length - 1].identifierRole = IdentifierRole.ImportDeclaration;
state.tokens[state.tokens.length - 4].isType = true;
state.tokens[state.tokens.length - 3].isType = true;
state.tokens[state.tokens.length - 2].isType = true;
state.tokens[state.tokens.length - 1].isType = true;
}

/**
* Just like named import specifiers, export specifiers can have from 1 to 4
* tokens, inclusive, and the number of tokens determines the role of each token.
*/
export function tsParseExportSpecifier(): void {
parseIdentifier();
if (match(tt.comma) || match(tt.braceR)) {
// export {foo}
state.tokens[state.tokens.length - 1].identifierRole = IdentifierRole.ExportAccess;
return;
}
parseIdentifier();
if (match(tt.comma) || match(tt.braceR)) {
// export {type foo}
state.tokens[state.tokens.length - 1].identifierRole = IdentifierRole.ExportAccess;
state.tokens[state.tokens.length - 2].isType = true;
state.tokens[state.tokens.length - 1].isType = true;
return;
}
parseIdentifier();
if (match(tt.comma) || match(tt.braceR)) {
// export {foo as bar}
state.tokens[state.tokens.length - 3].identifierRole = IdentifierRole.ExportAccess;
return;
}
parseIdentifier();
// export {type foo as bar}
state.tokens[state.tokens.length - 3].identifierRole = IdentifierRole.ExportAccess;
state.tokens[state.tokens.length - 4].isType = true;
state.tokens[state.tokens.length - 3].isType = true;
state.tokens[state.tokens.length - 2].isType = true;
state.tokens[state.tokens.length - 1].isType = true;
}

export function tsTryParseExportDefaultExpression(): boolean {
if (isContextual(ContextualKeyword._abstract) && lookaheadType() === tt._class) {
state.type = tt._abstract;
Expand Down
21 changes: 17 additions & 4 deletions src/parser/traverser/statement.ts
View file
Expand Up @@ -23,8 +23,10 @@ import {
tsAfterParseVarHead,
tsIsDeclarationStart,
tsParseExportDeclaration,
tsParseExportSpecifier,
tsParseIdentifierStatement,
tsParseImportEqualsDeclaration,
tsParseImportSpecifier,
tsParseMaybeDecoratorArguments,
tsParseModifiers,
tsStartParseFunctionParams,
Expand Down Expand Up @@ -1059,12 +1061,19 @@ export function parseExportSpecifiers(): void {
break;
}
}
parseExportSpecifier();
}
}

function parseExportSpecifier(): void {
if (isTypeScriptEnabled) {
tsParseExportSpecifier();
return;
}
parseIdentifier();
state.tokens[state.tokens.length - 1].identifierRole = IdentifierRole.ExportAccess;
if (eatContextual(ContextualKeyword._as)) {
parseIdentifier();
state.tokens[state.tokens.length - 1].identifierRole = IdentifierRole.ExportAccess;
if (eatContextual(ContextualKeyword._as)) {
parseIdentifier();
}
}
}

Expand Down Expand Up @@ -1164,6 +1173,10 @@ function parseImportSpecifiers(): void {
}

function parseImportSpecifier(): void {
if (isTypeScriptEnabled) {
tsParseImportSpecifier();
return;
}
if (isFlowEnabled) {
flowParseImportSpecifier();
return;
Expand Down
15 changes: 6 additions & 9 deletions src/transformers/CJSImportTransformer.ts
View file
Expand Up @@ -9,6 +9,7 @@ import getDeclarationInfo, {
DeclarationInfo,
EMPTY_DECLARATION_INFO,
} from "../util/getDeclarationInfo";
import getImportExportSpecifierInfo from "../util/getImportExportSpecifierInfo";
import shouldElideDefaultExport from "../util/shouldElideDefaultExport";
import type ReactHotLoaderTransformer from "./ReactHotLoaderTransformer";
import type RootTransformer from "./RootTransformer";
Expand Down Expand Up @@ -742,17 +743,13 @@ export default class CJSImportTransformer extends Transformer {
break;
}

const localName = this.tokens.identifierName();
let exportedName;
this.tokens.removeToken();
if (this.tokens.matchesContextual(ContextualKeyword._as)) {
this.tokens.removeToken();
exportedName = this.tokens.identifierName();
const specifierInfo = getImportExportSpecifierInfo(this.tokens);
while (this.tokens.currentIndex() < specifierInfo.endIndex) {
this.tokens.removeToken();
} else {
exportedName = localName;
}
if (!this.shouldElideExportedIdentifier(localName)) {
if (!specifierInfo.isType && !this.shouldElideExportedIdentifier(specifierInfo.leftName)) {
const localName = specifierInfo.leftName;
const exportedName = specifierInfo.rightName;
const newLocalName = this.importProcessor.getIdentifierReplacement(localName);
exportStatements.push(`exports.${exportedName} = ${newLocalName || localName};`);
}
Expand Down
83 changes: 15 additions & 68 deletions src/transformers/ESMImportTransformer.ts
View file
Expand Up @@ -8,6 +8,7 @@ import getDeclarationInfo, {
DeclarationInfo,
EMPTY_DECLARATION_INFO,
} from "../util/getDeclarationInfo";
import getImportExportSpecifierInfo from "../util/getImportExportSpecifierInfo";
import {getNonTypeIdentifiers} from "../util/getNonTypeIdentifiers";
import shouldElideDefaultExport from "../util/shouldElideDefaultExport";
import type ReactHotLoaderTransformer from "./ReactHotLoaderTransformer";
Expand Down Expand Up @@ -190,68 +191,22 @@ export default class ESMImportTransformer extends Transformer {
} else if (this.tokens.matches1(tt.braceL)) {
this.tokens.copyToken();
while (!this.tokens.matches1(tt.braceR)) {
if (
this.tokens.matches3(tt.name, tt.name, tt.comma) ||
this.tokens.matches3(tt.name, tt.name, tt.braceR)
) {
// type foo
this.tokens.removeToken();
this.tokens.removeToken();
if (this.tokens.matches1(tt.comma)) {
const specifierInfo = getImportExportSpecifierInfo(this.tokens);
if (specifierInfo.isType || this.isTypeName(specifierInfo.rightName)) {
while (this.tokens.currentIndex() < specifierInfo.endIndex) {
this.tokens.removeToken();
}
} else if (
this.tokens.matches5(tt.name, tt.name, tt.name, tt.name, tt.comma) ||
this.tokens.matches5(tt.name, tt.name, tt.name, tt.name, tt.braceR)
) {
// type foo as bar
this.tokens.removeToken();
this.tokens.removeToken();
this.tokens.removeToken();
this.tokens.removeToken();
if (this.tokens.matches1(tt.comma)) {
this.tokens.removeToken();
}
} else if (
this.tokens.matches2(tt.name, tt.comma) ||
this.tokens.matches2(tt.name, tt.braceR)
) {
// foo
if (this.isTypeName(this.tokens.identifierName())) {
this.tokens.removeToken();
if (this.tokens.matches1(tt.comma)) {
this.tokens.removeToken();
}
} else {
foundNonTypeImport = true;
} else {
foundNonTypeImport = true;
while (this.tokens.currentIndex() < specifierInfo.endIndex) {
this.tokens.copyToken();
if (this.tokens.matches1(tt.comma)) {
this.tokens.copyToken();
}
}
} else if (
this.tokens.matches4(tt.name, tt.name, tt.name, tt.comma) ||
this.tokens.matches4(tt.name, tt.name, tt.name, tt.braceR)
) {
// foo as bar
if (this.isTypeName(this.tokens.identifierNameAtIndex(this.tokens.currentIndex() + 2))) {
this.tokens.removeToken();
this.tokens.removeToken();
this.tokens.removeToken();
if (this.tokens.matches1(tt.comma)) {
this.tokens.removeToken();
}
} else {
foundNonTypeImport = true;
this.tokens.copyToken();
this.tokens.copyToken();
if (this.tokens.matches1(tt.comma)) {
this.tokens.copyToken();
if (this.tokens.matches1(tt.comma)) {
this.tokens.copyToken();
}
}
} else {
throw new Error("Unexpected import form.");
}
}
this.tokens.copyExpectedToken(tt.braceR);
Expand Down Expand Up @@ -313,26 +268,18 @@ export default class ESMImportTransformer extends Transformer {
this.tokens.copyExpectedToken(tt.braceL);

while (!this.tokens.matches1(tt.braceR)) {
if (!this.tokens.matches1(tt.name)) {
throw new Error("Expected identifier at the start of named export.");
}
if (this.shouldElideExportedName(this.tokens.identifierName())) {
while (
!this.tokens.matches1(tt.comma) &&
!this.tokens.matches1(tt.braceR) &&
!this.tokens.isAtEnd()
) {
const specifierInfo = getImportExportSpecifierInfo(this.tokens);
if (specifierInfo.isType || this.shouldElideExportedName(specifierInfo.leftName)) {
// Type export, so remove all tokens, including any comma.
while (this.tokens.currentIndex() < specifierInfo.endIndex) {
this.tokens.removeToken();
}
if (this.tokens.matches1(tt.comma)) {
this.tokens.removeToken();
}
} else {
while (
!this.tokens.matches1(tt.comma) &&
!this.tokens.matches1(tt.braceR) &&
!this.tokens.isAtEnd()
) {
// Non-type export, so copy all tokens, including any comma.
while (this.tokens.currentIndex() < specifierInfo.endIndex) {
this.tokens.copyToken();
}
if (this.tokens.matches1(tt.comma)) {
Expand All @@ -346,7 +293,7 @@ export default class ESMImportTransformer extends Transformer {

/**
* ESM elides all imports with the rule that we only elide if we see that it's
* a type and never see it as a value. This is in contract to CJS, which
* a type and never see it as a value. This is in contrast to CJS, which
* elides imports that are completely unknown.
*/
private shouldElideExportedName(name: string): boolean {
Expand Down

0 comments on commit b9bce13

Please sign in to comment.