AST: getText, getStart and getEnd crash when using createProgram

[theovosse]

TypeScript Version: 2.2.1

When walking the AST, it is possible to call getText() in scenario 1, but not in scenario 2

Scenario 1

sourceFile = ts.createSourceFile(fileName, readFileSync(fileName).toString(), ts.ScriptTarget.ES2016, /*setParentNodes */ true);
processFile(sourceFile);

Scenario 2

program = ts.createProgram(fileName, {});
sourceFile = program.getSourceFile(fileName);
processFile(sourceFile);

Expected behavior:
That node.getText() returns the text of an identifier both in scenario 1 and scenario 2.

Actual behavior:
Exception in TokenOrIdentifierObject.prototype.getText, most likely because this.parent is undefined, in scenario 2. Scenario 1 works as expected.

Additional comments
It also seems that some AST nodes have a text field containing the text, but this field is missing from the type declaration (the declared type is Node, but the internal type is sometimes a NodeObject and other times a TokenObject, both of which aren't declared and can't be used in a cast). Documentation would be most welcome.

[mhegazy]

both patterns should be setting the parent pointers. how are you calling createProgram? can you share a sample?

[vladima]

for createProgram case parent pointers will be set on demand during binding (we delay it since binding needs to walk the entire file which might be expensive). Binding in turn will be performed when typechecker is created so a a workaround after you create the program you can call getTypeChecker method on program to force binding.

[ajafff]

It also seems that some AST nodes have a text field containing the text, but this field is missing from the type declaration (the declared type is Node, but the internal type is sometimes a NodeObject and other times a TokenObject, both of which aren't declared and can't be used in a cast). Documentation would be most welcome.

@flwrd NodeObject and TokenObject are just the internal constructor names. Nodes actually have many different shapes. They are all defined in src/compiler/types.ts. You can distinguish them by kind and then cast to the correct interface.
As an alternative you can use typeguard functions, either write your own or just use a package and let someone else maintain them for you 😉

You mentioned getting the text of an identifier:

import {isIdentifier} from 'tsutils';
declare var node: ts.Node;
if (isIdentifier(node))
    console.log(node.text);
    // typescript "escapes" some identifiers.
    // to be sure you get the right text, you should use ts.unescapeIdentifier(node.text),
    // but that may not yet be available in your version of typescript.
    // see #14013 for details

[theovosse]

Sorry for the delay. Here is a sample:

/// <reference path="node_modules/@types/node/index.d.ts" />

import * as ts from "typescript";

let sourceFile: ts.SourceFile;

let level: number = 0;
function indent(n: number): string {
    let str = "";

    while (n-- > 0) {
        str += "  ";
    }
    return str;
}

const fileNames = process.argv.slice(2);

function printStructure(node: ts.Node): void {
    function dbgText(): string {
        let str = node.getText();
        let p = str.indexOf("\n");
        return p >= 0? str.slice(0, p - 1): str;
    }
    console.log(indent(level) + level + "." + ts.SyntaxKind[node.kind] + ": " + dbgText()); // !!!
    level++;
    ts.forEachChild(node, printStructure);
    level--;
}

let program = ts.createProgram(fileNames, {});
for (var fni = 0; fni < fileNames.length; fni++) {
    sourceFile = program.getSourceFile(fileNames[fni]);
    ts.forEachChild(sourceFile, printStructure);
}

But indeed, adding a call to program.getTypeChecker() does set the parent and solves the problem.

And thanks for the pointers to the type definitions/lib.