Adds the transformFiles API for tree transformations

[rbuckton]

The transformFiles function will take on some of the responsibilities of the existing emitter, and integrates it with our new visitor pattern.

This function takes in an array of Transformer callbacks, which have the following signature (found in types.ts):

export type Transformer = (context: TransformationContext) => (node: SourceFile) => SourceFile;

The TransformationContext is allocated at the start of transformFiles, and each Transformer is called sequentially with this context to initialize each transformer. The result of each initialization is a callback that will be used to transform each SourceFile.

The TransformationContext provides the following information/capabilities to a Transformer:

• Access to compiler options.
• Access to the emit resolver.
• The ability to generate unique names for nodes.
• The ability to hoist variable and function declarations (such as temporary variables, or hoisting declarations in SystemJS).
• The ability to explicitly enter and exit a lexical environment (used to track hoisted declarations).
• The ability to associate additional transformation- and emit-specific information and flags to a node.
• Hooks to perform certain just-in-time substitutions during the final print phase of the tree transforming emitter.

While TransformFlags provides us the ability to quickly make decisions about whether (and how) to transform a node, there are some transformations that would still require a full walk of the tree. This includes situations such as renaming block-scoped let/const declarations. As we do not want to do a full walk of the tree for every transformation phase, we save these final transformations until the last possible moment before we emit. We can do this by using the identifierSubstitution and expressionSubstitution hooks provided on the TransformationContext. The final print phase will then evaluate these substitutions on a just-in-time basis immediately before printing the node.

The following example builds on the examples in #6892 and #6983:

function createES7Transformer(context: TransformationContext) {
  const { 
    startLexicalEnvironment, 
    endLexicalEnvironment
  } = context;

  let currentSourceFile: SourceFile;

  // hook substitutions
  const previousIdentifierSubstitution = context.identifierSubstitution;
  context.identifierSubstitution = substituteIdentifier;

  const previousExpressionSubstitution = context.expressionSubstitution;
  context.expressionSubstitution = substituteExpression;

  function transformSourceFile(node: SourceFile) {
    const savedSourceFile = currentSourceFile;
    const visited = visitEachChild(node, visitor, /*environment*/ context);
    currentSourceFile = savedSourceFile;
    return visited;
  }

  function visitor(node: Node) {
    if (node.transformFlags & TransformFlags.ES7) {
      // This node is an ES7 node and needs a specific transformation
      return visitorWorker(node);
    }
    else if (node.transformFlags & TransformFlags.ContainsES7) {
      // This node contains an ES7 node somewhere in its subtree, so we must walk the subtree
      // NOTE: Added third argument to the following call to support entering and exiting
      //       lexical environments.
      return visitEachChild(node, visitor, /*environment*/ context);
    }
    else {
      // Neither this node, nor its subtree, require any transformation, so we can simply 
      // reuse the node and its subtree and avoid a costly walk
      return node;
    }
  }

  function visitorWorker(node: Node) {
    ...
  }

  ...

  function substituteIdentifier(node: Identifier) {
    // Evaluate any previous substitutions
    node = previousIdentifierSubstitution ? previousIdentifierSubstitution(node) : node;
    // Perform any other substitution we may need.
    // NOTE: You cannot rely on `parent` pointers on nodes created during transformation.
    // NOTE: You may be able to use `getOriginalNode` on a node to get to its representation in 
    //       the original source tree.
    return node;
  }

  function substituteExpression(node: Expression) {
    // Evaluate any previous substitutions
    node = previousExpressionSubstitution ? previousExpressionSubstitution(node) : node;
    // Perform any other substitution we may need.
    // NOTE: You cannot rely on `parent` pointers on nodes created during transformation.
    // NOTE: You may be able to use `getOriginalNode` on a node to get to its representation in 
    //       the original source tree.
    return node;
  }
}

Related Pull Requests:

• Adds the visitor API foundation for tree transformations. #6892 - Adds the visitor API foundation for tree transformations.
• Adds the Transform Flags concept for tree transformations #6983 - Adds the Transform Flags concept for tree transformations.
• Adds the transformFiles API for tree transformations #6986 - Adds the transformFiles API for tree transformations.
• Adds a simplified pretty printer for tree transformations #6987 - Adds a simplified pretty printer for tree transformations.
• Adds the TypeScript transformer #7191 - Adds the TypeScript transformer.
• Adds the JSX transformer #7192 - Adds the JSX transformer.
• Adds the ES7 transformer #7193 - Adds the ES7 transformer.
• Adds the ES6 transformer #7194 - Adds the ES6 transformer.
• Adds the Module transformers #7206 - Adds the Module transformer.

[rbuckton]

Paging for review: @mhegazy, @ahejlsberg, @yuit, @DanielRosenwasser, @RyanCavanaugh, @vladima

[ahejlsberg]

I don't understand the purpose of this one.

[rbuckton]

I will most likely remove these and inline them where necessary. All of the "coerce" functions are helpers internal to "factory.ts" that allow you to write createParameter("args") instead of createParameter(createIdentifier("args")), etc.