Web Tree-sitter

WebAssembly bindings to the Tree-sitter parsing library.

Setup

You can download the the tree-sitter.js and tree-sitter.wasm files from the latest GitHub release and load them using a standalone script:

<script src="/the/path/to/tree-sitter.js"/>

<script>
  const Parser = window.TreeSitter;
  Parser.init().then(() => { /* the library is ready */ });
</script>

You can also install the web-tree-sitter module from NPM and load it using a system like Webpack:

const Parser = require('web-tree-sitter');
Parser.init().then(() => { /* the library is ready */ });

Basic Usage

First, create a parser:

const parser = new Parser;

Then assign a language to the parser. Tree-sitter languages are packaged as individual .wasm files (more on this below):

const JavaScript = await Parser.Language.load('/path/to/tree-sitter-javascript.wasm');
parser.setLanguage(JavaScript);

Now you can parse source code:

const sourceCode = 'let x = 1; console.log(x);';
const tree = parser.parse(sourceCode);

and inspect the syntax tree.

console.log(tree.rootNode.toString());

// (program
//   (lexical_declaration
//     (variable_declarator (identifier) (number)))
//   (expression_statement
//     (call_expression
//       (member_expression (identifier) (property_identifier))
//       (arguments (identifier)))))

const callExpression = tree.rootNode.child(1).firstChild;
console.log(callExpression);

// { type: 'call_expression',
//   startPosition: {row: 0, column: 16},
//   endPosition: {row: 0, column: 30},
//   startIndex: 0,
//   endIndex: 30 }

Editing

If your source code changes, you can update the syntax tree. This will take less time than the first parse.

// Replace 'let' with 'const'
const newSourceCode = 'const x = 1; console.log(x);';

tree.edit({
  startIndex: 0,
  oldEndIndex: 3,
  newEndIndex: 5,
  startPosition: {row: 0, column: 0},
  oldEndPosition: {row: 0, column: 3},
  newEndPosition: {row: 0, column: 5},
});

const newTree = parser.parse(newSourceCode, tree);

Parsing Text From a Custom Data Structure

If your text is stored in a data structure other than a single string, you can parse it by supplying a callback to parse instead of a string:

const sourceLines = [
  'let x = 1;',
  'console.log(x);'
];

const tree = parser.parse((index, position) => {
  let line = sourceLines[position.row];
  if (line) return line.slice(position.column);
});

Generate .wasm language files

The following example shows how to generate .wasm file for tree-sitter JavaScript grammar.

IMPORTANT: emscripten or docker need to be installed.

First install tree-sitter-cli and the tree-sitter language for which to generate .wasm (tree-sitter-javascript in this example):

npm install --save-dev tree-sitter-cli tree-sitter-javascript

Then just use tree-sitter cli tool to generate the .wasm.

npx tree-sitter build-wasm node_modules/tree-sitter-javascript

If everything is fine, file tree-sitter-javascript.wasm should be generated in current directory.

Running .wasm in Node.js

Notice that executing .wasm files in node.js is considerably slower than running node.js bindings. However could be useful for testing purposes:

const Parser = require('web-tree-sitter');

(async () => {
  await Parser.init();
  const parser = new Parser();
  const Lang = await Parser.Language.load('tree-sitter-javascript.wasm');
  parser.setLanguage(Lang);
  const tree = parser.parse('let x = 1;');
  console.log(tree.rootNode.toString());
})();