stylites: tree-sitter bazelized

A bazel-enabled fork of tree-sitter.

Simeon Stylites was not a tree-sitter, but he was the original column-sitter.

Warning

EXPERIMENTAL! Source code is reorganized in order to make the build structure clearer, support better Bazel integration, and in general remove dependency on Rust (except where it is unavoidable, i.e. the parser generator itself). The bits of the tree-sitter-cli command processor that involve build functionality are migrated to bazel. Instead of parse and query subcommands of a CLI utility, we support a repl (planned). Etc.

Warning

the tree-sitter-cli tool is not used with Bazel. Equivalent functionalities are exposed via Bazel build, run, and test targets.

BAZEL STATUS: Very lightly tested, on MacOS only.

Step 1: $ CARGO_BAZEL_REPIN=1 bazel sync --only=crate-index

code organization

• src/runtime - the C runtime plus rust and wasm bindings.

• src/generator - the Rust module that generates parsers. Extracted from the code implementing the tree-sitter generate subcommand.

• tools - source code for the cli tool, and the highlight and tags modules. Still deciding what to do with this stuff.

• new - plan is to support cmds like bazel run @tree-sitter//new:grammar to generate the core bits of a grammar development package in a Bazel repo, including build targets to generate, compile, and test the parser.

• test/convert - plan is to implement a tool to convert "corpus" test files to bazel tests using cc_test and the Unity test framework.

build targets

• src/runtime:tree-sitter - the C runtime library.

• src/runtime/bindings/rust:tree-sitter - Rust bindings for the C runtime

• src/runtime/bindings/wasm:tree-sitter - wasm bindings for the C runtime (not yet implemented)

• //examples/imp contains an implementation of the [imp](https://softwarefoundations.cis.upenn.edu/lf-current/Imp.html) language (grammar from [How to write a tree-sitter grammar in an afternoon](https://siraben.dev/2022/03/01/tree-sitter.html). Builds the parser as a cc_library and includes a cc_test target that uses [Unity](https://github.com/ThrowTheSwitch/Unity).

• bazel run examples/imp:repl. This will launch a skeletal repl that currently just reads and echoes input. The plan is to support interactive parsing and querying, to replace the tree-sitter-cli command processor for those functions.

Original readme:

Tree-sitter is a parser generator tool and an incremental parsing library. It can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited. Tree-sitter aims to be:

• General enough to parse any programming language

• Fast enough to parse on every keystroke in a text editor

• Robust enough to provide useful results even in the presence of syntax errors

• Dependency-free so that the runtime library (which is written in pure C) can be embedded in any application

Links

• [Documentation](https://tree-sitter.github.io)

• [Rust binding](lib/binding_rust/README.md)

• [WASM binding](lib/binding_web/README.md)

• [Command-line interface](cli/README.md)