Support syntax highlighting with tree-sitter

[fcurts]

Please consider supporting tree-sitter grammars in addition to TextMate grammars. TextMate grammars are incredibly difficult to author and maintain and impossible to get right. The over 500 (!) issues reported against https://github.com/Microsoft/TypeScript-TmLanguage are a living proof of this.

This presentation explains the motivation and goals for tree-sitter: https://www.youtube.com/watch?v=a1rC79DHpmY

tree-sitter already ships with Atom and is also used on github.com.

[aeschli]

tree-sitter is cool technology, and we have our eyes on it.
I fully agree with you that TextMate grammars are challenging to implement and have limitations, but it's always a lot of work to create and maintain a grammar. That will not be different with Tree-Sitter.

If you already have experiences with specific grammars, e.g. the TypeScript grammar or the C-grammar, and think it is superior to the TextMate grammars, let us know. That would be the criteria for us to invest.

[Kroc]

This may help in the future with the whole 'embedding one language in another', which is an enfant terrible when it comes to TextMate grammars.

[omniomi]

There's also a request in #5408 for .sublime-syntax which has been open since Apr 2016 which would also be a step up from .tmLanguage.

While tree-sitter has an awesome concept I can't say the idea of writing grammar in JavaScript is all that appealing.

[fcurts]

@omniomi tree-sitter also supports writing grammars in pure JSON if that's what you prefer. The main & dramatic advantage of tree-sitter is that it's a full parsing system and not an ad-hoc, underspecified, horrifyingly complex yet extremely limited regex contraption.

[ahuertabhg]

Integrating tree-sitter would help solve this issue dotnet/vscode-csharp#2461

[sean-mcmanus]

@aeschli Atom has switched to tree sitter for C++ and no longer fixing issues with Text Mate: atom/language-c#232 (comment) . Please advise on how we should proceed for improving the C++ syntax highlighting/etc. experience.

[maxbrunsfeld]

👋 Just to reiterate - the Atom team doesn't intend to disrupt other apps like VSCode that are using modules like language-c. We will definitely continue to accept good PRs that update the text-mate grammar.

The reason that we've been closing issues like that is just to be explicit about the fact that our team won't be prioritizing work on them in the future, since Atom is moving away from text-mate grammars.

[bobbrow]

@sean-mcmanus we already have our own syntax highlighting stuff (shared with Visual Studio), but haven't been able to use it because we are waiting on an API that lets us turn off tmLanguage and provide the coloring ourselves: #585. Moving to tree-sitter is only relevant to us so long as #585 is incomplete.

[bklebe]

Tree-sitter is extensible for other programming languages, and in particular already supports Rust and Ruby as well. Are the Visual Studio APIs ready to be extended with new language support in those ways?