New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Tree-sitter parsers to be used for syntax highlighting and code folding #16299

Merged
merged 51 commits into from Jan 8, 2018

Conversation

Projects
None yet
10 participants
@maxbrunsfeld
Contributor

maxbrunsfeld commented Nov 30, 2017

Overview

This pull request adds a new config setting to Atom: core.useTreeSitterParsers, which defaults to false. If you set that setting to true, when editing files written in certain languages, Atom will use a new parsing system called Tree-sitter to provide improved syntax highlighting and code folding.

Syntax Highlighting Improvements

Using the syntax trees provided by Tree-sitter, we can produce syntax highlighting that is more consistent and specific than before. Different kinds of names in your code like types, functions, properties/fields, and local variables should be grouped by color in a more consistent way.

The new syntax highlighting uses the same CSS classes as the old system, so you should be able to continue using your current themes without disruption.

Code Folding Improvements

Currently, Atom's code folding is based on indentation: in a fold, lines with greater indentation are hidden. This usually gives good results, but it is unintuitive in some cases. Now, by basing the code folding on the syntax tree provided by Tree-sitter, we can fold code in an intuitive way regardless of its indentation.

Example

Code like this:

int add(int left,
        int right) {
  return left + right;
}

would previously fold like this:

int add(int a…

Now, we are able to fold it like this:

int add(int left,
        int right) {…}

or, if you choose to fold the first two lines:

int add(…) {…}

New Editing Commands

This PR adds two new text editing commands: editor:select-larger-syntax-node and editor:select-smaller-syntax-node. These commands can be used to quickly select syntactically meaningful units of code:

Example

extend-selection-2

Multi-Cursor Example

extend-selection-3

Supported Languages

Initially, the following languages will be supported:

Next Steps

The new parsing system is disabled by default because there are some known bugs and it needs more testing before we roll it out to all Atom users. After this PR lands, I'll be focused on fixing bugs in the new parsing system and the existing grammars so that soon, we can enable the new parsing system by default.

maxbrunsfeld added some commits Nov 30, 2017

Add initial TreeSitterLanguageMode implementation
Much of this is from the tree-sitter-syntax package.
Also, add a dependency on the tree-sitter module.
@alexandernst

This comment has been minimized.

Show comment
Hide comment
@alexandernst

alexandernst Nov 30, 2017

\o/ bravo  \o/

alexandernst commented Nov 30, 2017

\o/ bravo  \o/

@calebmeyer

This comment has been minimized.

Show comment
Hide comment
@calebmeyer

calebmeyer Dec 7, 2017

@maxbrunsfeld I noticed you have some commits on https://github.com/tree-sitter/tree-sitter-ruby, but it's not included in the list of initial languages. Do you have a sense of the scope of the work to get it integrated? Is this something there is interest in?

calebmeyer commented Dec 7, 2017

@maxbrunsfeld I noticed you have some commits on https://github.com/tree-sitter/tree-sitter-ruby, but it's not included in the list of initial languages. Do you have a sense of the scope of the work to get it integrated? Is this something there is interest in?

@mkutny

This comment has been minimized.

Show comment
Hide comment
@mkutny

mkutny Mar 25, 2018

@maxbrunsfeld , really decent feature, thanks for developing it!

Have noticed that the following snipped gets folded non-intuitively (as to my eyes). Want to know if it works as designed or I should open an issue.

Code likes this:

function add({
  a, b
}) {
  return a + b;
}

Folds into:

function add({…}) {
  return a + b;
}

with no possibility to fold further (no angle bracket to the left).

mkutny commented Mar 25, 2018

@maxbrunsfeld , really decent feature, thanks for developing it!

Have noticed that the following snipped gets folded non-intuitively (as to my eyes). Want to know if it works as designed or I should open an issue.

Code likes this:

function add({
  a, b
}) {
  return a + b;
}

Folds into:

function add({…}) {
  return a + b;
}

with no possibility to fold further (no angle bracket to the left).

@despairblue despairblue referenced this pull request Mar 29, 2018

Open

breaks in atom 1.25 #37

DSpeckhals added a commit to DSpeckhals/python-indent that referenced this pull request Mar 29, 2018

Allow new tree-sitter functionality
This commit adds the newer `scopeName` for Python. With the new syntax
parsing implemented with Tree Sitter
(atom/atom#16299,
atom/language-python#220), this new scope name
was added. Otherwise, the indentation seems to work as expected.

DSpeckhals added a commit to DSpeckhals/python-indent that referenced this pull request Mar 29, 2018

Allow new tree-sitter functionality (#61)
This commit adds the newer `scopeName` for Python. With the new syntax
parsing implemented with Tree Sitter
(atom/atom#16299,
atom/language-python#220), this new scope name
was added. Otherwise, the indentation seems to work as expected.
@maxbrunsfeld

This comment has been minimized.

Show comment
Hide comment
@maxbrunsfeld

maxbrunsfeld Apr 21, 2018

Contributor

@mkutny Ah yeah, that's a bug: Atom doesn't know what kind of 'fold arrow' to show in the gutter if a line is foldable and already has one fold. You should still be able to fold the body of the function with the keyboard though: Editor: Fold Current Row is the command; cmd-alt-[ on mac.

Contributor

maxbrunsfeld commented Apr 21, 2018

@mkutny Ah yeah, that's a bug: Atom doesn't know what kind of 'fold arrow' to show in the gutter if a line is foldable and already has one fold. You should still be able to fold the body of the function with the keyboard though: Editor: Fold Current Row is the command; cmd-alt-[ on mac.

@mkutny

This comment has been minimized.

Show comment
Hide comment
@mkutny

mkutny Apr 21, 2018

Yeah, that's understandable.

Still, one would expect the whole function to collapse, not only its args.

From grammar perspective

function add({
  a, b
}) {
  return a + b;
}

is equivalent to

function add({ a, b }) {
  return a + b;
}

but folding function line gives different results in these two cases.

mkutny commented Apr 21, 2018

Yeah, that's understandable.

Still, one would expect the whole function to collapse, not only its args.

From grammar perspective

function add({
  a, b
}) {
  return a + b;
}

is equivalent to

function add({ a, b }) {
  return a + b;
}

but folding function line gives different results in these two cases.

@maxbrunsfeld

This comment has been minimized.

Show comment
Hide comment
@maxbrunsfeld

maxbrunsfeld Jul 19, 2018

Contributor

@calebmeyer Just an update on the Ruby parser. It took a long time because of some internal issues, but it's now working well and will be included in Atom 1.30.

Contributor

maxbrunsfeld commented Jul 19, 2018

@calebmeyer Just an update on the Ruby parser. It took a long time because of some internal issues, but it's now working well and will be included in Atom 1.30.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment