Another batch of Clojure enhancements

[mauricioszabo]

Replaces #663 because this is more complete.

That was a journey, but basically Clojure highlights some interesting things:

1. Added a different Clojure grammar - https://github.com/mauricioszabo/tree-sitter-clojure - that adds quotes for strings (so we can add injection points on strings)
2. Highlights some interesting cases like syntax quoting and unquoting (a screenshot with a custom CSS to highlight these scopes):
   
3. Metadata highlighting (again, with custom CSS):
   

Missing:

• Make EDN grammar behave like Clojure (most plug-ins don't work if you don't detect the filetype as Clojure)
• Check if there are performance issues that can be solved in this new scope resolver
• Changelog update

[Spiker985]

Looks like this is still set up for the legacy tree-sitter implementation, it would need to use the parserSource key in order to be a "new tree-sitter" grammar.

It also looks like this adds some new functions to some base files - won't those potentially break other tree-sitter grammars?

[mauricioszabo]

@Spiker985 there's no legacy Clojure tree-sitter actually, so I'm unsure what you mean with this options.

As for the new options, they are just new things we support, if other grammars don't use these they won't be affected. But before everything, I need to fix these grammars (or even remove what I did) - injection points are not correct, and they are janky when editing these forms...

[savetheclocktower]

Looks like this is still set up for the legacy tree-sitter implementation, it would need to use the parserSource key in order to be a "new tree-sitter" grammar.

parserSource is a key that doesn't have a function right now except to help the user understand where a compiled WASM file comes from, and when it might need updating. That said, it does need to be present in order to make CI happy, which is why the “Validate WASM Grammar PR Changes” check is failing.

[savetheclocktower]

It also looks like this adds some new functions to some base files - won't those potentially break other tree-sitter grammars?

The changes introduced to core don't change how any existing features behave. We've got

• A new scope test called ancestorTypeNearerThan (which I think I proposed a while back as a way of solving a problem that Clojure had; it makes several different tests of node ancestry, and without something like this, they're bound to conflict with one another unless you can tell which one is the closest ancestor).
• A new argument to the content callback of atom.grammars.addInjectionPoint, which won't bother anything that isn't expecting a second argument. I didn't dig too deeply into how it's used in the PR because Clojure scares me, but including the Buffer as a second argument is fine with me, and doesn't have any real downsides that I can think of.

If the tests pass, I'm good.

[DeeDeeG]

We can try merging in the macOS CI fix from master branch if wanting a proper CI run for this PR.

[savetheclocktower]

You're almost certainly not better off converting the rest to a Set. Maybe if you were caching these items by string value (which might not be a bad idea), but otherwise I suspect it's quicker to keep the rest as an array and do a rejected.includes(current.type) test instead.

Probably not a huge deal, but I'd suggest doing a profiling stress test here. Any predicate that loops is worth optimizing.

[mauricioszabo]

So, I did profile - it indeed makes no difference, so I went with the Set because it's simpler.

What I found is that what a lot of nodes (like, 1 million tests for example) the Set implementation is faster, even when the Set contains 1 or 2 elements only. But Pulsar doesn't even supports 1 million "parent" nodes, so we're safe here :)

[DeeDeeG]

More or less rubber-stamp approved from me, hopefully others on the team with more knowledge in this area are happy with the PR now as well?

If they have any feedback, I defer to them, however.