Unexpected result from query/capture (in Emacs 29)

[magthe]

I'm trying to run this query

(cabal (sections (common_decl _ @type name: (section_name)? @name)))

against a parse tree with the following shape

(cabal
 (cabal_version cabal-version : (spec_version))
 (properties ...)
 (sections
  (common_decl common
   name: (section_name)
   properties: 
    (property_block ...))))

I get the following result

((type . #<treesit-node "common" in 231-237>) (name . #<treesit-node section_name in 238-242>) (type . #<treesit-node section_name in 238-242>) (type . #<treesit-node property_block in 245-465>))

which surprised me a bit; I didn't expect the @type to capture multiple times within a node like that. Apparently this is by design, but I also found that I'm not the first one to be suprised. The problem now is that I can't seem to find a way to prevent multiple captures of @type.

Using anchoring I can reduce the result slightly

(cabal (sections (common_decl . _ @type name: (section_name)? @name)))

gives

((type . #<treesit-node "common" in 231-237>) (name . #<treesit-node section_name in 238-242>) (type . #<treesit-node section_name in 238-242>))

Isn't this actually a bug? The first child should be captured in @type, but here both the first and the second are.

Some more questions:

• I tried putting a field on an anonymous node, i.e. a rule like this field('type', 'common'), but the resulting parser doesn't put a field on the node. Is that intended?
• I'd really like to be able to compare Emacs' behaviour with tree-sitter query, but I'm not sure how to get it working. Are there instructions to be found somewhere?

[sogaiu]

About:

I'd really like to be able to compare Emacs' behaviour with tree-sitter query, but I'm not sure how to get it working. Are there instructions to be found somewhere?

What's worked for me is something like:

tree-sitter query <name-of-file-containing-query> <name-of-file-with-source-code>

FWIW, I leanred this via the invocation tree-sitter query --help.

[magthe]

Ah, I guess I was reading too much into the warning that I get

Warning: You have not configured any parser directories!
Please run `tree-sitter init-config` and edit the resulting
configuration file to indicate where we should look for
language grammars.

It does indeed match the behaviour in Emacs

❯ tree-sitter query query-file cabal-file
gaboon.cabal
  pattern: 0
    capture: 0 - type, start: (9, 0), end: (9, 6), text: `common`
    capture: 1 - name, start: (9, 7), end: (9, 11), text: `base`
  pattern: 0
    capture: 0 - type, start: (9, 7), end: (9, 11), text: `base`

so either the bug is in both places, or, more likely, I'm not understanding the query language well enough.

[sogaiu]

I know you provided a parse tree, but may be it's practical to share the content of cabal-file as well?

[magthe]

Unfortunately that particular one is from my day job and I can't share, but it's easily translatable to something that's open.

With the following in place:

1. The grammar from https://gitlab.com/magus/tree-sitter-cabal
2. The attached sandi.cabal.zip
3. The query (cabal (sections (flag_decl . _ @type name: (section_name)? @name)))

I get the following:

❯ tree-sitter query sandi-query sandi.cabal

sandi.cabal
  pattern: 0
    capture: 0 - type, start: (25, 0), end: (25, 4), text: `flag`
    capture: 1 - name, start: (25, 5), end: (25, 17), text: `with-conduit`
  pattern: 0
    capture: 0 - type, start: (25, 5), end: (25, 17), text: `with-conduit`

What I'd expect is

❯ tree-sitter query sandi-query sandi.cabal

sandi.cabal
  pattern: 0
    capture: 0 - type, start: (25, 0), end: (25, 4), text: `flag`
    capture: 1 - name, start: (25, 5), end: (25, 17), text: `with-conduit`

[sogaiu]

Thanks, I was able to reproduce the behavior you reported.

I noticed in the grammar that for flag_decl, there is an anonymous node flag:

        flag_decl: $ => seq(
            repeat($.comment),
            'flag',
            field('name', $.section_name),
            field('properties', $.property_block),
        ),

and the query in question uses both an anchor . and the type of wildcard node _ which matches both named and anonymous nodes:

(cabal (sections (flag_decl . _ @type name: (section_name)? @name)))

At the end of the Anchors section of the docs, I see this text:

The restrictions placed on a pattern by an anchor operator ignore anonymous nodes.

I wonder if this is involved in the unexpected behavior.

Sorry, the above may not really help with directly addressing your original point.

[magthe]

I think it does help me understand. AFAIU

• _ matches any node, anonymous or names
• (_) matches any named node

What I could use is something that would match only anonymous nodes, but there doesn't seem to be a way to express that, right?

So, is there some other trick I can use? What are my options, given that I really do need to match the sections based on their kind, i.e. "flag", "common", "library", etc.

[sogaiu]

What I could use is something that would match only anonymous nodes, but there doesn't seem to be a way to express that, right?

I think that's right if you mean an arbitrary anonymous node. I believe one can match specific anonymous nodes by mentioning them explicitly (e.g. "flag"):

To match specific anonymous nodes, you write their name between double quotes.

via: https://tree-sitter.github.io/tree-sitter/using-parsers#anonymous-nodes

Not sure if the following is helpful, but I tried the following tweak [1] to grammar.js:

        flag_decl: $ => seq(
            repeat($.comment),
            field('type', alias('flag', $.flag_type)),
            field('name', $.section_name),
            field('properties', $.property_block),
        ),

Below is an invocation and result for a query with the tweaked grammar:

$ tree-sitter.0.20.7 query sandi-query sandi.cabal 
sandi.cabal
  pattern: 0
    capture: 0 - type, start: (25, 0), end: (25, 4), text: `flag`
    capture: 1 - name, start: (25, 5), end: (25, 17), text: `with-conduit`

May be other folks have better ideas.

---

[1] Regarding alias, there is some documentation in this section.

---

On a side note, the following issue might be related: #1865

[magthe]

I knew about the matching by mentioning the string of the anonymous node explicitly, I would just really like to avoid doing that if possible.

Oh, I'd read about alias but didn't realise it could be used here. It's a bit of a neat hack, but I think it would have been nicer if field('type', 'flag) had worked. This is very usable though.

Thanks for the help!

[magthe]

I think answers to all my questions, as well as a very workable solution to my problem, can be found above. Thus I'm closing this discussion.