Generate "incomplete" nodes in the presence of syntax errors?

[invrainbow]

When the standard Go parser encounters an incomplete selector or call expression, such as:

func main() {
  x := foo.       // incomplete selector expression
  y := foo.bar(   // incomplete call expression
}

it will still generate a selector/call expression node, with the incomplete bits marked as incomplete. This is useful for building IDE features like autocomplete and parameter hints. Whereas when Tree-sitter encounters these cases, it adds the individual tokens as nodes, but they aren't organized as selector/call expressions.

For example, with the standard Go parser, an easy way to implement autocomplete is to just walk the AST, searching for a selector expression node that contains the current cursor position. This isn't currently doable with Tree-sitter.

In general, is it possible to modify Tree-sitter to still generate a node with the correct type when the rule's first few tokens match, but the rest don't, and just mark the node as "incomplete"? I was reading that one of Tree-sitter's strengths is its ability to generate useful information in the face of syntax errors. This would be a great use case.

[sogaiu]

On a possibly related note, in the context of languages with lisp-like syntax, it would be useful to fill in missing delimiters to arrive at balanced expressions.

[invrainbow]

On a possibly related note, in the context of languages with lisp-like syntax, it would be useful to fill in missing delimiters to arrive at balanced expressions.

Saw you're the creator of tree-sitter-clojure. Does that mean this is a limitation of Tree-sitter, and can't be fixed by editing the grammar?

[sogaiu]

@invrainbow That is my impression but I am not sure.

[maxbrunsfeld]

Right now, there is no ability to configure Tree-sitter's error recovery. It just tries to minimize the size of the error nodes that it creates according to certain metrics.

I think it would be great to add some way of indicating, in a grammar, which rules should be "forcibly" created on error, even if a lot of their content is missing. This will take some design work to figure out how to model in the grammars.

[invrainbow]

Ok, in the meantime I've just abused the fact that an incomplete selector expr creates

tree
├─ operand
└─ error
   └─ anon_dot

and grabbed the operand by looking for an anon_dot > ts_node_parent() > ts_node_prev_sibling(). Something similar for call expressions.