Query compilation?

[ymarco]

Hi! My question is about the lines:

emacs/src/tree-sitter.c

Lines 1375 to 1380 in a4f90c5

	/* TODO: We could cache the query object, so that repeatedly
	querying with the same query can reuse the query object. It also
	saves us from expanding the sexp query into a string. I don't
	know how much time that could save though. */
	TSQuery *ts_query = ts_query_new (lang, source, strlen (source),
	&error_offset, &error_type);

Regarding error handling mostly.

In this branch queries are saved as strings and compiled in the internals on each use. In elisp-tree-sitter, you call tsc-make-query and use
the object it returns for calls to tsc-query-captures which is the analog for tree-sitter-query-capture.

What happens if your query is deformed, or simply has a typo in a node name? We call tree-sitter-query-capture on each keystroke in tree-sitter-font-lock-fontify-region. With the compilation occurring ahead-of-time it would fail once, but here wouldn't it barrage you with errors?
Especially with patterns that aren't set in stone and can be modified like font-lock keywords, I think compiling the query when the pattern is added is better than on each execution.

One nice thing though about compiling queries only when queried is that you can call ts_query_delete straight away. With users compiling queries it would need to be up to garbage collection, I think.

[casouri]

What happens if your query is deformed, or simply has a typo in a node name? We call tree-sitter-query-capture on each keystroke in tree-sitter-font-lock-fontify-region. With the compilation occurring ahead-of-time it would fail once, but here wouldn't it barrage you with errors?

Not quite barraging, jit-lock will just silently fail and leave a bunch of logs in Messages. I don't think error out when calling tree-sitter-query-capture is a grave problem, since 1) it doesn't barrage as you worried and 2) I don't expect queries in major modes to ship wrong code: it's not like a bug that could go undiscovered, if the query has a typo, the major mode writer will certainly find out when he/she tries to fontify a buffer.

I can see some advantages to compile the query ahead of time. 1) It would be helpful to know there is an error before calling tree-sitter-font-lock-fontify-region and see an unfontified buffer, not knowing what went wrong. I can add a function, say, tree-sitter-compile-query that checks a query (as in query pattern) and passes it on if its correct. 2) It could potentially saves recompilation of the query. But computing the query most probably takes negligible time.

On the other hand, compiling the query has downsides: I don't know what does tsc-make-query return, I assume an internal object? I try to minimize the number of new object types I introduce to Emacs, for hygiene. So far I've managed to add only parser object and node object. If there aren't good reasons I'm inclined to not add a query object. So far the advantages that I see aren't very convincing.

If you want to continue the discussion, I suggest we continue at emacs-devel, that way others who are more knowledgable than I can join and offer their opinion.

[ymarco]

Alright, let's move to emacs-devel.

[ymarco]

(I sent an email in the thread. Assuming I didn't screw up the email headers I think my message is waiting to be approved by a moderator—it's my first message on emacs-devel.)