mirrored from git://git.sv.gnu.org/emacs.git
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
/
parsing.texi
1992 lines (1649 loc) · 69.9 KB
/
parsing.texi
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
@c -*- mode: texinfo; coding: utf-8 -*-
@c This is part of the GNU Emacs Lisp Reference Manual.
@c Copyright (C) 2021--2023 Free Software Foundation, Inc.
@c See the file elisp.texi for copying conditions.
@node Parsing Program Source
@chapter Parsing Program Source
@cindex syntax tree, from parsing program source
Emacs provides various ways to parse program source text and produce a
@dfn{syntax tree}. In a syntax tree, text is no longer considered a
one-dimensional stream of characters, but a structured tree of nodes,
where each node representing a piece of text. Thus, a syntax tree can
enable interesting features like precise fontification, indentation,
navigation, structured editing, etc.
Emacs has a simple facility for parsing balanced expressions
(@pxref{Parsing Expressions}). There is also the SMIE library for
generic navigation and indentation (@pxref{SMIE}).
In addition to those, Emacs also provides integration with
@uref{https://tree-sitter.github.io/tree-sitter, the tree-sitter
library}) if support for it was compiled in. The tree-sitter library
implements an incremental parser and has support from a wide range of
programming languages.
@defun treesit-available-p
This function returns non-@code{nil} if tree-sitter features are
available for the current Emacs session.
@end defun
To be able to parse the program source using the tree-sitter library
and access the syntax tree of the program, a Lisp program needs to
load a language grammar library, and create a parser for that
language and the current buffer. After that, the Lisp program can
query the parser about specific nodes of the syntax tree. Then, it
can access various kinds of information about each node, and search
for nodes using a powerful pattern-matching syntax. This chapter
explains how to do all this, and also how a Lisp program can work with
source files that mix multiple programming languages.
@menu
* Language Grammar:: Loading tree-sitter language grammar.
* Using Parser:: Introduction to parsers.
* Retrieving Nodes:: Retrieving nodes from a syntax tree.
* Accessing Node Information:: Accessing node information.
* Pattern Matching:: Pattern matching with query patterns.
* Multiple Languages:: Parse text written in multiple languages.
* Tree-sitter Major Modes:: Develop major modes using tree-sitter.
* Tree-sitter C API:: Compare the C API and the ELisp API.
@end menu
@node Language Grammar
@section Tree-sitter Language Grammar
@cindex language grammar, for tree-sitter
@heading Loading a language grammar
@cindex loading language grammar for tree-sitter
@cindex language argument, for tree-sitter
Tree-sitter relies on language grammar to parse text in that
language. In Emacs, a language grammar is represented by a symbol.
For example, the C language grammar is represented as the symbol
@code{c}, and @code{c} can be passed to tree-sitter functions as the
@var{language} argument.
@vindex treesit-extra-load-path
@vindex treesit-load-language-error
Tree-sitter language grammar are distributed as dynamic libraries.
In order to use a language grammar in Emacs, you need to make sure
that the dynamic library is installed on the system. Emacs looks for
language grammar in several places, in the following order:
@itemize @bullet
@item
first, in the list of directories specified by the variable
@code{treesit-extra-load-path};
@item
then, in the @file{tree-sitter} subdirectory of the directory
specified by @code{user-emacs-directory} (@pxref{Init File});
@item
and finally, in the system's default locations for dynamic libraries.
@end itemize
In each of these directories, Emacs looks for a file with file-name
extensions specified by the variable @code{dynamic-library-suffixes}.
If Emacs cannot find the library or has problems loading it, Emacs
signals the @code{treesit-load-language-error} error. The data of
that signal could be one of the following:
@table @code
@item (not-found @var{error-msg} @dots{})
This means that Emacs could not find the language grammar library.
@item (symbol-error @var{error-msg})
This means that Emacs could not find in the library the expected function
that every language grammar library should export.
@item (version-mismatch @var{error-msg})
This means that the version of language grammar library is incompatible
with that of the tree-sitter library.
@end table
@noindent
In all of these cases, @var{error-msg} might provide additional
details about the failure.
@defun treesit-language-available-p language &optional detail
This function returns non-@code{nil} if the language grammar for
@var{language} exist and can be loaded.
If @var{detail} is non-@code{nil}, return @code{(t . nil)} when
@var{language} is available, and @code{(nil . @var{data})} when it's
unavailable. @var{data} is the signal data of
@code{treesit-load-language-error}.
@end defun
@vindex treesit-load-name-override-list
By convention, the file name of the dynamic library for @var{language} is
@file{libtree-sitter-@var{language}.@var{ext}}, where @var{ext} is the
system-specific extension for dynamic libraries. Also by convention,
the function provided by that library is named
@code{tree_sitter_@var{language}}. If a language grammar library
doesn't follow this convention, you should add an entry
@example
(@var{language} @var{library-base-name} @var{function-name})
@end example
to the list in the variable @code{treesit-load-name-override-list}, where
@var{library-base-name} is the basename of the dynamic library's file name,
(usually, @file{libtree-sitter-@var{language}}), and
@var{function-name} is the function provided by the library
(usually, @code{tree_sitter_@var{language}}). For example,
@example
(cool-lang "libtree-sitter-coool" "tree_sitter_cooool")
@end example
@noindent
for a language that considers itself too ``cool'' to abide by
conventions.
@cindex language grammar version, compatibility
@defun treesit-library-abi-version &optional min-compatible
This function returns the version of the language grammar
Application Binary Interface (@acronym{ABI}) supported by the
tree-sitter library. By default, it returns the latest ABI version
supported by the library, but if @var{min-compatible} is
non-@code{nil}, it returns the oldest ABI version which the library
still can support. language grammar libraries must be built for
ABI versions between the oldest and the latest versions supported by
the tree-sitter library, otherwise the library will be unable to load
them.
@end defun
@defun treesit-language-abi-version language
This function returns the @acronym{ABI} version of the language
grammar library loaded by Emacs for @var{language}. If @var{language}
is unavailable, this function returns @code{nil}.
@end defun
@heading Concrete syntax tree
@cindex syntax tree, concrete
A syntax tree is what a parser generates. In a syntax tree, each node
represents a piece of text, and is connected to each other by a
parent-child relationship. For example, if the source text is
@example
1 + 2
@end example
@noindent
its syntax tree could be
@example
@group
+--------------+
| root "1 + 2" |
+--------------+
|
+--------------------------------+
| expression "1 + 2" |
+--------------------------------+
| | |
+------------+ +--------------+ +------------+
| number "1" | | operator "+" | | number "2" |
+------------+ +--------------+ +------------+
@end group
@end example
We can also represent it as an s-expression:
@example
(root (expression (number) (operator) (number)))
@end example
@subheading Node types
@cindex node types, in a syntax tree
@cindex type of node, tree-sitter
@anchor{tree-sitter node type}
@cindex named node, tree-sitter
@anchor{tree-sitter named node}
@cindex anonymous node, tree-sitter
Names like @code{root}, @code{expression}, @code{number}, and
@code{operator} specify the @dfn{type} of the nodes. However, not all
nodes in a syntax tree have a type. Nodes that don't have a type are
known as @dfn{anonymous nodes}, and nodes with a type are @dfn{named
nodes}. Anonymous nodes are tokens with fixed spellings, including
punctuation characters like bracket @samp{]}, and keywords like
@code{return}.
@subheading Field names
@cindex field name, tree-sitter
@cindex tree-sitter node field name
@anchor{tree-sitter node field name}
To make the syntax tree easier to analyze, many language grammar
assign @dfn{field names} to child nodes. For example, a
@code{function_definition} node could have a @code{declarator} and a
@code{body}:
@example
@group
(function_definition
declarator: (declaration)
body: (compound_statement))
@end group
@end example
@heading Exploring the syntax tree
@cindex explore tree-sitter syntax tree
@cindex inspection of tree-sitter parse tree nodes
To aid in understanding the syntax of a language and in debugging of
Lisp program that use the syntax tree, Emacs provides an ``explore''
mode, which displays the syntax tree of the source in the current
buffer in real time. Emacs also comes with an ``inspect mode'', which
displays information of the nodes at point in the mode-line.
@deffn Command treesit-explore-mode
This mode pops up a window displaying the syntax tree of the source in
the current buffer. Selecting text in the source buffer highlights
the corresponding nodes in the syntax tree display. Clicking
on nodes in the syntax tree highlights the corresponding text in the
source buffer.
@end deffn
@deffn Command treesit-inspect-mode
This minor mode displays on the mode-line the node that @emph{starts}
at point. For example, the mode-line can display
@example
@var{parent} @var{field}: (@var{node} (@var{child} (@dots{})))
@end example
@noindent
where @var{node}, @var{child}, etc., are nodes which begin at point.
@var{parent} is the parent of @var{node}. @var{node} is displayed in
a bold typeface. @var{field-name}s are field names of @var{node} and
of @var{child}, etc.
If no node starts at point, i.e., point is in the middle of a node,
then the mode line displays the earliest node that spans point, and
its immediate parent.
This minor mode doesn't create parsers on its own. It uses the first
parser in @code{(treesit-parser-list)} (@pxref{Using Parser}).
@end deffn
@heading Reading the grammar definition
@cindex reading grammar definition, tree-sitter
Authors of language grammar define the @dfn{grammar} of a
programming language, which determines how a parser constructs a
concrete syntax tree out of the program text. In order to use the
syntax tree effectively, you need to consult the @dfn{grammar file}.
The grammar file is usually @file{grammar.js} in a language
grammar's project repository. The link to a language grammar's
home page can be found on
@uref{https://tree-sitter.github.io/tree-sitter, tree-sitter's
homepage}.
The grammar definition is written in JavaScript. For example, the
rule matching a @code{function_definition} node looks like
@example
@group
function_definition: $ => seq(
$.declaration_specifiers,
field('declarator', $.declaration),
field('body', $.compound_statement)
)
@end group
@end example
@noindent
The rules are represented by functions that take a single argument
@var{$}, representing the whole grammar. The function itself is
constructed by other functions: the @code{seq} function puts together
a sequence of children; the @code{field} function annotates a child
with a field name. If we write the above definition in the so-called
@dfn{Backus-Naur Form} (@acronym{BNF}) syntax, it would look like
@example
@group
function_definition :=
<declaration_specifiers> <declaration> <compound_statement>
@end group
@end example
@noindent
and the node returned by the parser would look like
@example
@group
(function_definition
(declaration_specifier)
declarator: (declaration)
body: (compound_statement))
@end group
@end example
Below is a list of functions that one can see in a grammar definition.
Each function takes other rules as arguments and returns a new rule.
@table @code
@item seq(@var{rule1}, @var{rule2}, @dots{})
matches each rule one after another.
@item choice(@var{rule1}, @var{rule2}, @dots{})
matches one of the rules in its arguments.
@item repeat(@var{rule})
matches @var{rule} for @emph{zero or more} times.
This is like the @samp{*} operator in regular expressions.
@item repeat1(@var{rule})
matches @var{rule} for @emph{one or more} times.
This is like the @samp{+} operator in regular expressions.
@item optional(@var{rule})
matches @var{rule} for @emph{zero or one} time.
This is like the @samp{?} operator in regular expressions.
@item field(@var{name}, @var{rule})
assigns field name @var{name} to the child node matched by @var{rule}.
@item alias(@var{rule}, @var{alias})
makes nodes matched by @var{rule} appear as @var{alias} in the syntax
tree generated by the parser. For example,
@example
alias(preprocessor_call_exp, call_expression)
@end example
@noindent
makes any node matched by @code{preprocessor_call_exp} appear as
@code{call_expression}.
@end table
Below are grammar functions of lesser importance for reading a
language grammar.
@table @code
@item token(@var{rule})
marks @var{rule} to produce a single leaf node. That is, instead of
generating a parent node with individual child nodes under it,
everything is combined into a single leaf node. @xref{Retrieving
Nodes}.
@item token.immediate(@var{rule})
Normally, grammar rules ignore preceding whitespace; this
changes @var{rule} to match only when there is no preceding
whitespaces.
@item prec(@var{n}, @var{rule})
gives @var{rule} the level-@var{n} precedence.
@item prec.left([@var{n},] @var{rule})
marks @var{rule} as left-associative, optionally with level @var{n}.
@item prec.right([@var{n},] @var{rule})
marks @var{rule} as right-associative, optionally with level @var{n}.
@item prec.dynamic(@var{n}, @var{rule})
this is like @code{prec}, but the precedence is applied at runtime
instead.
@end table
The documentation of the tree-sitter project has
@uref{https://tree-sitter.github.io/tree-sitter/creating-parsers, more
about writing a grammar}. Read especially ``The Grammar DSL''
section.
@node Using Parser
@section Using Tree-sitter Parser
@cindex tree-sitter parser, using
This section describes how to create and configure a tree-sitter
parser. In Emacs, each tree-sitter parser is associated with a
buffer. As the user edits the buffer, the associated parser and
syntax tree are automatically kept up-to-date.
@defvar treesit-max-buffer-size
This variable contains the maximum size of buffers in which
tree-sitter can be activated. Major modes should check this value
when deciding whether to enable tree-sitter features.
@end defvar
@cindex creating tree-sitter parsers
@cindex tree-sitter parser, creating
@defun treesit-parser-create language &optional buffer no-reuse
Create a parser for the specified @var{buffer} and @var{language}
(@pxref{Language Grammar}). If @var{buffer} is omitted or
@code{nil}, it stands for the current buffer.
By default, this function reuses a parser if one already exists for
@var{language} in @var{buffer}, but if @var{no-reuse} is
non-@code{nil}, this function always creates a new parser.
If that buffer is an indirect buffer, its base buffer is used instead.
That is, indirect buffers use their base buffer's parsers. If the
base buffer is narrowed, an indirect buffer might not be able to
retrieve information of the portion of the buffer text that are
invisible in the base buffer. Lisp programs should widen as necessary
should they want to use a parser in an indirect buffer.
@end defun
Given a parser, we can query information about it.
@defun treesit-parser-buffer parser
This function returns the buffer associated with @var{parser}.
@end defun
@defun treesit-parser-language parser
This function returns the language used by @var{parser}.
@end defun
@defun treesit-parser-p object
This function checks if @var{object} is a tree-sitter parser, and
returns non-@code{nil} if it is, and @code{nil} otherwise.
@end defun
There is no need to explicitly parse a buffer, because parsing is done
automatically and lazily. A parser only parses when a Lisp program
queries for a node in its syntax tree. Therefore, when a parser is
first created, it doesn't parse the buffer; it waits until the Lisp
program queries for a node for the first time. Similarly, when some
change is made in the buffer, a parser doesn't re-parse immediately.
@vindex treesit-buffer-too-large
When a parser does parse, it checks for the size of the buffer.
Tree-sitter can only handle buffer no larger than about 4GB. If the
size exceeds that, Emacs signals the @code{treesit-buffer-too-large}
error with signal data being the buffer size.
Once a parser is created, Emacs automatically adds it to the
internal parser list. Every time a change is made to the buffer,
Emacs updates parsers in this list so they can update their syntax
tree incrementally.
@defun treesit-parser-list &optional buffer
This function returns the parser list of @var{buffer}. If
@var{buffer} is @code{nil} or omitted, it defaults to the current
buffer. If that buffer is an indirect buffer, its base buffer is used
instead. That is, indirect buffers use their base buffer's parsers.
@end defun
@defun treesit-parser-delete parser
This function deletes @var{parser}.
@end defun
@cindex tree-sitter narrowing
@anchor{tree-sitter narrowing}
Normally, a parser ``sees'' the whole buffer, but when the buffer is
narrowed (@pxref{Narrowing}), the parser will only see the accessible
portion of the buffer. As far as the parser can tell, the hidden
region was deleted. When the buffer is later widened, the parser
thinks text is inserted at the beginning and at the end. Although
parsers respect narrowing, modes should not use narrowing as a means
to handle a multi-language buffer; instead, set the ranges in which the
parser should operate. @xref{Multiple Languages}.
Because a parser parses lazily, when the user or a Lisp program
narrows the buffer, the parser is not affected immediately; as long as
the mode doesn't query for a node while the buffer is narrowed, the
parser is oblivious of the narrowing.
@cindex tree-sitter parse string
@cindex parse string, tree-sitter
Besides creating a parser for a buffer, a Lisp program can also parse a
string. Unlike a buffer, parsing a string is a one-off operation, and
there is no way to update the result.
@defun treesit-parse-string string language
This function parses @var{string} using @var{language}, and returns
the root node of the generated syntax tree.
@end defun
@heading Be notified by changes to the parse tree
@cindex update callback, for tree-sitter parse-tree
@cindex after-change notifier, for tree-sitter parse-tree
@cindex tree-sitter parse-tree, update and after-change callback
@cindex notifiers, tree-sitter
A Lisp program might want to be notified of text affected by
incremental parsing. For example, inserting a comment-closing token
converts text before that token into a comment. Even
though the text is not directly edited, it is deemed to be ``changed''
nevertheless.
Emacs lets a Lisp program to register callback functions
(a.k.a.@: @dfn{notifiers}) for this kind of changes. A notifier
function takes two arguments: @var{ranges} and @var{parser}.
@var{ranges} is a list of cons cells of the form @w{@code{(@var{start}
. @var{end})}}, where @var{start} and @var{end} mark the start and the
end positions of a range. @var{parser} is the parser issuing the
notification.
Every time a parser reparses a buffer, it compares the old and new
parse-tree, computes the ranges in which nodes have changed, and
passes the ranges to notifier functions. Note that the initial parse
is also considered a ``change'', so notifier functions are called on
the initial parse, with range being the whole buffer.
@defun treesit-parser-add-notifier parser function
This function adds @var{function} to @var{parser}'s list of
after-change notifier functions. @var{function} must be a function
symbol, not a lambda function (@pxref{Anonymous Functions}).
@end defun
@defun treesit-parser-remove-notifier parser function
This function removes @var{function} from the list of @var{parser}'s
after-change notifier functions. @var{function} must be a function
symbol, rather than a lambda function.
@end defun
@defun treesit-parser-notifiers parser
This function returns the list of @var{parser}'s notifier functions.
@end defun
@node Retrieving Nodes
@section Retrieving Nodes
@cindex retrieve node, tree-sitter
@cindex tree-sitter, find node
@cindex get node, tree-sitter
@cindex terminology, for tree-sitter functions
Here's some terminology and conventions we use when documenting
tree-sitter functions.
A node in a syntax tree spans some portion of the program text in the
buffer. We say that a node is ``smaller'' or ``larger'' than another
if it spans, respectively, a smaller or larger portion of buffer text
than the other node. Since nodes that are deeper (``lower'') in the
tree are children of the nodes that are ``higher'' in the tree, it
follows that a lower node will always be smaller than a node that is
higher in the node hierarchy. A node that is higher up in the syntax
tree contains one or more smaller nodes as its children, and therefore
spans a larger portion of buffer text.
When a function cannot find a node, it returns @code{nil}. For
convenience, all functions that take a node as argument and return
a node, also accept the node argument of @code{nil} and in that case
just return @code{nil}.
@vindex treesit-node-outdated
Nodes are not automatically updated when the associated buffer is
modified, and there is no way to update a node once it is retrieved.
Using an outdated node signals the @code{treesit-node-outdated} error.
@heading Retrieving nodes from syntax tree
@cindex retrieving tree-sitter nodes
@cindex syntax tree, retrieving nodes
@cindex leaf node, of tree-sitter parse tree
@cindex tree-sitter parse tree, leaf node
@defun treesit-node-at pos &optional parser-or-lang named
This function returns a @dfn{leaf} node at buffer position @var{pos}.
A leaf node is a node that doesn't have any child nodes.
This function tries to return a node whose span covers @var{pos}: the
node's beginning position is less or equal to @var{pos}, and the
node's end position is greater or equal to @var{pos}.
If no leaf node's span covers @var{pos} (e.g., @var{pos} is in the
whitespace between two leaf nodes), this function returns the first
leaf node after @var{pos}.
Finally, if there is no leaf node after @var{pos}, return the first
leaf node before @var{pos}.
If @var{parser-or-lang} is a parser object, this function uses that
parser; if @var{parser-or-lang} is a language, this function uses the
first parser for that language in the current buffer, or creates one
if none exists; if @var{parser-or-lang} is @code{nil}, this function
tries to guess the language at @var{pos} by calling
@code{treesit-language-at} (@pxref{Multiple Languages}).
If this function cannot find a suitable node to return, it returns
@code{nil}.
If @var{named} is non-@code{nil}, this function looks only for named
nodes (@pxref{tree-sitter named node, named node}).
Example:
@example
@group
;; Find the node at point in a C parser's syntax tree.
(treesit-node-at (point) 'c)
@result{} #<treesit-node (primitive_type) in 23-27>
@end group
@end example
@end defun
@defun treesit-node-on beg end &optional parser-or-lang named
This function returns the @emph{smallest} node that covers the region
of buffer text between @var{beg} and @var{end}. In other words, the
start of the node is before or at @var{beg}, and the end of the node
is at or after @var{end}.
@emph{Beware:} calling this function on an empty line that is not
inside any top-level construct (function definition, etc.) most
probably will give you the root node, because the root node is the
smallest node that covers that empty line. Most of the time, you want
to use @code{treesit-node-at} instead.
If @var{parser-or-lang} is a parser object, this function uses that
parser; if @var{parser-or-lang} is a language, this function uses the
first parser for that language in the current buffer, or creates one
if none exists; if @var{parser-or-lang} is @code{nil}, this function
tries to guess the language at @var{beg} by calling
@code{treesit-language-at}.
If @var{named} is non-@code{nil}, this function looks for a named node
only (@pxref{tree-sitter named node, named node}).
@end defun
@defun treesit-parser-root-node parser
This function returns the root node of the syntax tree generated by
@var{parser}.
@end defun
@defun treesit-buffer-root-node &optional language
This function finds the first parser for @var{language} in the current
buffer, or creates one if none exists, and returns the root node
generated by that parser. If @var{language} is omitted, it uses the
first parser in the parser list. If it cannot find an appropriate
parser, it returns @code{nil}.
@end defun
Given a node, a Lisp program can retrieve other nodes starting from
it, or query for information about this node.
@heading Retrieving nodes from other nodes
@cindex syntax tree nodes, retrieving from other nodes
@subheading By kinship
@cindex kinship, syntax tree nodes
@cindex nodes, by kinship
@cindex syntax tree nodes, by kinship
@defun treesit-node-parent node
This function returns the immediate parent of @var{node}.
If @var{node} is more than 1000 levels deep in a parse tree, the
return value is undefined. Currently it returns @var{nil}, but that
could change in the future.
@end defun
@defun treesit-node-child node n &optional named
This function returns the @var{n}'th child of @var{node}. If
@var{named} is non-@code{nil}, it counts only named nodes
(@pxref{tree-sitter named node, named node}).
For example, in a node that represents a string @code{"text"}, there
are three children nodes: the opening quote @code{"}, the string text
@code{text}, and the closing quote @code{"}. Among these nodes, the
first child is the opening quote @code{"}, and the first named child
is the string text.
This function returns @code{nil} if there is no @var{n}'th child.
@var{n} could be negative, e.g., @code{-1} represents the last child.
@end defun
@defun treesit-node-children node &optional named
This function returns all of @var{node}'s children as a list. If
@var{named} is non-@code{nil}, it retrieves only named nodes.
@end defun
@defun treesit-node-next-sibling node &optional named
This function finds the next sibling of @var{node}. If @var{named} is
non-@code{nil}, it finds the next named sibling.
@end defun
@defun treesit-node-prev-sibling node &optional named
This function finds the previous sibling of @var{node}. If
@var{named} is non-@code{nil}, it finds the previous named sibling.
@end defun
@subheading By field name
@cindex nodes, by field name
@cindex syntax tree nodes, by field name
To make the syntax tree easier to analyze, many language grammar
assign @dfn{field names} to child nodes (@pxref{tree-sitter node field
name, field name}). For example, a @code{function_definition} node
could have a @code{declarator} node and a @code{body} node.
@defun treesit-node-child-by-field-name node field-name
This function finds the child of @var{node} whose field name is
@var{field-name}, a string.
@example
@group
;; Get the child that has "body" as its field name.
(treesit-node-child-by-field-name node "body")
@result{} #<treesit-node (compound_statement) in 45-89>
@end group
@end example
@end defun
@subheading By position
@cindex nodes, by position
@cindex syntax tree nodes, by position
@defun treesit-node-first-child-for-pos node pos &optional named
This function finds the first child of @var{node} that extends beyond
buffer position @var{pos}. ``Extends beyond'' means the end of the
child node is greater or equal to @var{pos}. This function only looks
for immediate children of @var{node}, and doesn't look in its
grandchildren. If @var{named} is non-@code{nil}, it looks for the
first named child (@pxref{tree-sitter named node, named node}).
@end defun
@defun treesit-node-descendant-for-range node beg end &optional named
This function finds the @emph{smallest} descendant node of @var{node}
that spans the region of text between positions @var{beg} and
@var{end}. It is similar to @code{treesit-node-at}. If @var{named}
is non-@code{nil}, it looks for smallest named child.
@end defun
@heading Searching for node
@defun treesit-search-subtree node predicate &optional backward all depth
This function traverses the subtree of @var{node} (including
@var{node} itself), looking for a node for which @var{predicate}
returns non-@code{nil}. @var{predicate} is a regexp that is matched
against each node's type, or a predicate function that takes a node
and returns non-@code{nil} if the node matches. The function returns
the first node that matches, or @code{nil} if none does.
By default, this function only traverses named nodes, but if @var{all}
is non-@code{nil}, it traverses all the nodes. If @var{backward} is
non-@code{nil}, it traverses backwards (i.e., it visits the last child
first when traversing down the tree). If @var{depth} is
non-@code{nil}, it must be a number that limits the tree traversal to
that many levels down the tree. If @var{depth} is @code{nil}, it
defaults to 1000.
@end defun
@defun treesit-search-forward start predicate &optional backward all
Like @code{treesit-search-subtree}, this function also traverses the
parse tree and matches each node with @var{predicate} (except for
@var{start}), where @var{predicate} can be a regexp or a function.
For a tree like the below where @var{start} is marked S, this function
traverses as numbered from 1 to 12:
@example
@group
12
|
S--------3----------11
| | |
o--o-+--o 1--+--2 6--+-----10
| | | |
o o +-+-+ +--+--+
| | | | |
4 5 7 8 9
@end group
@end example
Note that this function doesn't traverse the subtree of @var{start},
and it always traverse leaf nodes first, then upwards.
Like @code{treesit-search-subtree}, this function only searches for
named nodes by default, but if @var{all} is non-@code{nil}, it
searches for all nodes. If @var{backward} is non-@code{nil}, it
searches backwards.
While @code{treesit-search-subtree} traverses the subtree of a node,
this function starts with node @var{start} and traverses every node
that comes after it in the buffer position order, i.e., nodes with
start positions greater than the end position of @var{start}.
In the tree shown above, @code{treesit-search-subtree} traverses node
S (@var{start}) and nodes marked with @code{o}, where this function
traverses the nodes marked with numbers. This function is useful for
answering questions like ``what is the first node after @var{start} in
the buffer that satisfies some condition?''
@end defun
@defun treesit-search-forward-goto node predicate &optional start backward all
This function moves point to the start or end of the next node after
@var{node} in the buffer that matches @var{predicate}. If @var{start}
is non-@code{nil}, stop at the beginning rather than the end of a node.
This function guarantees that the matched node it returns makes
progress in terms of buffer position: the start/end position of the
returned node is always greater than that of @var{node}.
Arguments @var{predicate}, @var{backward} and @var{all} are the same
as in @code{treesit-search-forward}.
@end defun
@defun treesit-induce-sparse-tree root predicate &optional process-fn depth
This function creates a sparse tree from @var{root}'s subtree.
It takes the subtree under @var{root}, and combs it so only the nodes
that match @var{predicate} are left. Like previous functions, the
@var{predicate} can be a regexp string that matches against each
node's type, or a function that takes a node and return non-@code{nil}
if it matches.
For example, for a subtree on the left that consist of both numbers
and letters, if @var{predicate} is ``letter only'', the returned tree
is the one on the right.
@example
@group
a a a
| | |
+---+---+ +---+---+ +---+---+
| | | | | | | | |
b 1 2 b | | b c d
| | => | | => |
c +--+ c + e
| | | | |
+--+ d 4 +--+ d
| | |
e 5 e
@end group
@end example
If @var{process-fn} is non-@code{nil}, instead of returning the
matched nodes, this function passes each node to @var{process-fn} and
uses the returned value instead. If non-@code{nil}, @var{depth} is
the number of levels to go down from @var{root}. If @var{depth} is
@code{nil}, it defaults to 1000.
Each node in the returned tree looks like
@w{@code{(@var{tree-sitter-node} . (@var{child} @dots{}))}}. The
@var{tree-sitter-node} of the root of this tree will be nil if
@var{root} doesn't match @var{predicate}. If no node matches
@var{predicate}, the function returns @code{nil}.
@end defun
@heading More convenience functions
@defun treesit-filter-child node predicate &optional named
This function finds immediate children of @var{node} that satisfy
@var{predicate}.
The @var{predicate} function takes a node as the argument and should
return non-@code{nil} to indicate that the node should be kept. If
@var{named} is non-@code{nil}, this function only examines the named
nodes.
@end defun
@defun treesit-parent-until node predicate &optional include-node
This function repeatedly finds the parents of @var{node}, and returns
the parent that satisfies @var{pred}, a function that takes a node as
the argument and returns a boolean that indicates a match. If no
parent satisfies @var{pred}, this function returns @code{nil}.
Normally this function only looks at the parents of @var{node} but not
@var{node} itself. But if @var{include-node} is non-@var{nil}, this
function returns @var{node} if @var{node} satisfies @var{pred}.
@end defun
@defun treesit-parent-while node pred
This function goes up the tree starting from @var{node}, and keeps
doing so as long as the nodes satisfy @var{pred}, a function that
takes a node as the argument. That is, this function returns the
highest parent of @var{node} that still satisfies @var{pred}. Note
that if @var{node} satisfies @var{pred} but its immediate parent
doesn't, @var{node} itself is returned.
@end defun
@defun treesit-node-top-level node &optional type
This function returns the highest parent of @var{node} that has the
same type as @var{node}. If no such parent exists, it returns
@code{nil}. Therefore this function is also useful for testing
whether @var{node} is top-level.
If @var{type} is non-@code{nil}, this function matches each parent's
type with @var{type} as a regexp, rather than using @var{node}'s type.
@end defun
@node Accessing Node Information
@section Accessing Node Information
@cindex information of node, syntax trees
@cindex syntax trees, node information
@heading Basic information of Node
Every node is associated with a parser, and that parser is associated
with a buffer. The following functions retrieve them.
@defun treesit-node-parser node
This function returns @var{node}'s associated parser.
@end defun
@defun treesit-node-buffer node
This function returns @var{node}'s parser's associated buffer.
@end defun
@defun treesit-node-language node
This function returns @var{node}'s parser's associated language.
@end defun
Each node represents a portion of text in the buffer. Functions below
find relevant information about that text.
@defun treesit-node-start node
Return the start position of @var{node}.
@end defun
@defun treesit-node-end node
Return the end position of @var{node}.
@end defun
@defun treesit-node-text node &optional object
Return the buffer text that @var{node} represents, as a string. (If
@var{node} is retrieved from parsing a string, it will be the text
from that string.)
@end defun
@cindex predicates for syntax tree nodes
Here are some predicates on tree-sitter nodes:
@defun treesit-node-p object
Checks if @var{object} is a tree-sitter syntax node.
@end defun
@cindex compare tree-sitter syntax nodes
@cindex tree-sitter nodes, comparing
@defun treesit-node-eq node1 node2
Checks if @var{node1} and @var{node2} refer to the same node in a
tree-sitter syntax tree. This function uses the same equivalence
metric as @code{equal}. You can also compare nodes using @code{equal}
(@pxref{Equality Predicates}).
@end defun
@heading Property information
In general, nodes in a concrete syntax tree fall into two categories:
@dfn{named nodes} and @dfn{anonymous nodes}. Whether a node is named
or anonymous is determined by the language grammar
(@pxref{tree-sitter named node, named node}).
@cindex tree-sitter missing node
@cindex missing node, tree-sitter
Apart from being named or anonymous, a node can have other properties.
A node can be ``missing'': such nodes are inserted by the parser in
order to recover from certain kinds of syntax errors, i.e., something
should probably be there according to the grammar, but is not there.
This can happen during editing of the program source, when the source
is not yet in its final form.
@cindex tree-sitter extra node
@cindex extra node, tree-sitter
A node can be ``extra'': such nodes represent things like comments,
which can appear anywhere in the text.
@cindex tree-sitter outdated node
@cindex outdated node, tree-sitter
A node can be ``outdated'', if its parser has reparsed at least once
after the node was created.
@cindex tree-sitter node that has error
@cindex has error, tree-sitter node
A node ``has error'' if the text it spans contains a syntax error. It
can be that the node itself has an error, or one of its descendants
has an error.
@cindex tree-sitter, live parsing node
@cindex live node, tree-sitter
A node is considered @dfn{live} if its parser is not deleted, and the
buffer to which it belongs to is a live buffer (@pxref{Killing Buffers}).
@defun treesit-node-check node property
This function returns non-@code{nil} if @var{node} has the specified
@var{property}. @var{property} can be @code{named}, @code{missing},
@code{extra}, @code{outdated}, @code{has-error}, or @code{live}.
@end defun
@defun treesit-node-type node
Named nodes have ``types'' (@pxref{tree-sitter node type, node type}).
For example, a named node can be a @code{string_literal} node, where
@code{string_literal} is its type. The type of an anonymous node is
just the text that the node represents; e.g., the type of a @samp{,}
node is just @samp{,}.
This function returns @var{node}'s type as a string.
@end defun
@heading Information as a child or parent