Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions wgsl/extract-grammar.py
Original file line number Diff line number Diff line change
Expand Up @@ -426,6 +426,7 @@ def not_token_only(value):
grammar_package.write('}\n')

# External scanner for nested block comments
# For the API, see https://tree-sitter.github.io/tree-sitter/creating-parsers#external-scanners
# See: https://github.com/tree-sitter/tree-sitter-rust/blob/master/src/scanner.c

os.makedirs(os.path.join(grammar_path, "src"), exist_ok=True)
Expand Down Expand Up @@ -461,6 +462,11 @@ def not_token_only(value):
for (;;) {
switch (lexer->lookahead) {
case '\0':
/* This signals the end of input. Since nesting depth is
* greater than zero, the scanner is in the middle of
* a block comment. Block comments must be affirmatively
* terminated.
*/
return false;
case '*':
advance(lexer);
Expand Down
16 changes: 13 additions & 3 deletions wgsl/index.bs
Original file line number Diff line number Diff line change
Expand Up @@ -430,15 +430,25 @@ A <dfn noexport>block comment</dfn> is a kind of [=comment=] consisting of:
* Then any sequence of:
* A [=block comment=], or
* Text that does not contain either `*/` or `/*`
* Then either:
* The two characters `*/`, or
* The end of the source text.
* Then the two characters `*/`

Note: Block comments can be nested.
Since a block comment requires matching start and end text sequences, and allows arbitrary nesting,
a block comment cannot be recognized with a regular expression.
This is a consequence of the Pumping Lemma for Regular Languages.

<div class='example wgsl' heading='Comments'>
<xmp>
let f = 1.5; // This is line-ending comment.
let g = 2.5; /* This is a block comment
that spans lines.
/* Block comments can nest.
*/
But all block comments must terminate.
*/
</xmp>
</div>

## Tokens ## {#tokens}

A <dfn>token</dfn> is a contiguous sequence of characters forming one of:
Expand Down