Skip to content

String literals are missing their content #150

Description

@wetneb

The grammar defines string literals like this:

    interpreted_string_literal: $ => seq(
      '"',
      repeat(choice(
        $._interpreted_string_literal_basic_content,
        $.escape_sequence,
      )),
      token.immediate('"'),
    ),
    _interpreted_string_literal_basic_content: _ => token.immediate(prec(1, /[^"\n\\]+/)),

Note that _interpreted_string_literal_basic_content is marked as private. This means that as a user, when parsing a string literal that only contains basic content (no escape sequence), I will see the interpreted_string_literal as having only two children: the two double quotes delimiting it. There is no child for the actual content.

This means that "foo" and "bar" have the exact same representation (two double quotes, without content between them).

I have the impression that other grammars tend to emit a public node for the string content. This seems more natural to me as a user. See for instance:
https://github.com/tree-sitter/tree-sitter-javascript/blob/b6f0624c1447bc209830b195999b78a56b10a579/grammar.js#L951-L977

https://github.com/tree-sitter/tree-sitter-python/blob/8c65e256f971812276ff2a69a2f515c218ed7f82/grammar.js#L1076-L1080

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions