allow identifiers in string concatentations

[clason]

Resolves errors in strings of the form:

("failed in line %" PRIdLINENR)

[clason]

cherry-picked from neovim#2 @lewis6991 @amaanq

[amaanq]

@ahlinc would like your thoughts too

[ahlinc]

@ahlinc would like your thoughts too

I'd also alias the identifier node to the same node that represent a string content between double quotes but unfortunately there is no one.

It make sense to add a bellow additional fix on top of this PR:

diff --git i/grammar.js w/grammar.js
index 94e7f3d..b51cacd 100644
--- i/grammar.js
+++ w/grammar.js
@@ -1055,13 +1055,16 @@ module.exports = grammar({

     concatenated_string: $ => seq(
       $.string_literal,
-      repeat1(choice($.string_literal, $.identifier)),
+      repeat1(choice(
+        $.string_literal,
+        alias($.identifier, $.string),
+      )),
     ),

     string_literal: $ => seq(
       choice('L"', 'u"', 'U"', 'u8"', '"'),
       repeat(choice(
-        token.immediate(prec(1, /[^\\"\n]+/)),
+        alias(token.immediate(prec(1, /[^\\"\n]+/)), $.string),
         $.escape_sequence,
       )),
       '"',

[amaanq]

Maybe string_content is better than string, but I don't think identifiers should be aliased to string in concatenations

[amaanq]

Also gotta get rid of the appveyor thing after this

@clason tests need updates

[clason]

Used tree-sitter test -u; hope that's OK!

[amaanq]

Mentioned this earlier here, but the formatting of the header/test delimiters needs to not happen, it pollutes the diff (not your problem, it looks fine to me 😅)

[clason]

Would it be acceptable for me to do a separate commit before the change to reformat the corpus? I think the convenience of using tree-sitter test -u is worth a single reformat churn.

[amaanq]

hmm not sure what you mean, like a commit of additional tests?

[clason]

like a commit just calling tree-sitter test -u on the old parser to reformat everything (without additional changes), so that the next commit with the changes can use tree-sitter test -u to only update the now failing tests, without polluting the diff.

[amaanq]

Oooh, yeah I'm way too tired my bad for not getting that - that would be nicer for the future

[ahlinc]

but I don't think identifiers should be aliased to string in concatenations

@amaanq in concatenations identifiers aren't identifiers because they are concatenated literally and not by value.

[amaanq]

well it's like an identifier that holds a string, so it shouldn't be aliased as such so users can distinguish them imo

[ahlinc]

well it's like an identifier that holds a string, so it shouldn't be aliased as such so users can distinguish them imo

sorry, my bad, bash in my mind.

For C the PRIdLINENR would be an identifier that would be substituted by a macro pre proccessor to an another string.

[ahlinc]

Hmm, for the clarity may be it would still make sense to alias identifier to the preproc_arg for example, because it's not real C variable and in this place it's clear how it can appear in concatenation.

😢 Preproc directives parsing definitely requires improvements.

[amaanq]

Preproc directives parsing definitely requires improvements.

Agreed..