yaml syntax does not handle \ characters in multiline strings

[Nonoctis]

Steps to reproduce

Tested with this snippet:

---
- hosts: localhost
  vars:
    win_shell: |
      Move-Item -Path "C:\OldPath\*.dat" -Destination "C:\NewPath\"

  tasks:
    - debug:
        msg: "{{win_shell}}"

The result is this:

Expected behaviour

The syntax should not count \ characters as escaping characters in a multiline string,

Version of Vim

8.2

Environment

WSL 2.0 - Debian 11.5
Windows Terminal
xterm-256color
zsh

Can be reproduced on other distros

Logs and stack traces

No response

[porygonisaduck]

I'm also interested in tackling this bug, can I be assigned to this issue as well?

[chrisbra]

we do not assign anybody to issues. If you want to check it out, you may have a look at the yaml syntax file @ZyX-I is no longer activer, so any fixes will be appreciated.

But be aware, it may be a bit of a complex issue to fix (you have to know syntax highlighting rules and also regular expressions)

[porygonisaduck]

I was able to find a regular expression that works for this case but I cannot understand the syntax highlighting rules so I am pasting the regular expression that worked for me if anyone else wants to continue working on this.
(\t+)(.+\|)(\n)((\1\t)(.+)(\n)+)+
Here is the link to my work in progress using regex 101: https://regex101.com/r/KH4cy0/1

[chrisbra]

I tested this out a bit. Is there a reason, not to use single-quote strings here? Since this is about path-names in Windows, single-quotes seem more appropriate anyhow, as you do not want the \ to be considered as an escape character.

[Nonoctis]

When you use scalar blocks text is interpreted literally (except for {{ jinja2 }} templates.
Here is a quick playbook you can try:

---
- hosts:
  - localhost
  gather_facts: no
  tasks:
    - debug:
        msg: |
          Let's put some quotes here
          Then some backscaped character\n
          Then some more 'quotes'
          And even "doubl quotes"
          "This line breaks everything\"

    - name: This is where vim breaks
      debug:
        msg: Hello
      when: false

The result I get (using the yaml callback for better readability):

[chrisbra]

This seems to be caused by the skip expression in the yamlFlowString definition here:

vim/runtime/syntax/yaml.vim

Lines 113 to 119 in 6342e2c

	syn region yamlFlowString matchgroup=yamlFlowStringDelimiter start='"' skip='\\"' end='"'
	\ contains=yamlEscape
	\ nextgroup=yamlKeyValueDelimiter
	syn region yamlFlowString matchgroup=yamlFlowStringDelimiter start="'" skip="''" end="'"
	\ contains=yamlSingleEscape
	\ nextgroup=yamlKeyValueDelimiter
	syn match yamlEscape contained '\\\%([\\"abefnrtv\^0_ NLP\n]\|x\x\x\|u\x\{4}\|U\x\{8}\)'

I suppose this is there to allow to add quotes inside double-quoted strings. So to fix this, the following patch seems to work, but I am not sure this is correct, since you may loose the abibility to add quoted double quotes.

diff --git a/runtime/syntax/yaml.vim b/runtime/syntax/yaml.vim
index 49f7d049a..7f36b9a17 100644
--- a/runtime/syntax/yaml.vim
+++ b/runtime/syntax/yaml.vim
@@ -110,13 +110,13 @@ syn match yamlYAMLVersion   '\d\+\.\d\+' contained nextgroup=yamlComment
 execute 'syn match yamlReservedDirective contained nextgroup=yamlComment '.
             \string('%\%(\%(TAG\|YAML\)\s\)\@!'.s:ns_directive_name)

-syn region yamlFlowString matchgroup=yamlFlowStringDelimiter start='"' skip='\\"' end='"'
+syn region yamlFlowString matchgroup=yamlFlowStringDelimiter start='"' skip='\\' end='"'
             \ contains=yamlEscape
             \ nextgroup=yamlKeyValueDelimiter
 syn region yamlFlowString matchgroup=yamlFlowStringDelimiter start="'" skip="''"  end="'"
             \ contains=yamlSingleEscape
             \ nextgroup=yamlKeyValueDelimiter
-syn match  yamlEscape contained '\\\%([\\"abefnrtv\^0_ NLP\n]\|x\x\x\|u\x\{4}\|U\x\{8}\)'
+syn match  yamlEscape contained '\\\%([\\abefnrtv\^0_ NLP\n]\|x\x\x\|u\x\{4}\|U\x\{8}\)'
 syn match  yamlSingleEscape contained "''"

 syn match yamlBlockScalarHeader contained '\s\+\zs[|>]\%([+-]\=[1-9]\|[1-9]\=[+-]\)\='

So, why do you add the escaped \" in your example?. It looks wrong, there should still be a closing double quote.

[Nonoctis]

So, why do you add the escaped \" in your example?. It looks wrong, there should still be a closing double quote.

If my understanding of yaml is correct, it is actually not escaped, text inside a scalar block is literal, like text between single quotes in bash (with the only exception being that jinja templates are expanded), so \" really means the character \ followed by the character ".

[chrisbra]

ah, it's a literal block. I agree, it shouldn't then include the yamlFlowString with escapes. Unfortunately the current syntax file doesn't seem to distinguish literal blocks and it is too hard to follow the logic there and I do not know the language good enough to make an educated guess here :(

[daviewales]

Another example:

example: |
  test='{ example }'
  test2='{ example2 }'
string: 'here'

If you add a space after the =, or remove the space before the } on line two, the highlighting changes.

Also note that Jinja templates are not part of YAML, so the block should be considered simply as a literal string with no special characters, starting after the | and the indentation, and continuing onwards until the end of the indented block.