Inconsistent backslash escaping in literal strings

[KrisShannon]

Describe the bug
Can't write a literal string with a single backslash in it. To obtain a string with a single backslash requires the use of other functions.

Version of yq: 4.33.3
Operating system: mac
Installed via: binary release

Command
The command you ran:

yq -nr eval '"\\"'
yq -nr eval '"\"'
yq -nr eval '"\\"|match("^(.)")|.string'

Actual behavior

\\
Error: 1:1: invalid input text "\"\\\""
\

Expected behavior

Either:

\
Error: 1:1: invalid input text "\"\\\""
\

Or:

\\
\
\

Additional context
It appears that the only backslash escape that is recognised is \n. There is no \r, \t, \b, \xHH or \uXXXX. All other alphabetics are left alone with the preceding backslash left in the string.

I also tried to use single quotes and backquotes in case they were alternate string literal expressions but they also just gave the invalid input text error.

The actual workaround I am currently using is:

BACKSLASH='\' yq -nr eval 'strenv(BACKSLASH)'

(It's actually a much more complicated expression that happens to include ... | split(strenv(BACKSLASH)) | ... as part of it)

[KrisShannon]

Just downloaded the new v4.34.1 and tested with it.

Appears to be the same behaviour

[KrisShannon]

This appears to be the lexing of the literal strings:

func stringValue() yqAction {
	return func(rawToken lexer.Token) (*token, error) {
		log.Debug("rawTokenvalue: %v", rawToken.Value)
		value := unwrap(rawToken.Value)
		log.Debug("unwrapped: %v", value)
		value = strings.ReplaceAll(value, "\\\"", "\"")
		value = strings.ReplaceAll(value, "\\n", "\n")
		log.Debug("replaced: %v", value)
		return &token{TokenType: operationToken, Operation: createValueOperation(value, value)}, nil
	}
}

Replacing just \n and \" makes it very difficult to use literal strings consistently.

Because it doesn't replace \\ you can't just double all the backslashes in a string and surround it with "'s

Because it replaces \n you also can't create a string with for example the value C:\tmp\nigel.xlsx

[mikefarah]

Yeah that's annoying - and tricky to solve. Happy to take a PR for this.

[KrisShannon]

Created a pull request #1814 with a possible fix.