Make invalid Unicode escapes a tokenizer error

[krobelus]

We report a tokenizer error on some invalid escape sequences

$ echo \777
fish: Invalid token '\777'
echo \777
     ^

but on others we silently interpret them as NUL bytes, which breaks many of our internal assumptions.

$ echo \utest

$

The fixes to #8195 fixed resulting issues in autoloading.
We should report a tokenizer error and maybe revert the fixes.

[faho]

A very simple fix would be possible:

fish-shell/src/common.cpp

Lines 1229 to 1234 in 6fcb6f7

	if (res <= max_val) {
	result_char_or_none =
	static_cast<wchar_t>((byte_literal ? ENCODE_DIRECT_BASE : 0) + res);
	} else {
	errored = true;
	}

Make that if (res > 0 && res <= max_val). If no valid unicode escape is read this stays at 0.

That has the side effect of making \x00 an error as well, which is probably correct considering it can't be passed to commands, not even (currently) to builtins. So it causes things like #8468, which we currently answer by saying "don't do that, then", when, as far as I can tell, the only use for these is in string replace -r inside quotes. This could be handled with a flag to indicate NULLs are acceptable.

The error message might need some work (this would error with "invalid token" with no further explanation).

[faho]

I went for the simple fix - escapes with no valid digits are rejected. Other things, including NULLs (which are unusable in most contexts) can be done later.