string escape drops NUL and following text

[cben]

fish, version 2.7.0-513-g81dd4a45
Linux, Fedora 27, gnome-terminal (TERM=xterm-256cotor).
Confirmed without customization (sh -c 'env HOME=$(mktemp -d) fish').

Actual behavior: string escape doesn't handle NUL:

/h/b/fish-shell> printf 'foo\0bar' | cat -A
foo^@bar⏎
/h/b/fish-shell> printf 'foo\0bar' | string escape
foo

and string unescape has similar problem:

/h/b/fish-shell> printf 'escaped\\0text' | string unescape
escaped

Expected behavior:

/h/b/fish-shell> printf 'foo\0bar' | string escape
foo\0bar
/h/b/fish-shell> printf 'escaped\\0text' | string unescape | cat -A
escaped^@text⏎

Generally, it should be possible to round-trip any binary data through | string escape | string unescape |.

[faho]

Yup. The issue is that we're still using c-style strings all over the place. This might take a while to solve.

[ridiculousfish]

This is fixable in each individual case - if there's some particular case that's blocking you please let us know.

[faho]

I've now implemented this in a branch for the "script" escaping style (see the commit linked above).

Note that it's a bit hacky because it needs to get the wcstring, so I just added a method to get a reference to the arg_iterator_t's storage_, and since I still don't fully grok C++ reference lifetimes that might be not entirely correct.

I want to implement it for the other escape styles as well before I merge it.

[faho]

I have now converted all string subcommands except for match and replace. Those are a bit trickier because of the interactions with pcre.

Also, as it turns out escape, unescape, upper and lower have a bunch of duplicated code - the latter two do the exact same thing except one passes "std::towupper" and the other "std::towlower".