Add flag to `read` to split on NUL #1694

Closed
kballard opened this Issue Sep 20, 2014 · 6 comments

Comments

Projects
None yet
2 participants
@kballard
Contributor

kballard commented Sep 20, 2014

read should have a flag that makes it split on NUL instead of on $IFS. There are a number of commands that use NUL as a record separator, and it would be great if fish could loop over those. Since fish does not support using NUL on the command-line, there is no way to set $IFS to contain NUL, and therefore no way to split on NUL at the moment. Ideally, fish would learn to accept NUL everywhere, but that seems extremely unlikely to happen given the frequency with which fish has to use wchar_t* instead of wcstring.

Similarly, it would be great if command substitution could learn to optionally split on NUL instead of newline, but I can't think of any good syntax for that. However, if read can split on NUL, then the problem can be worked around by piping the output into read instead.

@kballard

This comment has been minimized.

Show comment
Hide comment
@kballard

kballard Sep 20, 2014

Contributor

@ridiculousfish I'm happy to implement this, but I want to run it by you first. Thoughts?

Contributor

kballard commented Sep 20, 2014

@ridiculousfish I'm happy to implement this, but I want to run it by you first. Thoughts?

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish Sep 20, 2014

Member

We try to support NUL bytes in echo output. Is there a particular use case you have in mind? Maybe find -print0?

Member

ridiculousfish commented Sep 20, 2014

We try to support NUL bytes in echo output. Is there a particular use case you have in mind? Maybe find -print0?

@kballard

This comment has been minimized.

Show comment
Hide comment
@kballard

kballard Sep 21, 2014

Contributor

I didn't realize echo -e could emit NUL, but I can't do a whole lot with that unless I'm piping it to another program. I certainly can't save it to a variable and use that later. Heck, I can't even tell if the NUL byte is successfully saved to the variable and simply echoing it later truncates on the NUL, or if the variable itself truncates on the NUL when it's saved.

The use case I have in mind right now is looping over the results of a command that uses NUL separators. find -print0 is a good example, though in my case it's actually git config --null. The git completions right now use git config for aliases, but an alias that contains a newline has the potential to confuse it, and I wanted to switch to using the --null flag, but I can't because I have no way of actually processing the output.

Contributor

kballard commented Sep 21, 2014

I didn't realize echo -e could emit NUL, but I can't do a whole lot with that unless I'm piping it to another program. I certainly can't save it to a variable and use that later. Heck, I can't even tell if the NUL byte is successfully saved to the variable and simply echoing it later truncates on the NUL, or if the variable itself truncates on the NUL when it's saved.

The use case I have in mind right now is looping over the results of a command that uses NUL separators. find -print0 is a good example, though in my case it's actually git config --null. The git completions right now use git config for aliases, but an alias that contains a newline has the potential to confuse it, and I wanted to switch to using the --null flag, but I can't because I have no way of actually processing the output.

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish Sep 21, 2014

Member

Sounds good to me. Is there precedence for the --null flag? The only analogous flag I'm familiar with is -0 from xargs.

Member

ridiculousfish commented Sep 21, 2014

Sounds good to me. Is there precedence for the --null flag? The only analogous flag I'm familiar with is -0 from xargs.

@kballard

This comment has been minimized.

Show comment
Hide comment
@kballard

kballard Sep 21, 2014

Contributor

find -print0 | xargs -0 is by far the most common usage of this, but using NUL as a separator does show up from time to time in other commands. It generally shows up in commands that have a chance of including newline in their data and are designed for consumption by computers. git config falls into this category quite nicely.

I can't think of any other cases offhand, but I know they exist.

Contributor

kballard commented Sep 21, 2014

find -print0 | xargs -0 is by far the most common usage of this, but using NUL as a separator does show up from time to time in other commands. It generally shows up in commands that have a chance of including newline in their data and are designed for consumption by computers. git config falls into this category quite nicely.

I can't think of any other cases offhand, but I know they exist.

@kballard kballard self-assigned this Sep 21, 2014

@kballard

This comment has been minimized.

Show comment
Hide comment
@kballard

kballard Sep 22, 2014

Contributor

I just realized I expressed this wrong. read does two things: it splits the input into lines, and then splits each line on $IFS. What I actually want to do is replace the line-splitting with splitting on NUL, not replace the $IFS splitting.

Contributor

kballard commented Sep 22, 2014

I just realized I expressed this wrong. read does two things: it splits the input into lines, and then splits each line on $IFS. What I actually want to do is replace the line-splitting with splitting on NUL, not replace the $IFS splitting.

@kballard kballard closed this in 8f8c4cd Sep 22, 2014

@kballard kballard added this to the next-minor milestone Sep 22, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment