kill the IFS variable

[krader1961]

The discussion about doing a major release of fish prompted someone to add a comment to an issue targeted for that major release. Specifically, they said that having to set IFS to get the desired behavior isn't a big deal. To which I replied:

Also, it should never be necessary to manipulate IFS in fish script. That var should die and removing it is something else we should consider for a major release. Whenever I have needed to, or seen it used to, change how strings are split into words in other shells (ksh, bash, zsh) it always made me want to vomit. It is the poster child for what is wrong with POSIX 1003 compatible shells.

So I'm opening this issue to ask that we consider doing so in the next fish major release.

[faho]

So, there's two ways we're using $IFS currently:

• Set it to something for use with read - e.g. set IFS '=' and then while read -l key value to read from an INI-style file

• Unset it or set it to empty to avoid splitting command substitutions (set IFS; set var (cat somefile)) - note that this only cares if IFS is unset or empty, not what it is set to (set IFS '=' does not split command substitutions on '=').

The former can currently already be replaced by just reading the line and then set line (string split '=' -- $line); set key $line[1]; set value $line[2]. It could be done easier if read had an option to specify the separator - something | while read -l --separator = key value.

The second could be solved by implementing "$()", but that still wouldn't make it easy to split on arbitrary separators but not newlines.

[krader1961]

Faho's first example, as they noted, can be handled by explicitly using something like string split on the data that has been read. Or, better, adding a read --separator flag. Manipulating IFS in that situation is dangerous since in POSIX 1003 shells it affects not just how read splits its input but how every string is tokenized into words.

There are only two places in fish where we unset IFS to get the second behavior noted by faho. One is __fish_urlencode which will die when I merge the fix for #4150 in the next day or two. The second is funced and that would be addressed by implementing "$()".

[faho]

Or, better, adding a read --separator flag.

I have some initial work on that. I'm calling it "--delimiter" (and "-d"), though, both because "-s" is already taken and for symmetry with things like cut.

[krader1961]

In case it isn't obvious: When I say "kill the IFS variable" I only mean that no core fish C++ code or fish script will be affected by it. The user can still set it and export it to affect non-fish sub-shells. Although why you would ever do so is a mystery. This change is to make it even clearer that fish is not a POSIX 1003 shell with all the extremely weird string tokenization rules implied by that standard.

That fish currently uses IFS in a couple of situations (e.g., the read command) is unfortunate. It saddens me that the people working on fish long before I came on the scene recognized how awful POSIX 1003 was but didn't make a clean break from it. However, I may have made the same decision given the need to be practical rather than pure and given the available developer resources. Hindsight is 20-20.

[faho]

In case it isn't obvious: When I say "kill the IFS variable" I only mean that no core fish C++ code or fish script will be affected by it. The user can still set it and export it to affect non-fish sub-shells.

Of course. Though since IFS isn't exported right now, we would actually remove it completely from fish.

[krader1961]

Set it to something for use with read - e.g. set IFS '=' and then while read -l key value to read from an INI-style file

Note that this is readily dealt with using our string builtin. For example, share/completions/dd.fish had that IFS pattern:

            set -l IFS =
            echo $operand_string | read -l operand value

I replaced that with this:

string replace = ' ' -- $operand_string | read -l operand complete

[krader1961]

Also, unsetting IFS to avoid splitting command substitutions can be handled by replacing

set -e IFS
set var (command_sub)

with

command_sub | read -lz var

Since the command substitution should not be producing null delimited output that is equivalent to the first idiom. So while there are good reasons to introduce things like read --delim=x and $() they aren't necessary to eliminate our uses of IFS.

[krader1961]

I've removed all the IFS uses from our completions and functions. That just leaves modifying the read command, its tests, and documentation to make this a done deal.

[krader1961]

Okay, we've done 95% of the work. The remaining use in read is needed during a transition period. We can open a new issue in the 3.2 or 3.3 timeframe to discuss removing the remaining fallback to IFS.