New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove caret (^) as redirector for stderr #4394

Closed
ThomasAH opened this Issue Sep 10, 2017 · 17 comments

Comments

Projects
None yet
@ThomasAH
Copy link

ThomasAH commented Sep 10, 2017

I propose removing the caret (^) as a redirector for stderr in fish 3.0.

Reason 1: This is surprising for users of other shells and may lead to data loss

There are many commands using the caret on the command line for special purposes, e.g. git (see #1873). While some special handling has been introduced to reduce the need for quoting (see Reason 2), this does not help in all cases, e.g. git log ^refA refB will create a file refA containing the error message of git.
Many other programs use ^, too, for me the most notable problem is grep.
I often use the command dpkg -l|grep -v ^ii to check for problems with not fully installed or purged Debian packages. In fish this prints nothing, and grep's error message is sent to the new file ii.
Or I use ps auxww|grep ^theuser to quickly find processes of some users.
Or grep ^http someconfigfile to see if the config file contains a setting about httpserver or httpproxy or ...

While this often just results in a stray file (which is already bad enough), this can easily overwrite existing files, because sometimes I am looking for something where a file of the same name exists, e.g. Makefile targets (which luckily can often be rebuilt), or filtered output of ls, e.g. ls | grep -v ^README to show me everything but README (which is now overwritten) and various README.foo files. Or ls | grep -v ^access.log | grep -v ^error.log to hide the current and rotated web server log files and see what other files are present. Yes, I could add quotes and I should add a backslash in front of the dot in the regexp, but I really don't expect a file accessblog here, so I don't do it in interactive use.

Reason 2: Workarounds have been introduced, but they introduce even more surprising behavior.

To workaround the problem in git (#1873), the handling of ^ was changed to be different than the handling of >, so echo foo^bar prints foo^bar, but echo foo ^ bar prints foo and creates an empty file bar. This is different from e.g. echo foo>bar and echo foo > bar which both do the same thing.

The need for this workaround indicates that using the caret as a redirection character is problematic.

Reason 3: The law of orthogonality (https://fishshell.com/docs/current/design.html#ortho)

^ is just a shortcut for 2>, so this syntax saves only a single character.
The user still has to know the difference between stdout and stderr to use this feature.
The user still has to know about file descriptors to redirect both, stdout and stderr, to the same file with echo Hello > all_output.txt ^&1.

@mqudsi

This comment has been minimized.

Copy link
Contributor

mqudsi commented Sep 10, 2017

I would be on board with this. It's just not a feature worth having.

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Sep 11, 2017

I like this too.

@ridiculousfish ridiculousfish added this to the fish-3.0 milestone Sep 11, 2017

@zx8

This comment has been minimized.

Copy link

zx8 commented Sep 13, 2017

Wow, I thought I was the only one who considered this a misfeature!

I'm very much in favour of its removal as well.

@krader1961

This comment has been minimized.

Copy link
Contributor

krader1961 commented Sep 14, 2017

FWIW, I agree this "feature" should be removed. But note that it doesn't affect just ^/a/path to redirect stderr to /a/path. It also affects ^| a_command. We have two such uses of the latter in the fish project: share/completions/apm.fish and share/functions/__fish_print_make_targets.fish. That there are only two such instances in the fish code suggests that use case is relatively rare. Nonetheless it is another use of the feature which will be broken if we implement this change.

@ThomasAH

This comment has been minimized.

Copy link

ThomasAH commented Sep 14, 2017

^| is nice for some situations, but there are already some alternative ways to do this:

if apm -h ^| string match -q "*Atom Package Manager*"
can be changed to
if apm -h 2>&1 | string match -q "*Atom Package Manager*"

make -C $directory -d g1 -rn >/dev/null ^| awk ...
can be changed to
make -C $directory -d g1 -rn 2>&1 >/dev/null | awk ...

If some really needs to print stdout and just wants to pipe/filter stderr, both can be swapped, e.g.:

ls -ld /etc /missing /invalid 3>&2 2>&1 1>&3 | grep miss
drwxr-xr-x 196 root root 12288 Sep 12 15:54 /etc
ls: cannot access /missing: No such file or directory

If above syntax is considered too complicated, the behaviour of | could be adjusted to match the behaviour of > by allowing file descriptors, so 2| would pipe stderr. This is more consistent, allows piping of other file descriptors, and accidents are far less likely to cause data loss, because e.g. grep foo 2|grep bar will not clobber existing files.

@mqudsi

This comment has been minimized.

Copy link
Contributor

mqudsi commented Sep 14, 2017

There's already a good workaround for it, one that is even less likely to be triggered by accident.

mqudsi@ZBook /m/c/U/Mahmoud> fish -c "echo hello >&2" | grep foo
hello
mqudsi@ZBook /m/c/U/Mahmoud> fish -c "echo hello >&2" 2>| grep foo
mqudsi@ZBook /m/c/U/Mahmoud> fish -c "echo hello >&2" 2>| grep hello
hello

I think it's a highly underrated feature, and I don't think any other "modern" shell makes it this easy.

@mqudsi mqudsi changed the title RFC: Remove caret (^) as redirector for stderr Remove caret (^) as redirector for stderr Oct 10, 2017

@mqudsi mqudsi added the RFC label Oct 10, 2017

@scooter-dangle

This comment has been minimized.

Copy link

scooter-dangle commented Feb 15, 2018

What's fish's story on deprecation warnings? I pretty liberally use ^/dev/null when I intend to ignore errors. (Although it will be pretty annoying to get a deprecation notice on stderr if I'm already explicitly ignoring errors for a particular command/function.)

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Apr 1, 2018

The 3.0 release is making these sort of changes without a deprecation phase, in part for the reason you say

ridiculousfish pushed a commit to ridiculousfish/fish-shell that referenced this issue Apr 1, 2018

Peter Ammon
Remove caret redirection
This removes the caret as a shorthand for redirecting stderr.

Note that stderr may be redirected to a file via 2>/some/path...
and may be redirected with a pipe via 2>|.

Fixes fish-shell#4394

lithammer pushed a commit to lithammer/z that referenced this issue Apr 2, 2018

Peter Renström
Use traditional redirection for stderr
The caret (`^`) has been removed as a redirector for `stderr` in Fish v3, see fish-shell/fish-shell#4394 for details.

jethrokuan added a commit to jethrokuan/z that referenced this issue Apr 2, 2018

Use traditional redirection for stderr (#58)
The caret (`^`) has been removed as a redirector for `stderr` in Fish v3, see fish-shell/fish-shell#4394 for details.
@kballard

This comment has been minimized.

Copy link
Contributor

kballard commented Apr 9, 2018

No deprecation phase is problematic because it means existing scripts will simply break without warning. This apparently includes fisherman and at least one fisherman plugin.

Could we at least add a deprecation phase for master which we then turn off when releasing 3.0? That way people testing on master can try and at least flag shared components (like fisherman) that need to be updated.

@ridiculousfish ridiculousfish reopened this Apr 9, 2018

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Apr 9, 2018

Reopening so we can think harder about how to manage this.

@lrm29

This comment has been minimized.

Copy link
Contributor

lrm29 commented Apr 9, 2018

I've just run into this breaking the fish scripts in (https://github.com/cea-hpc/modules). Would have been useful to have seen a deprecation message rather than "^: No such file or directory"!

Note that also need to update the doc to remove references to the caret:

https://fishshell.com/docs/current/tutorial.html#tut_pipes_and_redirections

neersighted added a commit to neersighted/fisherman that referenced this issue Apr 14, 2018

Use standard sh-style redirection for stderr
`^` has been removed as a stderr redirection operator (fish-shell/fish-shell#4394) in future versions of fish.
@alcarithemad

This comment has been minimized.

Copy link

alcarithemad commented Apr 27, 2018

For "reasons", I've been writing a parser for the fish shell language, and found this issue while looking to see if echo a^/dev/null was a reported bug, as the behavior surprised me. I quite like ^ as an operator and am somewhat sad to see it go, but it does make my parsing work easier.

I'd comment more on the removal but in short: I disagree with reason 1, think the workaround is reason 2 is bad, and have no opinion on reason 3. Fish should either go all-in on ^ being exactly like > and <, or get rid of it.

@ridiculousfish

This comment has been minimized.

Copy link
Member

ridiculousfish commented Apr 28, 2018

UIs are a messy affair.

@faho

This comment has been minimized.

Copy link
Member

faho commented May 8, 2018

#4940 includes this as one of our "feature" flags, so we do have actual deprecation now.

Closing!

@faho faho closed this May 8, 2018

gyakovlev added a commit to jorgebucaran/fisher that referenced this issue May 16, 2018

Use standard sh-style redirection for stderr (#416)
`^` has been removed as a stderr redirection operator
(fish-shell/fish-shell#4394) in future versions of fish.

Closes: #416

jorgebucaran pushed a commit to jorgebucaran/fisher that referenced this issue Sep 7, 2018

Use standard sh-style redirection for stderr (#416)
`^` has been removed as a stderr redirection operator
(fish-shell/fish-shell#4394) in future versions of fish.

Closes: #416
@jorgebucaran

This comment has been minimized.

Copy link

jorgebucaran commented Sep 18, 2018

👏 Good job everyone and thank you, @ThomasAH for getting the ball rolling on this.

matchai added a commit to matchai/spacefish that referenced this issue Dec 28, 2018

Replace caret redirection
Fish plans to deprecate caret stderr redirection in favor of POSIX-compatible redirection (fish-shell/fish-shell#4394)

jichu4n added a commit to jichu4n/fish-command-timer that referenced this issue Dec 30, 2018

Use 2> instead of ^ for stderr redirection.
The ^ operator has been deprecated in fish-shell/fish-shell#4394 in
favor of "2>".

Fixes issue #18.
@adeftu

This comment has been minimized.

Copy link

adeftu commented Jan 4, 2019

Issue is marked with fish 3.0.0 milestone and closed. However current 3.0.0 version does not include it.

@floam

This comment has been minimized.

Copy link
Member

floam commented Jan 4, 2019

It's hidden behind a feature flag, stderr-nocaret.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment