Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove caret (^) as redirector for stderr #4394

Closed
ThomasAH opened this issue Sep 10, 2017 · 19 comments
Closed

Remove caret (^) as redirector for stderr #4394

ThomasAH opened this issue Sep 10, 2017 · 19 comments
Labels
Milestone

Comments

@ThomasAH
Copy link

@ThomasAH ThomasAH commented Sep 10, 2017

I propose removing the caret (^) as a redirector for stderr in fish 3.0.

Reason 1: This is surprising for users of other shells and may lead to data loss

There are many commands using the caret on the command line for special purposes, e.g. git (see #1873). While some special handling has been introduced to reduce the need for quoting (see Reason 2), this does not help in all cases, e.g. git log ^refA refB will create a file refA containing the error message of git.
Many other programs use ^, too, for me the most notable problem is grep.
I often use the command dpkg -l|grep -v ^ii to check for problems with not fully installed or purged Debian packages. In fish this prints nothing, and grep's error message is sent to the new file ii.
Or I use ps auxww|grep ^theuser to quickly find processes of some users.
Or grep ^http someconfigfile to see if the config file contains a setting about httpserver or httpproxy or ...

While this often just results in a stray file (which is already bad enough), this can easily overwrite existing files, because sometimes I am looking for something where a file of the same name exists, e.g. Makefile targets (which luckily can often be rebuilt), or filtered output of ls, e.g. ls | grep -v ^README to show me everything but README (which is now overwritten) and various README.foo files. Or ls | grep -v ^access.log | grep -v ^error.log to hide the current and rotated web server log files and see what other files are present. Yes, I could add quotes and I should add a backslash in front of the dot in the regexp, but I really don't expect a file accessblog here, so I don't do it in interactive use.

Reason 2: Workarounds have been introduced, but they introduce even more surprising behavior.

To workaround the problem in git (#1873), the handling of ^ was changed to be different than the handling of >, so echo foo^bar prints foo^bar, but echo foo ^ bar prints foo and creates an empty file bar. This is different from e.g. echo foo>bar and echo foo > bar which both do the same thing.

The need for this workaround indicates that using the caret as a redirection character is problematic.

Reason 3: The law of orthogonality (https://fishshell.com/docs/current/design.html#ortho)

^ is just a shortcut for 2>, so this syntax saves only a single character.
The user still has to know the difference between stdout and stderr to use this feature.
The user still has to know about file descriptors to redirect both, stdout and stderr, to the same file with echo Hello > all_output.txt ^&1.

@mqudsi
Copy link
Contributor

@mqudsi mqudsi commented Sep 10, 2017

I would be on board with this. It's just not a feature worth having.

@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Sep 11, 2017

I like this too.

@ridiculousfish ridiculousfish added this to the fish-3.0 milestone Sep 11, 2017
@zx8
Copy link

@zx8 zx8 commented Sep 13, 2017

Wow, I thought I was the only one who considered this a misfeature!

I'm very much in favour of its removal as well.

@krader1961
Copy link
Contributor

@krader1961 krader1961 commented Sep 14, 2017

FWIW, I agree this "feature" should be removed. But note that it doesn't affect just ^/a/path to redirect stderr to /a/path. It also affects ^| a_command. We have two such uses of the latter in the fish project: share/completions/apm.fish and share/functions/__fish_print_make_targets.fish. That there are only two such instances in the fish code suggests that use case is relatively rare. Nonetheless it is another use of the feature which will be broken if we implement this change.

@ThomasAH
Copy link
Author

@ThomasAH ThomasAH commented Sep 14, 2017

^| is nice for some situations, but there are already some alternative ways to do this:

if apm -h ^| string match -q "*Atom Package Manager*"
can be changed to
if apm -h 2>&1 | string match -q "*Atom Package Manager*"

make -C $directory -d g1 -rn >/dev/null ^| awk ...
can be changed to
make -C $directory -d g1 -rn 2>&1 >/dev/null | awk ...

If some really needs to print stdout and just wants to pipe/filter stderr, both can be swapped, e.g.:

ls -ld /etc /missing /invalid 3>&2 2>&1 1>&3 | grep miss
drwxr-xr-x 196 root root 12288 Sep 12 15:54 /etc
ls: cannot access /missing: No such file or directory

If above syntax is considered too complicated, the behaviour of | could be adjusted to match the behaviour of > by allowing file descriptors, so 2| would pipe stderr. This is more consistent, allows piping of other file descriptors, and accidents are far less likely to cause data loss, because e.g. grep foo 2|grep bar will not clobber existing files.

@mqudsi
Copy link
Contributor

@mqudsi mqudsi commented Sep 14, 2017

There's already a good workaround for it, one that is even less likely to be triggered by accident.

mqudsi@ZBook /m/c/U/Mahmoud> fish -c "echo hello >&2" | grep foo
hello
mqudsi@ZBook /m/c/U/Mahmoud> fish -c "echo hello >&2" 2>| grep foo
mqudsi@ZBook /m/c/U/Mahmoud> fish -c "echo hello >&2" 2>| grep hello
hello

I think it's a highly underrated feature, and I don't think any other "modern" shell makes it this easy.

@mqudsi mqudsi changed the title RFC: Remove caret (^) as redirector for stderr Remove caret (^) as redirector for stderr Oct 10, 2017
@mqudsi mqudsi added the RFC label Oct 10, 2017
@scooter-dangle
Copy link

@scooter-dangle scooter-dangle commented Feb 15, 2018

What's fish's story on deprecation warnings? I pretty liberally use ^/dev/null when I intend to ignore errors. (Although it will be pretty annoying to get a deprecation notice on stderr if I'm already explicitly ignoring errors for a particular command/function.)

@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Apr 1, 2018

The 3.0 release is making these sort of changes without a deprecation phase, in part for the reason you say

ridiculousfish pushed a commit to ridiculousfish/fish-shell that referenced this issue Apr 1, 2018
This removes the caret as a shorthand for redirecting stderr.

Note that stderr may be redirected to a file via 2>/some/path...
and may be redirected with a pipe via 2>|.

Fixes fish-shell#4394
jethrokuan added a commit to jethrokuan/z that referenced this issue Apr 2, 2018
The caret (`^`) has been removed as a redirector for `stderr` in Fish v3, see fish-shell/fish-shell#4394 for details.
@lilyball
Copy link
Member

@lilyball lilyball commented Apr 9, 2018

No deprecation phase is problematic because it means existing scripts will simply break without warning. This apparently includes fisherman and at least one fisherman plugin.

Could we at least add a deprecation phase for master which we then turn off when releasing 3.0? That way people testing on master can try and at least flag shared components (like fisherman) that need to be updated.

@ridiculousfish ridiculousfish reopened this Apr 9, 2018
@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Apr 9, 2018

Reopening so we can think harder about how to manage this.

@lrm29
Copy link
Contributor

@lrm29 lrm29 commented Apr 9, 2018

I've just run into this breaking the fish scripts in (https://github.com/cea-hpc/modules). Would have been useful to have seen a deprecation message rather than "^: No such file or directory"!

Note that also need to update the doc to remove references to the caret:

https://fishshell.com/docs/current/tutorial.html#tut_pipes_and_redirections

@alcarithemad
Copy link

@alcarithemad alcarithemad commented Apr 27, 2018

For "reasons", I've been writing a parser for the fish shell language, and found this issue while looking to see if echo a^/dev/null was a reported bug, as the behavior surprised me. I quite like ^ as an operator and am somewhat sad to see it go, but it does make my parsing work easier.

I'd comment more on the removal but in short: I disagree with reason 1, think the workaround is reason 2 is bad, and have no opinion on reason 3. Fish should either go all-in on ^ being exactly like > and <, or get rid of it.

@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Apr 28, 2018

UIs are a messy affair.

@faho
Copy link
Member

@faho faho commented May 8, 2018

#4940 includes this as one of our "feature" flags, so we do have actual deprecation now.

Closing!

@faho faho closed this May 8, 2018
gyakovlev added a commit to jorgebucaran/fisher that referenced this issue May 16, 2018
`^` has been removed as a stderr redirection operator
(fish-shell/fish-shell#4394) in future versions of fish.

Closes: #416
jorgebucaran pushed a commit to jorgebucaran/fisher that referenced this issue Sep 7, 2018
`^` has been removed as a stderr redirection operator
(fish-shell/fish-shell#4394) in future versions of fish.

Closes: #416
@jorgebucaran
Copy link

@jorgebucaran jorgebucaran commented Sep 18, 2018

👏 Good job everyone and thank you, @ThomasAH for getting the ball rolling on this.

matchai added a commit to matchai/spacefish that referenced this issue Dec 28, 2018
Fish plans to deprecate caret stderr redirection in favor of POSIX-compatible redirection (fish-shell/fish-shell#4394)
jichu4n added a commit to jichu4n/fish-command-timer that referenced this issue Dec 30, 2018
The ^ operator has been deprecated in fish-shell/fish-shell#4394 in
favor of "2>".

Fixes issue #18.
@adeftu
Copy link

@adeftu adeftu commented Jan 4, 2019

Issue is marked with fish 3.0.0 milestone and closed. However current 3.0.0 version does not include it.

@floam
Copy link
Member

@floam floam commented Jan 4, 2019

It's hidden behind a feature flag, stderr-nocaret.

uberjay added a commit to uberjay/fish-ghq that referenced this issue Feb 19, 2019
See this GH issue for more information:

fish-shell/fish-shell#4394
markstos added a commit to markstos/fish-cookbook that referenced this issue Jun 13, 2019
It's deprecated in 3.0 and will be removed.

Ref: fish-shell/fish-shell#4394
@ron-wolf
Copy link

@ron-wolf ron-wolf commented Nov 12, 2019

@floam Doesn’t that kind of defeat the purpose? Either run into weird behavior with ^—which is un-fish-like for the reasons @ThomasAH mentioned above—or add an extra line to your config files, to handle your otherwise smooth shell’s one weird behavior.

@faho
Copy link
Member

@faho faho commented Nov 12, 2019

@ron-wolf The feature flag is purely a transition thing, and will be removed in a future version of fish.

getkey added a commit to getkey/dotfiles that referenced this issue Mar 1, 2020
JoshCheek added a commit to JoshCheek/dotfiles that referenced this issue Mar 28, 2020
JoshCheek added a commit to JoshCheek/dotfiles that referenced this issue Mar 28, 2020
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet