Syntax improvements for redirection & pipes

[dhasenan]

Assuming I have a process that writes to both stderr and stdout, and I want to pipe both to tee, I should be able to do:

./a.out ^&1 | tee log

This ends up with 'log' being empty.

I suspect that fish is assigning the stderr input stream to the stdout sink, then assigning the stdout input stream to the pipe sink. Another layer of indirection would likely solve this -- instead of copying the stdout sink, you assign a wrapper sink instead. This would require caution to catch the case of redirecting >&2 ^&1, of course, but it would allow people to redirect and pipe at the same time and make it work.

[dag]

I ran into this issue almost immediately so I'd love to see a solution.

[JanKanis]

Can anyone give or link to an exact description of how bash handles the interaction of redirections and pipes? I can't find it in the bash reference manual. I always thought the bash equivalent of the o.p.'s command would give the same result as fish, but it turns out that command 2>&1 | tee myfile sends both stdout and stder of command to tee.

My (apparently wrong) mental model of redirection in bash is that cmd1 2>&1 | cmd2 does:

• assign the value of fd 1 (the terminal) to cmd1's fd2
• assign the value of the pipe to cmd1's fd1.
  Fish (afaik) works like that, but what does bash do exactly?

[gustafj]

In the manual for Bash under Pipelines i found this:
This connection is performed before any redirections specified by the command.
see http://www.gnu.org/software/bash/manual/bashref.html#Pipelines

So the redirection would be like this if my understanding is correct:

• connect the fd1 to the pipe
• assign the value of fd1 (the pipe) to cmd1's fd2

It is possible to do this today in fish using special syntax 2>| ex command 2>| tee myfile
In Bash this is equivalent to |& which is just syntactic sugar for 2>&1 |

I did a few trials using fish, bash and tcsh (note that my tcsh dont support to only redirect stderr, as noted in that cmd):

cmd is perl -e 'print {*STDOUT} "stdout\n"; print {*STDERR} "stderr\n";' in all below:

tcsh > cmd | cat - > /dev/null
stderr
tcsh > cmd |& cat - > /dev/null
tcsh > cmd |& cat -
stderr
stdout
tcsh > "The  shell  cannot  presently  redirect  diagnostic  output without also redirecting standard output"

bash-4.1$ cmd | cat - > /dev/null
stderr
bash-4.1$ cmd |& cat - > /dev/null
bash-4.1$ cmd |& cat -
stderr
stdout
bash-4.1$ cmd 2>&1 | cat - > /dev/null


fish > cmd | cat - > /dev/null
stderr
fish > cmd 2>| cat - > /dev/null
fish > cmd 2>| cat -
stdout
stderr
fish > cmd 2>&1 | cat - > /dev/null 
stderr

[dag]

My expectation would be that 2>| would pipe stderr but not stdout. And also that if that syntax works then so should ^|, doing the same. Piping both is likely more useful, but might arguably break "The law of user focus" in that it's not intuitive.

[gustafj]

I like the ^| syntax, feels more "fish-like", and i also agree that when using 2>| (or ^|) one could expect it to only pipe stderr.
But then what syntax would be appropriate to use when piping both (which is probably what you most often want, except for when only piping stdout).

Proposal:

• Only stdout |, >| or 1>|
• Only stderr ^| or 2>|
• Both stderr & stdout ^>| or >^| or any combination of redirection followed by pipe, ex: ^&1 | or >&2 ^|

[dag]

The good thing is that the syntax is beginning to look like a fish ...

[ridiculousfish]

Let’s try to fix this one in next-minor

[ridiculousfish]

If you can pipe stdout and stderr independently, how do pipes associate? For example:

foo ^| bar | baz

Does baz get the output of foo or bar? Anyone know how other shells handle this?

[timbertson]

If we had subshells, I assume the answer would be that you'd need to do the following if you want stderr to go through multiple processes:

$ foo ^| (bar | baz) | frob

stderr goes through baz | bar, while stdout goes through frob. I thought the above would be possible in bash, but it seems you only get one pipe in bash (http://stackoverflow.com/questions/2342826/how-to-pipe-stderr-and-not-stdout)

Is there any subshell-like thing in fish? I often want to do stuff in another dir temporarily, which I used to do in bash with (cd DIR; do-something). I miss that.

[LunaBorowska]

@gfxmonk: But we do.

foo ^| begin
    bar | baz
end | frob

[anddam]

@glitchmr that's brilliant

[ridiculousfish]

It sounds like the proposed answer to my question is that a stderr-only pipe would be standalone, and not part of the larger pipeline. That is, foo ^| bar | baz is foo ^| bar and foo | baz. If you wanted to pipe stderr further, you use begin/end to create a block. That seems reasonable to me.

[mrshu]

This has been a serious issue for me too so I'd love to see a solution too.

@glitchmr that looks very fishy!

[ridiculousfish]

With the commit 4899086 , the original problem ./a.out ^&1 | tee log should be addressed. You can assign stdout to stderr, and that happens after the pipe, so that tee sees both streams.

I want to leave this open for further discussion on syntax improvements for redirections. The current thinking is that foo | bar pipes stdout, and foo ^| bar pipes stderr. The syntax for piping both is unclear, even though there's agreement that piping both is more common than piping stderr independently.