Skip to content

echo "foo" >&2 is not colorized #16

Closed
JVApen opened this Issue Jan 3, 2012 · 10 comments

6 participants

@JVApen
JVApen commented Jan 3, 2012

This program is what I was looking for,
and it works great using python, like the example,
but for some reason, it doesn't work with shell-scripts,

for example:

#!/bin/sh
echo "stdout"
echo "stderr" >&2

is all in white in the gnome-terminal (ubuntu 11.10 + gnome3)

@sickill
Owner
sickill commented Jan 3, 2012

I'm not sure if shell is using write() to print the messages. Let's check that.

@kaihendry

Noticed the same problem. :/

@sickill
Owner
sickill commented Jan 30, 2012

Ok guys, I will strace it and we'll see what the problem is.

@Bentis
Bentis commented Feb 8, 2012

Possibly related: error messages from gcc (g++) is not catched and printed in red. Same system config. as OP.
Edit: Library is loaded OK, and Python example code works.

@sickill
Owner
sickill commented Apr 28, 2012

@Bentis there is stream-functions branch where I added more wrapper for functions like fputs, fprintf etc. I checked and gcc/g++ is correctly colored on this branch. I'll use it for few more days and if no error appear I'll merge it into master.

@sickill
Owner
sickill commented Apr 28, 2012

As for shell scripts, I don't know what bash/sh and other shell are doing but it seems they detect that output would be printed to interactive shell and they rewire stderr descriptor (2) to stdout one (1) right before printing...

@sickill
Owner
sickill commented Apr 28, 2012

After reading a lot about process file descriptors, redirection and stuff the
conclusion is: there's no way to colorize text printed by "echo" command with stderred.

If you add 3rd line to your shell script:

#!/bin/sh
echo "stdout"
echo "stderr" >&2
python -c 'import os; os.write(2, "lolza")'

and run it you'll see that "lolza" is printed correctly in red. But it didn't
colorized output of second echo command. Why?

Because echo always writes to FD 1. >&2 just sets up duplication of
file descriptor and means more or less "when command on the left writes to FD
1 please actually write data to the same target place where FD 2 points to"
.

So echo writes to FD 1 and stderr checks if process is writing to FD 2. And
that unfortunately can't happen with echo.

See here for more info about the topic:
http://wiki.bash-hackers.org/howto/redirection_tutorial
http://sgf-dma.blogspot.com/2010/09/draftpart-shell-io-redirection.html

One solution I was evaluating was not to check if process is writing to FD with the number "2" but instead check if underlying file description structure is the same for FD 2 and FD to which process is writing. I tried that but it doesn't work properly because by default every process has both FD 1 and FD 2 connected to the same /dev/pts/X file (where X is number) resulting in ALL text (stdout also) being colorized.

@sickill sickill closed this Apr 28, 2012
@trapd00r

But is this only for echo, or for every builtin command?
I've made some tests here: http://devel.japh.se/stderred/stderr_err.html

This is what it looks like for me:

stderr

@sickill
Owner
sickill commented May 10, 2012

@trapd00r it's for all the built-in commands and external ones that write to stdout (your stream redirection to stderr doesn't count here).

@revintec

I have some thoughts about this:
is it because that the Terminal GUI(whatever is responsible to display the data onto screen) uses the same FD as both stdout and stderr for the child/grandchild they launch.
GUI -> bash(stdout/stderr redirected to same file BEFORE process start) -> some program(inherit stdout/stderr from bash, which are redirected to the same file)

That's why in this case echo >&2 doesn't work.
Like you said, the ACTUAL FILE that receive the data is the same, so you can't distinguish it, and redirection happens AFTER writing, so you lose the first chance to catch the data.

I'm written terminals myself, here is why I thinks we HAVE TO use the same file for both stdout and stderr: if we use the different one, we can't properly display them to the screen.
Long version: say programA writes to stderr and then immediately writes to stdout, from the terminal's point of view, it sees that stdout and stderr both got some data, but it can't know which one is written first, so it can't display it properly(pipes doesn't contains timestamp for data flows, and IO are blocking so you have to use threads, so synchronization problems, and still can't get timestamp, non-blocking IO is the same, no guarantee of FIFO for events), this is THE PROBLEM.

I know how to solve this on windows(which I use before I bought this MACBook Pro), but I'm new to OSX programming, so I'm unable to come with actual OSX code(not yet) even I know how to solve the problem. It involves the following: multi-threading, with blocking IO and message passing, IO redirection.
Basic idea is:
recreate different FDs for stdout/stderr(done AFTER shell is launched) for shell progress, hook all children's write related functions, provide extra info to shell, shell got data, solves THE PROBLEM, do highlighting, then pipe THE ORIGINAL DATA back to the original FD shell have got when it starts.
And this way, solves this problem, any redirection or what, should work fine, and the highlight instructions(ESC[xxm) doesn't pollute the stderr, it's the perfect solution.

Hope to see this feature soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.