Is it possible to leave stderr messages from the compiler on stderr and not redirect them to stdout?

[tpimh]

I understand that default ninja behavior is to only print to stdout, but I wonder if it's possible to implement this as an option like done in this branch: https://github.com/dendy/ninja/tree/stderr

[jhasse]

What do you need it for?

[tpimh]

I would like to separate build output into too files: build.log for informational output and build.err for warnings and errors. Redirecting stdout to one file and stderr to another works for GNU Make, but not for Ninja.

[jhasse]

You won't know which warning or error belongs to which build output line. What's the reason behind the two files?

[tpimh]

Mostly just warning detection. If there are warnings, they should be either fixed or disabled. The warning usually contains source filename and line number, so there is no need to find corresponding lines of ninja's own output.

[jhasse]

I see. Would #480 be enough for your use-case?

The reason I'm asking is: We're trying to keep the number of options as low as possible.

[tpimh]

I'm not sure how that could possibly be the solution for my problem. Probably there may be a very hacky solution with NINJA_STATUS variable, but I can't think of one right now.
On the other hand if split stderr is implemented, --quiet that disables all ninja's own output and only prints compiler output would be as easy as ninja --split-stderr >/dev/null.

[jhasse]

I'm not sure how that could possibly be the solution for my problem.

You would run ninja --quiet and could detect all the warnings. (I'm aware that --quiet isn't implemented yet)

[tpimh]

But in this case I will not get any ninja's own output. I would like to keep both, but split them.

[jhasse]

Why?

[tpimh]

In current build script I print a dot for every line on stdout and an x for every line on stderr. Saving both stdout and stderr into separate files. This keeps the CI from stopping because of the lack of output, gives a pretty visual output that helps to detect new warnings and errors and provides separate readable build logs with warning descriptions. This works for other build systems, but not for ninja as there is no way to tell if a line came from stderr or stdout. This is how I did it: tpimh/nenuzhnix-build@722ece3 (still to be rewritten properly)

[jhasse]

Set NINJA_STATUS="[ninja][%f/%t] " and then you grep for "^[ninja]" in process_out to see if it's ninja output or compiler output.

[tpimh]

Yes, can be done this way. Thanks!

[tpimh]

This is my script for restoring justice over stdout and stderr:

filter() {
  local l n
  while read -r l; do
    n=${l#\[ninja\]}
    [ "x$l" != "x$n" ] && printf "%s\n" "$n" || printf "%s\n" "$l" >&2
  done
}

NINJA_STATUS="[ninja][%f/%t] " ninja | filter

It may be a bit slow, but it works for me. Feel free to use and improve.

UPD: replaced my obviously shitty code with a much better version by Vladimir Oleynik dzo@simtreas.ru

[sjg20]

For me the way that ninja ignores sfderr causes 'kate' to ignore all warnings/errors, which means that the build plugin doesn't work.

[subdiff]

Same for QtCreator.

Just a simple script to wrap ninja with stdout > stderr is enough. But it would be nice if it would work out of the box.

[kloczek]

Can you share what and how you are actually using as the wrpaper?

[tpimh]

I have my own ninja wrapper. It's really bad, but it works. Written in POSIX shell, could be much easier in bash, but it's unfortunately not an option for me.

[subdiff]

$ cat /usr/bin/ninja-stderr

#!/bin/sh
/usr/bin/ninja "$@" 1>&2

[sjg20]

Is there any chance this might be 'fixed' in ninja itself? If ninja considers all of its output to be errors, then perhaps it should just move to stderr. If it considers some of it to be informational, not indicating warnings/failures, then that part should go to stdout IMO.

[jhasse]

@sjg20 #1210 would allow this to be implemented in a cleaner way.

[dendy]

Hello, I am the author of the patch for this issue mentioned in the first comment: https://github.com/dendy/ninja/tree/stderr

Patch is currently updated with Linux/Mac/Windows support and rebased onto master branch.

I consider this issue as critical. Without separation of the stdout and stderr usage of the ninja is limited mostly to the CLI. IDEs and tools which expect the proper separation of output, like it was in the Makefiles, will not work with ninja at all.

Here is the thing. Decision where to put an output should be made by the executing command, not the ninja itself. If some command decides to put portion of output to the stderr, that should be possible and properly pop up at the ninja's end in the stderr. Generally speaking stdout and stderr supposed to be completely independent, and it is up to the command itself to ensure that both streams are contiguous. And whoever is parsing the output should expect that stdout and stderr are not connected together.

Now, I understand why ninja is combining both outputs, to keep them in sync with printing the execution command, so it is easier to see whole output in the solid chunk. Although such approach has it is own benefits in some cases, it is not the only one that is used in the build environments.

Particularly using ninja from IDE requires to parse stderr. Not combined stdout and stderr, but stderr alone. Each line printed by the executing command into stderr is considered as a separate issue. The executing command arguments usually don't matter in the IDE context, and even if they do there are other ways to connect errors and command line, rather than eliminating the stderr completely. In worst case it will be the same experience as with Makefiles.

I strongly believe that this feature is absolutely necessary. It should be up to the user, which method is preferable in particular case. It could be environment variable, command line argument, or some extra parameter in the build.ninja, or combination of those. But user should have control over this, whether output of the stdout and stderr should be merged together or not.

For the 5 good years I never use combined output in any of my projects, either they are built from the command line or from IDE. And I am totally happy with the results.

[jhasse]

If I understood you correctly, stdout and stderr are just synchronized in the terminal by chance and the order may actually be totally random if you're unlucky. If this is true, people (like me) are using stderr wrong. Its usage should be discouraged as it's too easy to shoot yourself in the food. It isn't intuitive (i.e. the terminal mixes them by default). If someone wants "error" output separately he should explicitly write it to a file.

[sjg20]

If I understood you correctly, stdout and stderr are just synchronized in the terminal by chance and the order may actually be totally random if you're unlucky. If this is true, people (like me) are using stderr wrong. Its usage should be discouraged as it's too easy to shoot yourself in the food. It isn't intuitive (i.e. the terminal mixes them by default). If someone wants "error" output separately he should explicitly write it to a file.

Excuse me but I think this is misconceived. The stdout output is not all that useful, to put it mildly. in fact I suppress it. It spits out a lot of stuff that indicates normal operation. For development and IDEs we want to see warnings and errors, nothing else. At present ninja provides no way to do this. The combining of stdout and stderr is strange, perhaps a Windows thing, but at the least ninja should provide a way to avoid this non-standard and (in my opinion) pointless output.

In general builds are mutil-threaded and the only thing matters is the filename, line number and message so I can work through those to fix the build. Even the compilation order is not that interesting.

So I propose changing the default to separate stdout and stderr. IMO that would be much more sensible, following make, gcc and the like. If that is not acceptable, and we are happy to have this tool be different from everything else I have seen, then dendy@'s proposal looks good to me :-)

In any case, please can we skip the circular discussion and implement something to fix this problem. We are currently using Zephyr with Cmake and engineers are asking about changing over to make it generate a Makefile instead...

[jhasse]

At present ninja provides no way to do this.

Ninja might add --quiet in a future update so that you can hide status updates. Currently you can simulate that using NINJA_STATUS and the scripts above.

[dendy]

There are two types of terminals:

1. When terminal does not care about separating appearance of the stdout/stderr streams, so it combines them together by redirecting both streams into the same pipe. These are common console terminals. Same behavior when redirecting output with 2>&1. When write() syscall is used then output might appear in order. But any buffering on the writer side will cause output to be scrambled between stdout and stderr. You can never rely on what you see in the terminal as well as what you get in the 2>&1.
2. When terminal does care about stdout and stderr separation, for instance, to colorize the errors with red. Or hide/show one of the streams. In this case output will not be scrambled. All IDEs rely on this behavior to properly process command results.

Its usage should be discouraged as it's too easy to shoot yourself in the foot

It is all fine with stderr stream as long as command which is writing into it is following the core logic, why those streams were invented years ago in first place: they are independent. There is no synchronization between them by design, nor any stream reader should rely on data order between streams.

Ninja cannot dictate policy to the executing commands how they should use stdout/stderr, nor to the caller how they should process output from the commands. Because nor of those is under control of the ninja, all those tools already exist.

If someone wants "error" output separately he should explicitly write it to a file.

If someone wants stderr/stdout contents separately they should simply read appropriate stream. It is how all tools behave and there is nothing wrong with that.

Don't want to touch anyone's feelings, but no matter how I like the ideas implemented by ninja, for me personally ninja was totally useless until I implemented patch for stdout/stderr separation several years ago.

[sjg20]

+1 we have a hundred lines of Python code trying to clean up the ninja output and it is still not right. It messes up downstream things also - e.g. Zephyr is quite broken in terms of what is a warning/error and what is normal behaviour.

I wonder if the problem is that this tool came from Windows?

Do you have a link to the stdout/stderr patch? We could perhaps just use that.

[dendy]

@sjg20 You are totally right, this issue should be fixed at the Ninja level. Generally there is no way to process Ninja output in attempt to differentiate between stdout/stderr after outputs were merged.

I believe the problem came out because of misunderstanding what outputs really are, and how contract between process output writers are readers should be. Probably in an attempt to solve the messed output because of the single pipe shared across multiple processes, this principle was explicitly highlighted in the Ninja documentation. But it has nothing to do with this issue.

Link to the patch is in the initial comment to this ticket, let me know it works for you.

[sjg20]

@dendy OK thank you. I filed https://issuetracker.google.com/187728036 so we'll see what we end up doing. Someone also suggesting switching Cmake to generate plain Makefiles.

[CarloWood]

This is a critical problem for me too - since @jhasse seems to refuse to add a simple option for it - where is the ninja fork that DOES implement this currently? Thanks.

[dendy]

@CarloWood https://github.com/dendy/ninja/tree/stderr

[orgads]

@jhasse My proposal is to enable this feature in experimental mode. I am quite happy with the current patch (see link at the top).

With some caveats:

1. Default behavior should stay as in current implementation, to combine stdout and stderr.
2. Add environment variable to override the default behavior, e.g.: NINJA_USE_STDERR=1. This is needed in case it is convenient for the user to set behavior globally.
3. Add command line argument to also override the default behavior, which will have more priority than environment variable, e.g.: ninja --use-stderr=1. For instance, this is needed to use this feature in the IDE, without changing behavior globally.
4. (optionally) Add new binding, e.g. "use_stderr". This will allow to enable feature per individual command. Could be useful for generators.

Advertise this update in the new ninja version and collect feedback from the users about issues.

@jhasse can you please explain why do you object to having this as opt-in, as suggested here? This is a real pain to many users.

[jhasse]

I'm no longer maintaining Ninja.

[jhasse]

misclicked, sorry

[orgads]

I'm no longer maintaining Ninja.

Then who can approve this?

[kloczek]

Any update? 🤔

[dendy]

@jhasse Do you know who is the current maintainer of the ninja? We are still looking to accept this feature as optional, without changing the default behavior. Who could help to proceed with this?

[jhasse]

No one really.

Ninja cannot dictate policy to the executing commands how they should use stdout/stderr, nor to the caller how they should process output from the commands.

Well, it can at least discourage the use of stderr by not providing an option to split stdout/stderr.

[kloczek]

Combining stdout with stderr by default and lack of ability to not combine those two streams is really big problem on automatically analysing warnings and errors 😞
IMO current behaviour should stay as default but it should be provided command line option to keep separated stdout and stderr.

[sjg20]

+1 we need a proper stderr. This is such a pain. Why is this so controversial??

[dendy]

@jhasse Ninja is only executing other tools, and behavior of those tools how to produce data for stdout and stderr already well defined. There are billions of those tools, Ninja may not change they behavior.

In the case when parser of the Ninja output wants proper separation of stderr — there should be such an option.

My proposal above is to add this feature. And keep original behavior (combined output) as default. So everybody will win and be happy.

Have to rebase my patch onto recent master branch. Though first I will need your agreement in general to accept this feature.

[kloczek]

Gentle ping .. any update? 🤔

[tpimh]

@kloczek just use Daniel's fork, @jhasse is never going to merge this.

[kloczek]

@kloczek just use Daniel's fork, @jhasse is never going to merge this.

OK so where is that fork and/or where I can find exact patch/commit? 😋

[kloczek]

https://github.com/jhasse/ninja seems is no longer maintained 🤔

[dendy]

@kloczek Look at the veeeery top message, you will find a link to my branch: https://github.com/dendy/ninja/tree/stderr
I will check if it needs to be updated by rebasing it onto the latest ninja mainline.

[kloczek]

@kloczek Look at the veeeery top message, you will find a link to my branch: https://github.com/dendy/ninja/tree/stderr I will check if it needs to be updated by rebasing it onto the latest ninja mainline.

Can you resync your fork with ninja master? 🤔
Currently that commit is not possible to apply on top of last version

+ /usr/lib/rpm/rpmuncompress /home/tkloczko/rpmbuild/SOURCES/ninja-build-Add-option-to-split-stdout-and-stderr.patch
6 out of 8 hunks FAILED -- saving rejects to file src/build.cc.rej
2 out of 3 hunks FAILED -- saving rejects to file src/build.h.rej
1 out of 11 hunks FAILED -- saving rejects to file src/subprocess-posix.cc.rej