stderr/cerr inconsistency between clang and clang++ -std=c++11

[llvmbot]

Bugzilla Link	11373
Resolution	FIXED
Resolved on	May 12, 2014 00:41
Version	trunk
OS	Linux
Reporter	LLVM Bugzilla Contributor
CC	@DougGregor,@efriedma-quic

Extended Description

Hello. I am not sure what component this goes under, but here's the problem: under Linux, clang++ prematurely terminates before cerr has a chance to write, though this does not seem to happen with clang. Below are two examples, the C version that works and C++ analog that does not. Note that I have successfully compiled llvm/clang/clang++/libc++ on my SuSE Linux machine.

clang++ version:

/tools/llvm/bin/clang++ --version
clang version 3.1 (trunk 144502)
Target: x86_64-unknown-linux-gnu
Thread model: posix

clang/C program that works:

cat t4.c
#include <stdio.h>
#include <assert.h>
int main(int argc, char* argv[])
{
fprintf(stderr, "Line 1 testing cerr\n");
fprintf(stderr, "Line 2 testing cerr\n");
assert(1 == 0);
return 0;
}
/tools/llvm/bin/clang t4.c -o t4
./t4
Line 1 testing cerr
Line 2 testing cerr
t4: t4.c:7: int main(int, char **): Assertion `1 == 0' failed.
Aborted

clang++/C++11 program that does not work:

cat t3.cpp
#include
#include
using namespace std;
int main(int argc, char* argv[])
{
cerr << "Line 1 testing cerr" << endl;
cerr << "Line 2 testing cerr" << endl;
assert(true == false);
return 0;
}
/tools/llvm/bin/clang++ -std=c++11 -stdlib=libc++ t3.cpp -o t3
./t3
Line 1 testing cerrAborted

Note how Line 2 never gets printed. My understanding what that cerr is not buffered so it should print both lines before the assertion terminates the program, right?

I would be grateful for any information that would resolve this problem. Many thanks!

[llvmbot]

I would like to just add that this particular bug does not show up under Mac OS X Lion's Xcode's clang++ compiler.

[efriedma-quic]

A couple more datapoints would be useful: is this specific to C++11 mode, or does it also show up in C++98 mode? What is the result of using libstdc++ with clang?

[llvmbot]

A couple more datapoints would be useful: is this specific to C++11 mode, or
does it also show up in C++98 mode? What is the result of using libstdc++ with
clang?

Yes, if one removes the -std=c++11 argument, the behavior is still the same. I am not sure what you mean about using libstdc++ with clang, since it is a C++ library. If you mean doing something like

/tools/llvm/bin/clang t4.c -stdlib=libc++ -o t4
clang: warning: argument unused during compilation: '-stdlib=libc++'

then ./t4 still works fine like the original compilation of t4.c.

[efriedma-quic]

I mean, does just "clang++ t3.cpp" (without specifying -stdlib) do what you expect?

[llvmbot]

I mean, does just "clang++ t3.cpp" (without specifying -stdlib) do what you
expect?

Hello. Sorry, I am afraid it still does the same [wrong] thing, i.e.

cat t3.cpp
#include
#include
using namespace std;
int main(int argc, char* argv[])
{
cerr << "Line 1 testing cerr" << endl;
cerr << "Line 2 testing cerr" << endl;
assert(true == false);
return 0;
}
clang++ t3.cpp -o t3
./t3
Line 1 testing cerrAborted

[efriedma-quic]

Hmm... what happens with g++?

[llvmbot]

Hmm... what happens with g++?

It works. I.e.

g++ --version
g++ (GCC) 4.7.0 20111114 (experimental)
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

g++ t3.cpp -o t3
./t3
Line 1 testing cerr
Line 2 testing cerr
t3: t3.cpp:8: int main(int, char**): Assertion `true == false' failed.
Aborted

[llvmbot]

Try recompiling libc++ with a printf statement in iostream.cpp, ios_base::Init::Init(). It could be that this initialization function is never being run. And it is this initialization that sets the unitbuf flag for cerr (making it unbuffered).

[llvmbot]

Try recompiling libc++ with a printf statement in iostream.cpp,
ios_base::Init::Init(). It could be that this initialization function is never
being run. And it is this initialization that sets the unitbuf flag for cerr
(making it unbuffered).

Hello. I modified iostream.cpp:

...
_VSTD::unitbuf(cerr);
cerr.tie(&cout);

fprintf(stderr, "ios_base::Init::Init called\n");

wcin.tie(&wcout);

...

and then compiled libc++ and then compiled t3.cpp:

clang++ -std=c++11 -stdlib=libc++ t3.cpp -o t3
./t3
ios_base::Init::Init called
Line 1 testing cerrAborted

so it looks like Init() is being called. Strange. Perhaps something is amiss with _VSTD::unitbuf(cerr)?

[llvmbot]

_VSTD::unitbuf(cerr) is just setting a bit in a word, and I think a bug there would impact all platforms equally. Plus the code is just dirt simple. You can find it in . I'm now suspecting something in <__std_stream>, __stdoutbuf<_CharT>::overflow(int_type __c). I expect _always_noconv to always be true in this example, and thus fwrite should always be called. Could it be that fwrite is executing asynchronously on Linux?

[llvmbot]

_VSTD::unitbuf(cerr) is just setting a bit in a word, and I think a bug there
would impact all platforms equally. Plus the code is just dirt simple. You
can find it in . I'm now suspecting something in <__std_stream>,
__stdoutbuf<_CharT>::overflow(int_type __c). I expect _always_noconv to
always be true in this example, and thus fwrite should always be called. Could
it be that fwrite is executing asynchronously on Linux?

Yes, __always_noconv is true. If I add a fprintf(stderr, ...) statement, I get the output:

ohsumit@mothra:~> ./t3
always noconv true
Lalways noconv true
ialways noconv true
nalways noconv true
ealways noconv true
always noconv true
1always noconv true
always noconv true
talways noconv true
ealways noconv true
salways noconv true
talways noconv true
ialways noconv true
nalways noconv true
galways noconv true
always noconv true
calways noconv true
ealways noconv true
ralways noconv true

I also put in a fprintf statement in the false section, but that part is never executed. Note that fwrite is buffered in Linux. It looks like this is not called for all the characters that are to be printed out.

[llvmbot]

One additional note: I am a little puzzled why GNU g++ works on this program while clang++ does not. Is there something I am missing? Maybe how I compiled clang/llvm? Many thanks

[llvmbot]

I'm not able to replicate the symptom as I do not have access to a Linux machine. Thus I will be unable to track down the cause of the symptoms you're seeing. I recommend investigating the member functions of __stdoutbuf using printf statements. You might also vary your test function, such as using '\n' instead of endl, and see how that impacts the output.

[llvmbot]

I'm not able to replicate the symptom as I do not have access to a Linux
machine. Thus I will be unable to track down the cause of the symptoms you're
seeing. I recommend investigating the member functions of __stdoutbuf using
printf statements. You might also vary your test function, such as using '\n'
instead of endl, and see how that impacts the output.

Hello. Interestingly, if I replace endl by "\n", I get

./t3
Line 1 testing cerr
Aborted

whereas before recall I had

Line 1 testing cerrAborted

Is there something different about how endl is processed instead of \n? Also, if I change the program to:

for (size_t i = 1; i <= 100; ++i)
    cerr << "Line " << i << " testing cerr" << endl;
assert(true == false);

it outputs

Line Aborted

while if I replace the cerr with cout, it prints all 100 lines properly before exiting. It seems like __stdoutbuf doesn't have any dependencies on a particular stream, while it looks like the code in iostream.cpp does set up the characteristics of cerr vs cout. Any thoughts would be most appreciated. Many thanks.

[llvmbot]

I think I figured out the key cause: if I comment out in iostream.cpp the lines with _VSTD::unitbuf, then everything works properly. The big question is why. I am not totally sure what's going on here, I am afraid. It seems like unitbuf may be setting something incorrectly under Linux?