Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zig c++ does not include stdc++, clang++ does #9832

Closed
folkertdev opened this issue Sep 24, 2021 · 11 comments
Closed

zig c++ does not include stdc++, clang++ does #9832

folkertdev opened this issue Sep 24, 2021 · 11 comments
Milestone

Comments

@folkertdev
Copy link

I hit a case where zig c++ is not a drop-in replacement for a clang that uses the equivalent llvm version (llvm 12, zig 0.8.0, clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-20.04)

This command works

clang++ -v main.cc.o  -o onnx2c   libonnx2c_lib.a /usr/local/lib/libprotobuf.so 

while this does not

zig c++ -v main.cc.o  -o onnx2c   libonnx2c_lib.a /usr/local/lib/libprotobuf.so 

The zig verson reports missing c++ stdlib symbols, e.g.

ld.lld: error: undefined symbol: std::ostream::put(char)
>>> referenced by graph.cc
>>>               graph.cc.o:(toC::LSTM::print_activation(std::ostream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const) in archive libonnx2c_lib.a
>>> referenced by graph.cc
>>>               graph.cc.o:(toC::LSTM::print_lstm_kernel(std::ostream&, bool) const) in archive libonnx2c_lib.a
>>> referenced by graph.cc
>>>               graph.cc.o:(toC::LSTM::print_lstm_kernel(std::ostream&, bool) const) in archive libonnx2c_lib.a
>>> referenced 89 more times

Here are the full linker invocations:

Clang:

"/usr/bin/ld" --hash-style=both --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o onnx2c /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/10/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/10 -L/usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/10/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/lib/x86_64-linux-gnu/../../lib64 -L/usr/lib/gcc/x86_64-linux-gnu/10/../../.. -L/home/folkertdev/Downloads/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-20.04/bin/../lib -L/lib -L/usr/lib main.cc.o libonnx2c_lib.a /usr/local/lib/libprotobuf.so -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-linux-gnu/10/crtend.o /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/crtn.o

Zig c++

ld.lld -error-limit=0 -z stack-size=16777216 --gc-sections --eh-frame-hdr -m elf_x86_64 -o /home/folkertdev/.cache/zig/o/10863395297406fc6a07a3d66e5dae94/onnx2c /home/folkertdev/.cache/zig/o/d72a08251764ca3b50ad6f9c2178502c/Scrt1.o /home/folkertdev/.cache/zig/o/fd2f7ed75d6a8bb131dbeb450b22976c/crti.o -L /usr/local/lib64 -L /usr/local/lib -L /usr/lib/x86_64-linux-gnu -L /lib64 -L /lib -L /usr/lib64 -L /usr/lib -L /lib/x86_64-linux-gnu -dynamic-linker /lib64/ld-linux-x86-64.so.2 main.cc.o libonnx2c_lib.a /usr/local/lib/libprotobuf.so /home/folkertdev/.cache/zig/o/8d7f892320208e3eb85beca5cbadba3b/libcompiler_rt.a /home/folkertdev/.cache/zig/o/c1bb35f3336ba0415196640af079cbb6/libc++abi.a /home/folkertdev/.cache/zig/o/498c83bc6b6945fe94a3be48eb4ce9e0/libc++.a /home/folkertdev/.cache/zig/o/668363e59260ab7aeb85271c2fd416e5/libunwind.a /home/folkertdev/.cache/zig/o/68130f6cc2832b2537f2ccc0002f97c2/libm.so.6 /home/folkertdev/.cache/zig/o/68130f6cc2832b2537f2ccc0002f97c2/libpthread.so.0 /home/folkertdev/.cache/zig/o/68130f6cc2832b2537f2ccc0002f97c2/libc.so.6 /home/folkertdev/.cache/zig/o/68130f6cc2832b2537f2ccc0002f97c2/libdl.so.2 /home/folkertdev/.cache/zig/o/68130f6cc2832b2537f2ccc0002f97c2/librt.so.1 /home/folkertdev/.cache/zig/o/68130f6cc2832b2537f2ccc0002f97c2/libld.so.2 /home/folkertdev/.cache/zig/o/68130f6cc2832b2537f2ccc0002f97c2/libutil.so.1 /home/folkertdev/.cache/zig/o/b61066544a773e7c5cad2d3678dfea62/libc_nonshared.a /home/folkertdev/.cache/zig/o/3b50ad42196c5bd14aad1360daafeb7d/crtn.o

The difference that makes a difference is these arguments that clang does give to the linker, and zig does not:

    -L/usr/lib/gcc/x86_64-linux-gnu/10\
    -lstdc++\

The zig linker invocation does include

/home/folkertdev/.cache/zig/o/c1bb35f3336ba0415196640af079cbb6/libc++abi.a\
/home/folkertdev/.cache/zig/o/498c83bc6b6945fe94a3be48eb4ce9e0/libc++.a\

But that is apparently not enough? Also I tried

zig c++ -L/usr/lib/gcc/x86_64-linux-gnu/10 -v main.cc.o -o onnx2c libonnx2c_lib.a /usr/local/lib/libprotobuf.so -lstdc++

(include the gnu path and the stdc++ flag) in the zig c++ command, but the -lstdc++ flag is not present in the linker invocation. So

  • is this a bug, or am I missing some c++ detail here?
  • is there a way to force -lstdc++ to make it to the linker, either with zig c++ or zig build-exe
@mikdusan
Copy link
Member

Which object contains graph.cc.o ? I'm going to guess: libprotobuf.so and it has a dependency on libstdc++.so. To verify this dep, ldd /usr/local/lib/libprotobuf.so.

No single executable/binary should use 2 different c++ libraries.

The solve to this is to build protobuf (and any of its dependencies that are c++ releated) with zig c++.

@folkertdev
Copy link
Author

no, graph.cc is contained in libonnx2c_lib.a. That .a is constructed completely by zig. But if I understand you correctly building protobuf with zig c++ would still solve the issue?

@mikdusan
Copy link
Member

mikdusan commented Sep 24, 2021

protobuf might be another issue here but let's push that aside for now. How is libonnx2c_lib.a built? Can you show the command-line for building some .o for it?

@folkertdev
Copy link
Author

Sure, here is what currently works

zig build-lib src/graph.cc src/graph_print.cc src/node.cc src/onnx.pb.cc src/tensor.cc src/util.cc -lc++  -I/home/folkertdev/tg/scailable/sclbl-onnx-to-c/build -isystem /home/folkertdev/tg/scailable/sclbl-onnx-to-c/./src -I/home/folkertdev/tg/scailable/sclbl-onnx-to-c/src -I/home/folkertdev/tg/scailable/sclbl-onnx-to-c/src/utils -I/usr/local/include -I/home/folkertdev/tg/scailable/sclbl-onnx-to-c/aixlog/include -cflags -std=c++11 -O3 -Wall -Wl, -Wno-misleading-indentation -Wno-error=unused-variable     -- -femit-bin=all.cc.o

zig ar qc libonnx2c_lib.a all.cc.o 

zig ranlib libonnx2c_lib.a

clang++ -I/home/folkertdev/tg/scailable/sclbl-onnx-to-c/build -isystem /home/folkertdev/tg/scailable/sclbl-onnx-to-c/./src  -O3 -Wall -Wno-misleading-indentation -Wno-error=unused-variable    -I/home/folkertdev/tg/scailable/sclbl-onnx-to-c/src -I/home/folkertdev/tg/scailable/sclbl-onnx-to-c/src/utils -I/home/folkertdev/tg/scailable/sclbl-onnx-to-c/aixlog/include -Wall -Wno-unused-variable -std=c++11 -o main.cc.o -c src/main.cc

clang++ -v main.cc.o  -o onnx2c   libonnx2c_lib.a /usr/local/lib/libprotobuf.so 

Now with your information, I tried switching both of those clang++ to zig c++ (had only tried them independently so far). That seems to make some progress, but ultimately hits a bunch of errors like this one:

ld.lld: error: duplicate symbol: toC::Node::onnx_ir_version
>>> defined at node.cc
>>>            node.cc.o:(toC::Node::onnx_ir_version) in archive libonnx2c_lib.a
>>> defined at node.cc
>>>            node.cc.o:(.bss+0x0) in archive libonnx2c_lib.a

@mikdusan
Copy link
Member

re: zig build-lib ... can you check if any of the -I or -isystem args supply a directory which may supply c++ include files. Looking to rule out conflict. _ZIG_/lib/libcxx/include is where the proper files are found.

Also this is a bit tricky, would like to see the include dir search list used for .cpp → .o

  1. remove local zig-cache
  2. reduce the zig build-lib line to only use src/graph.cc and remove the remaining .cc files from command and add --verbose-cc
  3. cut-n-paste the huge command line for .cpp → .o, which should be something like zig clang ... and append to that command-line -v

so something like this:

rm -fr zig-cache
zig build-lib src/graph.cc -lc++ ..... --verbose-cc
zig clang ..... -v

which should yield something like this:

clang -cc1 version 12.0.1 based upon LLVM 12.0.1 default target x86_64-unknown-linux-gnu
#include "..." search starts here:
#include <...> search starts here:
 /home/mike/project/zig/work/main/lib/libcxx/include
 /home/mike/project/zig/work/main/lib/libcxxabi/include
 /home/mike/project/zig/work/main/lib/libunwind/include
 /home/mike/project/zig/work/main/lib/include
 /home/mike/project/zig/work/main/lib/libc/include/x86_64-linux-gnu
 /home/mike/project/zig/work/main/lib/libc/include/generic-glibc
 /home/mike/project/zig/work/main/lib/libc/include/x86_64-linux-any
 /home/mike/project/zig/work/main/lib/libc/include/any-linux-any
End of search list.

@andrewrk
Copy link
Member

andrewrk commented Sep 24, 2021

C++ pretends to have an ABI, but it's a lie. That's the crucial problem here.

Zig provides libc++, however the libc++ that zig provides cannot be mixed with any C++ objects compiled from other compilers. If you do, you get undefined behavior. So you can't use zig c++ on libonnx2c_lib.a or /usr/local/lib/libprotobuf.so because these C++ objects were compiled with something other than zig c++.

I'm upset that C++ works this way but there is nothing we can do about it.

zig c++ is a drop-in C++ compiler if you use it for all your C++ objects; if not, it does not help you.

@folkertdev
Copy link
Author

To clarify, even combining output of zig build-obj and zig c++ would run into trouble?

@andrewrk
Copy link
Member

If the input file were C++ code, yes. You can learn about more intricacies of C++ to find out what versions of compilers do or don't work together, or what features of the language are affected by this ABI mismatch. But the simplest way to remember it is that you have to use the same C++ compiler for all C++ code.

@andrewrk
Copy link
Member

The upshot here is that if you have libonnx2c_lib.a or /usr/local/lib/libprotobuf.so on your system then these were presumably built by the system C++ compiler, so you should be able to rely on that to build your other C++ objects.

@folkertdev
Copy link
Author

yes, the system c++ compiler works fine. I just would have liked to terminate a CMakeLists.txt and use build.zig instead, and that now seems like it won't work out

@andrewrk
Copy link
Member

If we want to support that use case, it will require zig build to additionally support being an external compiler driver. That's something that may very well be in scope of the project, but it is not currently implemented.

@andrewrk andrewrk added this to the 0.9.0 milestone Nov 20, 2021
Javyre added a commit to Javyre/wf-lua that referenced this issue Nov 22, 2021
Want to reduce my c++ usage in order to preserve my mental health :P

This should build fine anywhere zig is present. clang++ is still a
dependency since the zig c++ compiler differs in ABI to clang/gcc.
See: ziglang/zig#9832
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants