Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cquery fails to compile due to poor decisions about finding libclang and ncurses libraries #483

Closed
eklitzke opened this issue Mar 1, 2018 · 5 comments

Comments

@eklitzke
Copy link

commented Mar 1, 2018

I would like to use cquery, but I cannot even compile it due to poor decisions it makes about locating libclang (and ncurses).

When I try to build cquery:

$ ./waf configure build
Setting top to                           : /extra/code/cquery 
Setting out to                           : /extra/code/cquery/build 
Checking for 'clang++' (C++ compiler)    : /usr/lib64/ccache/clang++ 
Checking for header stdio.h              : yes 
Checking for clang
Found extracted at build/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04
Checking for library clang               : not found 
The configuration failed
(complete log in /extra/code/cquery/build/config.log)

OK. So I look at config.log and see this at the end:

Checking for library clang
==>

int main(int argc, char **argv) {
        (void)argc; (void)argv;
        return 0;
}

<==
[1/2] Compiling ^[[32mbuild/.conf_check_3333084be387ac9f9cbc8d37d53cf0ec/test.cpp^[[0m

['/usr/lib64/ccache/clang++', '-g', '-Wall', '-Wno-sign-compare', '-Werror', '-std=c++14', '-O3', '-I../../../release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/include', '-DNDEBUG=1', '-DHAVE_STDIO_H=1', '../../test.cpp', '-c', '-o/extra/code/cquery/build/.conf_check_3333084be387ac9f9cbc8d37d53cf0ec/testbuild/release/test.cpp.1.o']
[2/2] Linking ^[[33mbuild/.conf_check_3333084be387ac9f9cbc8d37d53cf0ec/testbuild/release/testprog^[[0m

['/usr/lib64/ccache/clang++', 'test.cpp.1.o', '-o/extra/code/cquery/build/.conf_check_3333084be387ac9f9cbc8d37d53cf0ec/testbuild/release/testprog', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/extra/code/cquery/build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib', '-lclang', '-rdynamic']
err: /usr/bin/ld: warning: libtinfo.so.5, needed by /extra/code/cquery/build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so, not found (try using -rpath or -rpath-link)
/extra/code/cquery/build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so: undefined reference to `setupterm'
/extra/code/cquery/build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so: undefined reference to `tigetnum'
/extra/code/cquery/build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so: undefined reference to `set_curterm'
/extra/code/cquery/build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so: undefined reference to `del_curterm'
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)

from /extra/code/cquery: Test does not build: Traceback (most recent call last):
  File "/extra/code/cquery/.waf-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d/waflib/Configure.py", line 327, in run_build
    bld.compile()
  File "/extra/code/cquery/.waf-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d/waflib/Build.py", line 176, in compile
    raise Errors.BuildError(self.producer.error)
BuildError: Build failed
 -> task in 'testprog' failed with exit status 1 (run with -v to display more information)

not found
from /extra/code/cquery: The configuration failed

This is extremely broken, in at least two different ways. I'm on Fedora 27. First, I already have libclang a working copy of libclang 5.0.1 installed:

$ rpm -qa clang-libs
clang-libs-5.0.1-3.fc27.x86_64

$ ls -l /usr/lib64/libclang.so*
lrwxrwxrwx. 1 root root     13 Feb  9 23:21 /usr/lib64/libclang.so -> libclang.so.5
lrwxrwxrwx. 1 root root     15 Feb  9 23:21 /usr/lib64/libclang.so.5 -> libclang.so.5.0
-rwxr-xr-x. 1 root root 557880 Feb  9 23:28 /usr/lib64/libclang.so.5.0

So why is cquery trying to use a vendored copy of libclang? I already have a perfectly fine copy installed, and it's just as recent as the bundled version.

The second issue is that you cannot reasonably expect to ship dynamic libraries this way. First I see what the vendored libclang links against:

$ ldd ./build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so
ldd: warning: you do not have execution permission for `./build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so'
	linux-vdso.so.1 (0x00007ffedbf07000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f85d2452000)
	librt.so.1 => /lib64/librt.so.1 (0x00007f85d224a000)
	libtinfo.so.5 => not found
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f85d202b000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f85d1e14000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f85d1abf000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f85d1738000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f85d1521000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f85d113e000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f85d60e0000)

Huh? Why is there a warning from ldd? Well, apparently the .so file is incorrectly set to mode 644:

$ ls -l $(readlink -f ./build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so)
-rw-r--r--. 1 evan evan 66733547 Mar  1 11:34 /extra/code/cquery/build/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5.0

Whatever, that's not really the issue. The issue is that Ubuntu 14 uses libtinfo.so.5 and my system uses a newer version, libtinfo.so.6:

$ ls -l /usr/lib64/libtinfo*
-rw-r--r--. 1 root root 297614 Aug  3  2017 /usr/lib64/libtinfo.a
-rw-r--r--. 1 root root 374654 Aug  3  2017 /usr/lib64/libtinfo_g.a
lrwxrwxrwx. 1 root root     13 Aug  3  2017 /usr/lib64/libtinfo.so -> libtinfo.so.6
lrwxrwxrwx. 1 root root     15 Aug  3  2017 /usr/lib64/libtinfo.so.6 -> libtinfo.so.6.0
-rwxr-xr-x. 1 root root 182152 Aug  3  2017 /usr/lib64/libtinfo.so.6.0

What I would like to see:

  • cquery should at the very minimum check if the system libclang will work before trying the bundled version
  • I would go further and say that using a vendored copy of a .so is broken by design, and should be "opt in" not "opt out"
  • If you actually want to vendor libclang like this, the right way to do it is to have waf build libclang locally, not ship a binary blob.
@MaskRay

This comment has been minimized.

Copy link
Member

commented Mar 1, 2018

Related to Fedora #474

To use a system clang, https://github.com/cquery-project/cquery/wiki/Build#system-clang---llvm-configllvm-config
./waf configure --variant=system --llvm-config=/usr/bin/llvm-config
The aur build script may also be of interested
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=cquery-git

Each platform may have different llvm-config paths (some may have suffixes, some are located in unconventional paths (Nix), Windows does not have llvm-config.exe)

build/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04.tar.xz does depend on libtinfo.so.5 which some distributions do not provide by default (Arch, Fedora, ...). But you may install some compatibility ncurses libraries.

The system libclang.so.5.0.1 SIGSEGV when indexing default arguments of template template parameters. The default --bundled-clang=5.0.1 waf option has a byte patch https://github.com/cquery-project/cquery/blob/master/wscript#L70 http://maskray.me/blog/2017-12-25-cquery-updates-and-libclang-one-byte-patch

libclang cannot be built locally easily, because you would need to download clang+llvm and build thousands of .cpp files. It may likely take more than 3 hours to build on a laptop.

@jacobdufault

This comment has been minimized.

Copy link
Member

commented Mar 1, 2018

cquery should at the very minimum check if the system libclang will work before trying the bundled version

We'd be happy to take a look at a PR.

I would go further and say that using a vendored copy of a .so is broken by design, and should be "opt in" not "opt out"

Certain versions of clang do not work well. Using vendored deps is the most reliable solution for the vast majority of users.

If you actually want to vendor libclang like this, the right way to do it is to have waf build libclang locally, not ship a binary blob.

Takes too long, as @MaskRay said.

The ideal long-term solution is to ship a prebuilt cquery (so you just install the extension in your client of choice and you're done).

@eklitzke

This comment has been minimized.

Copy link
Author

commented Mar 2, 2018

If the ideal solution means catering to only the "vast majority of users" you should also understand that you're alienating users who understand and care about how such software is built. If you want to ship pre-built binaries to the vast majority of users that's fine. But that's not what I asked for: I want to build software the right way.

This is just my personal opinion: Shipping pre-built binaries is antithetical to the open source tradition, and is inherently non-portable. FWIW, rtags is able to detect my system libclang and link against it just fine. I'm familiar enough with autotools and pkg-config but I don't know enough about waf to help with a PR. If you're able to figure this out at some point in the future I'll try cquery again.

@eklitzke eklitzke closed this Mar 2, 2018

@MaskRay

This comment has been minimized.

Copy link
Member

commented Mar 2, 2018

If you consider ./waf configure --llvm-config /usr/bin/llvm-config && ./waf build as unfriendly antithetical open source tradition, I have to say bye.

Yes, waf is alien enough, but it is also supported by FreeBSD ports https://github.com/HenryHu/ports/blob/master/devel/cquery/Makefile

You like the numerous bin/ scripts/ web/ cmake/ directories in the rtags source tree, I don't.

@MaskRay

This comment has been minimized.

Copy link
Member

commented Mar 20, 2018

@eklitzke Sorry for my inconsiderate words "I have to say bye." But I would not be active for this project. If you want to read the long story you may join https://gitter.im/cquery-project/Lobby
quicknir or myrgy in the spacemacs gitter room will be interested to discuss C++ things in Emacs

@DaanDeMeyer has contributed a cmake build system and I hope that might make you feel friendlier.

There are quite a few goodies in https://github.com/cquery-project/emacs-cquery that might attract you. Take a look at https://github.com/cquery-project/cquery/wiki/Emacs , consider contributing these features to rtags or take over emacs-cquery.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.