Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static compilation? #9

Open
BeeeWall opened this issue Nov 7, 2017 · 27 comments
Open

Static compilation? #9

BeeeWall opened this issue Nov 7, 2017 · 27 comments

Comments

@BeeeWall
Copy link

BeeeWall commented Nov 7, 2017

I want to be able to compile this statically, so I can include it in an app on Android. However, when I try to do so, a bunch of files give the error

/usr/bin/ld: /usr/lib/arm-linux-gnueabihf/libc.a(libc-start.o): relocation R_ARM_THM_MOVW_ABS_NC against `_dl_starting_up' can not be used when making a shared object; recompile with -fPIC            /usr/lib/arm-linux-gnueabihf/libc.a: error adding symbols: Bad value

Googling the issue tells me that it is trying to compile dynamically, despite me specifically saying to compile statically only (the full configure was

./configure --enable-static --disable-shared CPPFLAGS="-fPIC -fPIE -static" LDFLAGS="-static /usr/lib/arm-linux-gnueabihf/libz.a /usr/lib/arm-linux-gnueabihf/libxml2.a /usr/lib/gcc/arm-linux-gnueabihf/6/libstdc++.a /usr/lib/arm-linux-gnueabihf/libm.a /usr/lib/gcc/arm-linux-gnueabihf/6/libgcc.a /usr/lib/arm-linux-gnueabihf/libc.a /usr/lib/arm-linux-gnueabihf/libdl.a /usr/lib/arm-linux-gnueabihf/libicui18n.a /usr/lib/arm-linux-gnueabihf/libicuuc.a /usr/lib/arm-linux-gnueabihf/libicudata.a /usr/lib/arm-linux-gnueabihf/liblzma.a /usr/lib/arm-linux-gnueabihf/libpthread.a"

). Is something wrong with my configure, does it just not build statically, or what? I am building on an Android tablet in Termux, using Debian For Termux.

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 15, 2017

Hmm, I'm not sure. Statically compiling xoreos-tools used to work, but I haven't done it for a while now (and I don't have static versions of all the libraries at hand at the moment).

Could you post your config.log and a full log of a compile in verbose mode (i.e. with make V=1)?

(Also, you might want to try to add --enable-static-boost to the configure call, in case boost.m4 picks up dynamic versions?)

@BeeeWall
Copy link
Author

config.log (hastebin)
make.log (hastebin)

Configure slightly modified since first post:
./configure --enable-static --disable-shared CPPFLAGS=-fPIC -fPIE -static LDFLAGS=-static /usr/lib/arm-linux-gnueabihf/libz.a /usr/lib/arm-linux-gnueabihf/libxml2.a /usr/lib/gcc/arm-linux-gnueabihf/6/libstdc++.a /usr/lib/arm-linux-gnueabihf/libm.a /usr/lib/gcc/arm-linux-gnueabihf/6/libgcc.a /usr/lib/arm-linux-gnueabihf/libc.a /usr/lib/arm-linux-gnueabihf/libdl.a /usr/lib/arm-linux-gnueabihf/libicui18n.a /usr/lib/arm-linux-gnueabihf/libicuuc.a /usr/lib/arm-linux-gnueabihf/libicudata.a /usr/lib/arm-linux-gnueabihf/liblzma.a /usr/lib/arm-linux-gnueabihf/libpthread.a --disable-rpath --with-pic --disable-silent-rules

@BeeeWall
Copy link
Author

Adding --enable-static-boost tells me it is an unrecognized option.

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 16, 2017

Errm, you're compiling the xoreos-tools 0.0.4 release package, it seems? Could you compile the current git version instead? There's been quite some changes since then, including the build system (and the dependency on boost wasn't added until after 0.0.4 too).

@BeeeWall
Copy link
Author

BeeeWall commented Nov 16, 2017

How do I build the git version? There's no configure file.

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 16, 2017

https://wiki.xoreos.org/index.php?title=Git
https://wiki.xoreos.org/index.php?title=Compiling_xoreos-tools

In short:

  • To get the sources: git clone git://github.com/xoreos/xoreos-tools.git
  • Then you can either use the autotools build system (i.) or CMake (ii.)
    1. auttools:
      1. ./autogen.sh to create the configure scripts
      1. ./configure (with options, optionally from a different directory for out-of-tree builds)
      1. make
    1. CMake:
      1. cmake . (with options, optionally from a different directory for out-of-tree builds)
      1. make

Basically, to get from the git checkout to the state of the release tarball, you only need to execute the autogen.sh script.

@BeeeWall
Copy link
Author

I'm now trying to static compile with
./configure --enable-static --disable-shared --disable-rpath --with-pic --disable-silent-rules --enable-static-boost \ CPPFLAGS="-fPIC -fPIE" \ LDFLAGS="-static \ /usr/lib/arm-linux-gnueabihf/libz.a \ /usr/lib/arm-linux-gnueabihf/libxml2.a \ /usr/lib/arm-linux-gnueabihf/libboost_filesystem.a \ /usr/lib/arm-linux-gnueabihf/libboost_system.a \ /usr/lib/arm-linux-gnueabihf/libboost_regex.a \ /usr/lib/arm-linux-gnueabihf/libboost_atomic.a \ /usr/lib/arm-linux-gnueabihf/libboost_locale.a \ /usr/lib/gcc/arm-linux-gnueabihf/6/libstdc++.a \ /usr/lib/arm-linux-gnueabihf/libm.a \ /usr/lib/gcc/arm-linux-gnueabihf/6/libgcc.a \ /usr/lib/arm-linux-gnueabihf/libc.a \ /usr/lib/arm-linux-gnueabihf/libdl.a \ /usr/lib/arm-linux-gnueabihf/libicui18n.a \ /usr/lib/arm-linux-gnueabihf/libicuuc.a \ /usr/lib/arm-linux-gnueabihf/libicudata.a \ /usr/lib/arm-linux-gnueabihf/liblzma.a \ /usr/lib/arm-linux-gnueabihf/librt.a \ /usr/lib/arm-linux-gnueabihf/libpthread.a \ /usr/lib/arm-linux-gnueabihf/libboost_chrono.a \ /usr/lib/arm-linux-gnueabihf/libboost_thread.a"
However, I'm getting the error cannot find the flags to link with Boost regex. config.log here.
The error seems to be /usr/bin/ld: cannot find -lboost_regex-d.

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 16, 2017

EDIT: shouldn't, not should, in the second sentence.

Could you add -L/usr/lib/arm-linux-gnueabihf/ to the LDFLAGS?

You shouldn't need to add the .a files in LDFLAGS, the configure script should find them automatically.

In the log there, it does find at least one version of boost, but then it doesn't link because the other stuff in the LDFLAGS can't find pthread. That might be an issue with the linking order, pthread.a might need to go earlier or later (I can't ever remember which way).

In either case, it would be best if the configure script just picked them up itself. I.e. take them out, and just add the -L flag to the correct directory. Likewise, --enable-static for configure should already add -static to the LDFLAGS.

@BeeeWall
Copy link
Author

It didn't work.
root@localhost:~/xoreos-tools# ldd src/gff2xml libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0xb6ea0000) libxml2.so.2 => /usr/lib/arm-linux-gnueabihf/libxml2.so.2 (0xb6d7d000) libboost_filesystem.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_filesystem.so.1.62.0 (0xb6d5c000) libboost_system.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_system.so.1.62.0 (0xb6d49000) libboost_regex.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_regex.so.1.62.0 (0xb6c86000) libboost_atomic.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_atomic.so.1.62.0 (0xb6c74000) libboost_locale.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_locale.so.1.62.0 (0xb6bc7000) libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0xb6abb000) libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb6a43000) libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb6a19000) libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb692b000) /lib/ld-linux-armhf.so.3 (0x0f000000) libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0xb6918000) libicui18n.so.57 => /usr/lib/arm-linux-gnueabihf/libicui18n.so.57 (0xb676e000) libicuuc.so.57 => /usr/lib/arm-linux-gnueabihf/libicuuc.so.57 (0xb6646000) libicudata.so.57 => /usr/lib/arm-linux-gnueabihf/libicudata.so.57 (0xb4db9000) liblzma.so.5 => /lib/arm-linux-gnueabihf/liblzma.so.5 (0xb4d8f000) librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0xb4d79000) libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0xb4d53000) libboost_chrono.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_chrono.so.1.62.0 (0xb4d3e000) libboost_thread.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_thread.so.1.62.0 (0xb4d12000)
It seems to build libraries statically (I saw a few references), but not binaries. That may be intended, however, based on the help line--enable-static[=PKGS] build static libraries [default=yes].
make.log
config.log

@BeeeWall
Copy link
Author

Also, the error seems to be caused by me adding -static to LDFLAGS. I tried adding it to the configure in the previous post and got the same lboost_regex error.

@BeeeWall
Copy link
Author

If I remove the -static from my post two days ago, it gives the same error as in the original issue (for obvious reasons, without -static it is making a shared object), so it is definitely the -static causing the regex error.

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 19, 2017

Ah, yes, the issue seems to be that libtool hijacks -static for static libraries. See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=11064 .

A workaround is to use --static (note the two hyphens) in the LDFLAGS.

I've put installed static versions of the build dependency libraries into the prefix /opt/static/, and then the folllowing configure line works for me, to produce statically linked xoreos-tools binaries:

./configure --enable-static-boost --with-boost=/opt/static LDFLAGS="--static -L/opt/static/libs/ -lpthread" XML2_CONFIG="/opt/static/bin/xml2-config"

  • --enable-static-boost --with-boost=/opt/static is necessary so that boost.m4 finds the correct libraries
  • --static -L/opt/static/libs/ in the LDFLAGS is necessary for static linking in general
  • -pthread in the LDFLAGS is necessary for several boost libraries (regex among them)
  • XML2_CONFIG, so that the correct libxml2 version is found

@BeeeWall
Copy link
Author

Still getting the same error.
config.log

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 19, 2017

  • -with-boost needs to be /home/xoreos-tools-deps, not /home/xoreos-tools-deps/include
  • It needs to be -pthread, not -lpthread. Subtle difference, but important

I.e. your line should be
./configure --enable-static-boost --with-boost=/home/xoreos-tools-deps LDFLAGS="--static -L/home/xoreos-tools-deps/lib -pthread" XML2_CONFIG=/home/xoreos-tools-deps/bin/xml2-config

@BeeeWall
Copy link
Author

Now I get a Boost locale error, config.log. Also, thanks for your help so far.

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 20, 2017

Seems like you're missing ICU now. You might need to put the ICU libs into the LDFLAGS as well.

I didn't catch that, because I compiled the static boost libs without ICU.

@BeeeWall
Copy link
Author

Where do I find those? I have the libs from ldd, do I need all of them? And do I need to specify some sort of Include path for headers (if so, are they the headers from the unicode include subdirectory)?

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 23, 2017

You'll need the ones shown by ldd if linking dynamically, yes.

Looking at my Boost include directory, I don't think you'll need the ICU headers (which would be the ones in the unicode directory, yes) when compiling xoreos. In fact, a program using Boost would only need them when using regex/icu.hpp, which xoreos does not. All other uses of ICU in Boost seems to be internal only.

@BeeeWall
Copy link
Author

I already had all libraries from the ldd command. I then tried with all ICU libs, and then including the headers. Still got the same error.

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 24, 2017

Just to make sure, you do have static versions of the ICU libs, right?

Otherwise, I'm currently out of guesses, too. I'll need to see for myself, compile a static version of Boost with ICU. Not tonight, though, sorry. :P

@BeeeWall
Copy link
Author

Yes, I do have static ICU libs. Thanks for the help so far.

@BeeeWall
Copy link
Author

Could it be related to the fact that I don't have libibverbs installed? It can never configure, and keeps throwing an error, so it isn't fully installed.

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 25, 2017

It really shouldn't...

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 25, 2017

Ah, I know why now. The problem is the linking order.

When putting the ICU libraries into LDFLAGS, they're put before boost_locale. But to correctly link, they need to come after boost_locale.

Unfortunately, there's currently no way to do that with xoreos' build system. As a workaround, you might want to recompile your static boost libraries without ICU? Not the best solution, I know.

There's two potential future fixes for that, both in our copy of boost.m4:

  1. Have the Boost.Locale check (BOOST_LOCALE) also try to link with the ICU libraries
  2. Extend BOOST_FIND_LIBS to see if BOOST_$1_LDFLAGS (which for Boost.Locale expands to BOOST_LOCALE_LDFLAGS) is already defined, and if so, use that instead of trying to find the flags itself

Both solutions can work in tandem, too.

@BeeeWall
Copy link
Author

I'm using prebuild binaries from the package libboost-dev-all (or something along those lines). Currently, my chroot got messed up and I'm redoing it, but after that I'll try listing the libs in LDFLAGS again, now that we've (well, mostly you've) figured out the error I had with it. Maybe that will solve the incorrect order? Or will it just append my LDFLAGS to the incorrect one?

@DrMcCoy
Copy link
Member

DrMcCoy commented Nov 25, 2017

No, that won't help. The LDFLAGS are always pasted before the library it tries to link.

See https://github.com/xoreos/xoreos-tools/blob/master/m4/boost.m4#L484 . The original LDFLAGS value (which is temporary saved in $boost_save_LDFLAGS) is put before the extra value found by boost.m4. Likewise, the original value of LIBS is put before the boost library it tries.

And the order of LDFLAGS and LIBS among each other is thus that LDFLAGS comes before LIBS (which is done by _BOOST_AC_LINK_IFELSE through the use of autoconf's ac_link, so not something that can or should be changed).

As far as I can see, there isn't really any way to do what is necessary for properply linking the ICU libs with boost_locale here. One (or both) of my solutions above is necessary for that.

@BeeeWall
Copy link
Author

If I took object files for the binaries (by configuring and making without the --static LDFLAG), would I then be able to manually run linker with the libraries, or would I have to search through configure for a bunch of flags as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants