Fix compiling issues with gcc v6.x #89

colinhd8 · 2017-06-20T10:17:37Z

A few days ago i switch os from fedora to ubuntu, and use the latest build.sh to compile rtorrent. But have some problems about compile:
1). line 133 of build.sh: if it was not comment out, the error occurs(build with ./build.sh all):

	                 from main.cc:60:
	../rak/priority_queue_default.h: In destructor ‘rak::priority_item::~priority_item()’:
	../rak/priority_queue_default.h:56:95: warning: throw will always call terminate() [-Wterminate]
	       throw torrent::internal_error("priority_item::~priority_item() called on a queued item.");
	                                                                                               ^
	../rak/priority_queue_default.h:56:95: note: in C++11 destructors default to noexcept
	mv -f .deps/main.Tpo .deps/main.Po
	/bin/bash ../libtool  --tag=CXX   --mode=link g++  -g -DDEBUG -Wall   -I/home/test/lib/rtorrent-0.9.7-1.5.1/include  -I/home/test/lib/rtorrent-0.9.7-1.5.1/include   -L/home/test/lib/rtorrent-0.9.7-1.5.1/lib -o rtorrent main.o libsub_root.a ui/libsub_ui.a core/libsub_core.a display/libsub_display.a input/libsub_input.a rpc/libsub_rpc.a utils/libsub_utils.a  -lncursesw -lcppunit -L/home/test/lib/rtorrent-0.9.7-1.5.1/lib -lcurl -ltorrent -lpthread -L/home/test/lib/rtorrent-0.9.7-1.5.1/lib   -lxmlrpc_server -lxmlrpc  -lxmlrpc_xmlparse -lxmlrpc_xmltok -lxmlrpc_util -lpthread 
	libtool: link: g++ -g -DDEBUG -Wall -I/home/test/lib/rtorrent-0.9.7-1.5.1/include -I/home/test/lib/rtorrent-0.9.7-1.5.1/include -o rtorrent main.o  -L/home/test/lib/rtorrent-0.9.7-1.5.1/lib libsub_root.a ui/libsub_ui.a core/libsub_core.a display/libsub_display.a input/libsub_input.a rpc/libsub_rpc.a utils/libsub_utils.a -lncursesw -lcppunit /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libcurl.so /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libtorrent.so -lxmlrpc_server -lxmlrpc -lxmlrpc_xmlparse -lxmlrpc_xmltok -lxmlrpc_util -lpthread -Wl,-rpath -Wl,/home/test/lib/rtorrent-0.9.7-1.5.1/lib -Wl,-rpath -Wl,/home/test/lib/rtorrent-0.9.7-1.5.1/lib
	/home/test/lib/rtorrent-0.9.7-1.5.1/lib/libtorrent.so: undefined reference to `torrent::PeerConnection<(torrent::Download::ConnectionType)1>::offer_chunk()'
	/home/test/lib/rtorrent-0.9.7-1.5.1/lib/libtorrent.so: undefined reference to `torrent::PeerConnection<(torrent::Download::ConnectionType)0>::offer_chunk()'
	collect2: error: ld returned 1 exit status
	Makefile:548: recipe for target 'rtorrent' failed
	make[3]: *** [rtorrent] Error 1
	make[3]: Leaving directory '/home/test/rtorrent-ps/rtorrent-0.9.7/src'
	Makefile:614: recipe for target 'all-recursive' failed
	make[2]: *** [all-recursive] Error 1
	make[2]: Leaving directory '/home/test/rtorrent-ps/rtorrent-0.9.7/src'
	Makefile:450: recipe for target 'all-recursive' failed
	make[1]: *** [all-recursive] Error 1
	make[1]: Leaving directory '/home/test/rtorrent-ps/rtorrent-0.9.7'
	Makefile:382: recipe for target 'all' failed
	make: *** [all] Error 2

2). it's seems that all of the necessary xmlrpc's so file can be found at the compile time, but some of them can't be found at runtime without change LD_LIBRARY_PATH.(build with ./build.sh all, of course firstly comment out line 133)
the command of (ldd rtorrent)'s output:

        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fd3df4a0000)
        libcurl.so.4 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libcurl.so.4 (0x00007fd3df235000)
        libtorrent.so.19 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libtorrent.so.19 (0x00007fd3def02000)
        libxmlrpc_server.so.3 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libxmlrpc_server.so.3 (0x00007fd3decfb000)
        libxmlrpc.so.3 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libxmlrpc.so.3 (0x00007fd3deadd000)
        libxmlrpc_util.so.3 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libxmlrpc_util.so.3 (0x00007fd3de8d7000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd3de6b9000)
        /lib64/ld-linux-x86-64.so.2 (0x0000564f45cd9000)
        libxmlrpc_xmlparse.so.3 => not found

strange thing is libxmlrpc_xmlparse.so.3 not found,

but libxmlrpc_server.so.3,
libxmlrpc.so.3,
libxmlrpc_util.so.3
can be found. In fact all of them are in the same directory.

For the problem 1, i test it with other os, and all of them have the same result.These os are:
Fedora 24 x64 GCC:6.3.1 20161221 (Red Hat 6.3.1-1) (GCC)
CentOs 7 x64 GCC:gcc version 6.2.1 20160916 (Red Hat 6.2.1-3) (GCC)

For the problem 2, only ubuntu has this problem.

ubuntu version: ubuntu-server 4.10.0-22-generic #24-Ubuntu SMP Mon May 22 17:43:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

The text was updated successfully, but these errors were encountered:

chros73 · 2017-06-20T13:17:52Z

About 1.:

I heard about it, I'm pretty sure it's the "compiler's fault" for whatever reason :)
rakshasa only uses versions up to 4.8 with travisCI
you can try to build a simple rtorrent/libtorrent on that box and if it fails (it should) then create an issue on rtorrent/libtorrent project

About 2.

pyroscope did something about it but I haven't applied his changes 'cause I couldn't try it out
give it a try: here , and here

colinhd8 · 2017-06-20T15:45:22Z

Thank you for your reply.

About 1:
ok, i'll let it go.

About 2:
i have tried it, but still have the problem. For now, i will modify the start script to set LD_LIBRARY_PATH.

chros73 · 2017-07-17T10:08:35Z

I took a look about point 1. (using Debian 9 Stretch):

only compiling rtorrent from git master branches has this problem and only with gcc v6.x
- the xmlrpc check fails here
- I think it only affects the check itself not the compilation (I haven't tried out yet)
bug is distro dependent, it only depends on the version of gcc
all previous gcc versions are ok (4.x, 5.x) , I don't know anything about 7.x

Solution for now is what you mentioned: thank you for this, you saved a lot of time for me! :)

instead of the line export CXXFLAGS="$CFLAGS" in build.sh, use this for now:
- [[ -n "$CFLAGS" ]] && export CXXFLAGS="$CFLAGS"
it's fine to not set it on Linux (it should be empty anyway)

Honestly I don't know what causes the problem:

it's like setting this environment variable overwrites the same variable in checks.m4
since 0.9.6 isn't affected by this:
- maybe it's a problem with rtorrent
- or maybe it's a problem with autoconfigure that build.sh uses

I need more investigation.

@pyroscope, what do you think?

chros73 · 2017-07-17T10:31:26Z

For the problem 2, only ubuntu has this problem.

Which version/flavour is this exactly? Can you link the iso image? (I want to play with it, maybe there's a simpler solution.)
Was it: 17.04 (Zesty Zapus) ?

colinhd8 · 2017-07-17T13:09:44Z

Yes, it is 17.04 (Zesty Zapus) . And like you say, it only affects the check itself not the compilation. But when you run rtorrent, you also need set LD_LIBRARY_PATH, else you will fail for not found the so files.
And it seems that when rtorrent was released, some changes were made to it so it was different to the master version(of course the master version was the version of released version based on), and they have
different compile result.

chros73 · 2017-07-17T15:16:20Z

Yes, it is 17.04 (Zesty Zapus)
you also need set LD_LIBRARY_PATH, else you will fail for not found the so files.

Thanks for the confirmation. I asked for the exact distro because Debian 9 - strangely enough - doesn't have this rpath problem.

it only affects the check itself not the compilation

Thanks, as I thought, I'll try out today, just in case, disabling the check in checks.m4

of course the master version was the version of released version based on

:) That's what I thought as well 1 year ago :) , as it turned out it's not the case: different fixes, small enhancements are cherry-picked "randomly"!
It's still a usual habit (probably inherited from the days of svn) that certain projects (e.g. rtorrent/libtorrent, openwrt, lede-project, etc.) use master branch as a develop branch, and they have a "release" branch for releasing from, e.g.:

lede: lede-17.01
rtorrent/libtorrent: branch-0.9

The only difference is regarding to the other projects, that they release a major (!) version in every 1-2 years, not 8-10! :) More than that, they backport fixes from master to the release branch and not applying any major changes that can brake the previous minor version (that's true also for rtorrent).
Not to mention that when they release a major version, it's usually based on the master branches.

The drawback is (if it's not the case) that the code in master branch is untested by the public so when the next major version is released it can be full of bugs! That's exactly what happens in the case of rtorrent. :)

As you can see, I don't like this dev approach at all, hence I'm doing in a different way :)

So, in summary: this issue can be easily the fault of rtorrent but as easily be a fault of our autoconf chain.

colinhd8 · 2017-07-17T16:16:41Z

Thanks, as I thought, I'll try out today, just in case, disabling the check in checks.m4

The check i mentioned was in the build.sh file.(echo -n "Check that static linking worked: "), and it's seems that we are not discuss the same check.

And thank you so much for telling me the detail about release, i learned a lot.

pyroscope · 2017-07-17T16:43:39Z

./build.sh docker_deb ubuntu:zesty

… works fine. git tip? meh, until the current mess is properly merged to something coherent.

chros73 · 2017-07-20T16:26:06Z

it's seems that we are not discuss the same check.

:) Probably.

About issue 1.:
btw: how did you find out that commenting out CXXFLAGS solves the problem???! :)

Ok, I've played with it a lot: it's definitely the problem of libtorrent (and probably rtorrent?), but I don't have a clue why. I'll create an issue about it in libtorrent project, maybe rakshasa will pick it up.

Since I'm not sure at all that rtorrent isn't affected by this hence my dirty fix will be not just commenting out that line but unsetting it if gcc varsion >= 6, since this env variable can be easily set in the user's enviroment as well:

unset CXXFLAGS

The interesting bit about this nasty bug, that compilation is going fine until the given method/object is tried to be used: thus breaks the xmlrpc check in the beginning of compilation of rtorrent.

I tested 0.9.6 on the same system: it doesn't suffer from this bug.

git tip? meh,

:D Otherwise it's rock stable, even with my patches :D

chros73 · 2017-07-21T09:38:52Z

I've created the issue in libtorrent: rakshasa/libtorrent#155
I've also committed the fix for now for it, it's on develop branch for now.

I've created a new issue for rpath linking bug: #93

chros73 · 2017-07-23T17:12:47Z

@colinhd8 : new version is out, can you test it e.g. on Ubuntu 17.07? (that's the most problematic system, but more system would be better)
You don't need to set the LD_LIBRARY_PATH env var: unset it, just in case.

New build is tested on:

Ubuntu 14.04
Ubuntu 17.04
Debian 9

Thanks for the bug reports!

colinhd8 · 2017-07-24T14:59:49Z

btw: how did you find out that commenting out CXXFLAGS solves the problem???! :)

I forgot how. May compare you build file with pyroscope's ?

And thanks to your work for the new version, i'll try it and report the result.

chros73 · 2017-07-24T18:18:11Z

I forgot how. May compare you build file with pyroscope's ?

:) It's in his as well.
My point is: knowing what I know now, I don't think that I'd have ever thought about this :)
So, thank You!

Btw: I added a small optimization example code. If you can compile it normally, then uncomment that line (after you check which march your CPU belongs to), and compile it again :)

colinhd8 · 2017-07-25T17:51:15Z

@chros73 I have compiled on Fedora 26(gcc 7.1.1 20170622, and seems that openssl patch works fine) and ubuntu server 17.04， all of them are ok now(wl rpath is not need for fedora). Thank you.

Next i will try the optimization code, and thank you again for the tips and great work.

chros73 · 2017-07-25T19:05:58Z

I have added more (final) info onto the rpath linking issue.

all of them are ok now.

Good to hear it! :)

wl rpath is not need for fedora

Good to know. But it only gives benefit (moving, copying main directory), so just leave it there :)

Next i will try the optimization code

With newer gcc (> 5) it won't do much for now due to this check.
But: I have heard that this hack isn't needed on some v7.x distros (e.g. Slackware gcc v7.1):

so if you want to use Fedora then try to comment out that check and see whether it compiles fine, so you can have the optimization :)

So, in summary: I don't understand where the CXXFLAGS bug comes from :)

pyroscope · 2017-07-26T13:09:49Z

Apply Occam's Razor, and an empty CXXFLAGS will likely defeat some defaults that are needed for the git head code, and would need to be included in the custom flags. Now find out what those defaults are.

colinhd8 · 2017-07-26T15:47:47Z

But: I have heard that this hack isn't needed on some v7.x distros (e.g. Slackware gcc v7.1):

I have tried it and below is the results(rtorrent version: latest version from master branch):

Fedora 26 gcc 7.1.1 20170622
1. comment out [[ -z "$GCC_MAIN_VER" ]] || [[ $GCC_MAIN_VER -gt 5 ]] && export FIX_LT_GCC_BUG="yes"
  result: build success, so it seems that CXXFLAG is ok for gcc 7.1.1.
2. keep the line mentioned above comment out, then modify
  # export CFLAGS="-march=core2 -pipe -O2 -fomit-frame-pointer${CFLAGS:+ }${CFLAGS}" to
  export CFLAGS="-march=haswell -pipe -O2 -fomit-frame-pointer${CFLAGS:+ }${CFLAGS}"(since my gcc version >= 4.9.0)
  result: build success.
Ubuntu Server 17.04 gcc 6.3.0 20170406
1. comment out [[ -z "$GCC_MAIN_VER" ]] || [[ $GCC_MAIN_VER -gt 5 ]] && export FIX_LT_GCC_BUG="yes"
  result: build fail
2. keep the line mentioned above comment out, then modify
  # export CFLAGS="-march=core2 -pipe -O2 -fomit-frame-pointer${CFLAGS:+ }${CFLAGS}" to
  export CFLAGS="-march=haswell -pipe -O2 -fomit-frame-pointer${CFLAGS:+ }${CFLAGS}"
  result: build success.

chros73 · 2017-07-26T21:28:24Z

@colinhd8 : thanks so much for testing!
I wasn't sure at first whether I correctly understand what you meant in 2. II. but I think it was:

#[[ -z "$GCC_MAIN_VER" ]] || [[ $GCC_MAIN_VER -gt 5 ]] && export FIX_LT_GCC_BUG="yes"
export CFLAGS="-march=haswell -pipe -O2 -fomit-frame-pointer${CFLAGS:+ }${CFLAGS}"

So, at first, I haven't realized this "shocking" news! :D

@pyroscope

an empty CXXFLAGS will likely defeat some defaults that are needed for the git head code, and would need to be included in the custom flags. Now find out what those defaults are.

You were right! Thanks!

I found out what's the real issue by fiddling more. Take a look at the B. part of this comment.

colinhd8 · 2017-07-27T14:15:57Z

@chros73

wasn't sure at first whether I correctly understand what you meant in 2. II. but I think it was:

Yes, you are right. Below was what i have done:
1.
[[ -z "$GCC_MAIN_VER" ]] || [[ $GCC_MAIN_VER -gt 5 ]] && export FIX_LT_GCC_BUG="yes"
=>
#[[ -z "$GCC_MAIN_VER" ]] || [[ $GCC_MAIN_VER -gt 5 ]] && export FIX_LT_GCC_BUG="yes"

# export CFLAGS="-march=core2 -pipe -O2 -fomit-frame-pointer${CFLAGS:+ }${CFLAGS}"
=>
export CFLAGS="-march=haswell -pipe -O2 -fomit-frame-pointer${CFLAGS:+ }${CFLAGS}"

found out what's the real issue by fiddling more. Take a look at the B. part of this comment.

I also notice that there's a little difference between two make files's CXXFLAG , but not sure which parameter was the reason. Thanks for point out the real issue and thanks to @pyroscope for the tips.

chros73 · 2017-07-27T18:41:11Z

And here's the proper fix for the issue (not implemented yet). :)

I learned a lot during the last week, it was edifying. :) E.g.:

if compilation fails, ask the user to rerun without any flags! Right, @pyroscope ? :) (you also enabled the -j4 option for make)

The next piece of work will be, mostly modifying the build process (I'm not just writing down for you but for myself as well :) ):

implementing this fix
trying to get rid of the absolute rpath from binaries (rtorrent, libcurl.so)
modifying the install option (we just want to copy the compiled dir into /opt)
- updating the building instruction
terminating building process any time when the previous step is failed
- lack of this caused one of my misunderstanding about the issue
creating auto optimized builds by default: use -march=native in CFLAGS
- make it configurable in build script: e.g. OPTIMIZE=true
- updating the building instruction

@colinhd8 : thanks, again, for your effort! It would have taken way-way more time without you!

Btw, what happened the color-theme configs?! :D

colinhd8 · 2017-07-28T07:12:38Z

Btw, what happened the color-theme configs?! :D

I have just create a pr, please review it. :D

chros73 · 2017-07-28T07:25:00Z

:D Thanks, I'll take a look at it during the weekend.

chros73 added bug build git help wanted labels Jul 11, 2017

chros73 changed the title ~~compile issues with ubuntu~~ Compile issues with gcc v6.3.0 and rpath linking Jul 17, 2017

chros73 mentioned this issue Jul 17, 2017

Update build.sh #72

Closed

chros73 changed the title ~~Compile issues with gcc v6.3.0 and rpath linking~~ Fix compiling issues with gcc v6.x Jul 21, 2017

chros73 pushed a commit that referenced this issue Jul 21, 2017

Fix compiling issues with gcc v6.x (Closes #89)

c938676

chros73 pushed a commit that referenced this issue Jul 22, 2017

Add gcc optimization hint on Linux (See #89)

e0d5816

chros73 pushed a commit that referenced this issue Jul 22, 2017

Add known issues section into Readme (See #89)

ba13c79

chros73 closed this as completed in d138a80 Jul 23, 2017

chros73 added this to the 1.5.3-0.9.7 milestone Jul 23, 2017

chros73 mentioned this issue Jul 30, 2017

Fix compiling issues with gcc v6.x and libtool properly #106

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix compiling issues with gcc v6.x #89

Fix compiling issues with gcc v6.x #89

colinhd8 commented Jun 20, 2017 •

edited

Loading

chros73 commented Jun 20, 2017

colinhd8 commented Jun 20, 2017

chros73 commented Jul 17, 2017

chros73 commented Jul 17, 2017 •

edited

Loading

colinhd8 commented Jul 17, 2017

chros73 commented Jul 17, 2017

colinhd8 commented Jul 17, 2017 •

edited

Loading

pyroscope commented Jul 17, 2017

chros73 commented Jul 20, 2017 •

edited

Loading

chros73 commented Jul 21, 2017

chros73 commented Jul 23, 2017

colinhd8 commented Jul 24, 2017

chros73 commented Jul 24, 2017

colinhd8 commented Jul 25, 2017

chros73 commented Jul 25, 2017

pyroscope commented Jul 26, 2017

colinhd8 commented Jul 26, 2017 •

edited

Loading

chros73 commented Jul 26, 2017

colinhd8 commented Jul 27, 2017 •

edited

Loading

chros73 commented Jul 27, 2017

colinhd8 commented Jul 28, 2017

chros73 commented Jul 28, 2017

Fix compiling issues with gcc v6.x #89

Fix compiling issues with gcc v6.x #89

Comments

colinhd8 commented Jun 20, 2017 • edited Loading

chros73 commented Jun 20, 2017

colinhd8 commented Jun 20, 2017

chros73 commented Jul 17, 2017

chros73 commented Jul 17, 2017 • edited Loading

colinhd8 commented Jul 17, 2017

chros73 commented Jul 17, 2017

colinhd8 commented Jul 17, 2017 • edited Loading

pyroscope commented Jul 17, 2017

chros73 commented Jul 20, 2017 • edited Loading

chros73 commented Jul 21, 2017

chros73 commented Jul 23, 2017

colinhd8 commented Jul 24, 2017

chros73 commented Jul 24, 2017

colinhd8 commented Jul 25, 2017

chros73 commented Jul 25, 2017

pyroscope commented Jul 26, 2017

colinhd8 commented Jul 26, 2017 • edited Loading

chros73 commented Jul 26, 2017

colinhd8 commented Jul 27, 2017 • edited Loading

chros73 commented Jul 27, 2017

colinhd8 commented Jul 28, 2017

chros73 commented Jul 28, 2017

colinhd8 commented Jun 20, 2017 •

edited

Loading

chros73 commented Jul 17, 2017 •

edited

Loading

colinhd8 commented Jul 17, 2017 •

edited

Loading

chros73 commented Jul 20, 2017 •

edited

Loading

colinhd8 commented Jul 26, 2017 •

edited

Loading

colinhd8 commented Jul 27, 2017 •

edited

Loading