Add mlpack::backtrace utility. #535

Kirizaki · 2016-03-01T22:35:08Z

When build: cmake -DDEBUG=ON ../, Log::Assert() & Log::Fatal prints backtrace.
ONLY FOR LINUX

Add find_package LibBfd.cmake / LibDL.cmake and inc. dirs.

Add note about printing a backtrace.

Include header file.

Include header & source files for backtrace.

Printing backtrace for Log::Assert.

Prints backtrace for Log::Fatal.

Add linking to libraries for backtrace.

Test the RMSprop optimizer with logistic regression and a feedforward…

... libs

Update from master

rcurtin · 2016-03-01T22:44:45Z

Sweet! I will look through this and test it tomorrow.

Kirizaki · 2016-03-02T13:44:39Z

Why it doesn't build correctly? I didn't change anything in tests... Maybe I just merge my branch with master which has some bug and then make my PR or my changes has conflicts with test no. 4 on which travis is crushing?

merge from mlpack

rcurtin · 2016-03-02T18:35:26Z

The travis build was a random test failure, it has nothing to do with your PR, so don't worry about it. Probabilistic tests can be difficult to deal with for this reason...

Kirizaki · 2016-03-03T12:22:51Z

Ohh, that's the thing!

rcurtin · 2016-03-03T19:48:17Z

CMakeLists.txt

+  if(LIBBFD_FOUND AND LIBDL_FOUND)
+    include_directories(${LIBBFD_INCLUDE_DIRS})
+    include_directories(${LIBDL_INCLUDE_DIRS})
+    add_definitions(-DHAS_BFD_DL)


I think you need to add set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -rdynamic") here; if I didn't do this, then if an assert or fatal error was thrown in any of the mlpack programs (like mlpack_allknn), it wouldn't give a backtrace but would tell me that I needed to compile with -rdynamic.

rcurtin · 2016-03-03T20:06:19Z

I tested this on Linux, it works great when I make the necessary change to CMakeLists.txt. I tried to compile on OS X and I get this:

[2%] Building CXX object src/mlpack/CMakeFiles/mlpack.dir/core/util/backtrace.cpp.o
clang: warning: argument unused during compilation: '-rdynamic'
In file included from /Users/ryan_curtin/src/mlpack-kirizaki/src/mlpack/core/util/backtrace.cpp:22:
/opt/local/include/bfd.h:35:2: error: config.h must be included before this header
#error config.h must be included before this header
 ^
1 error generated.

I think that this has been tested on OS X and is known to not work, so maybe we disable HAS_BFD_DL in the CMake configuration if we are on OS X? Either way we should be sure that compilation works out-of-the-box on OS X.

Also there is the warning from clang that -rdynamic is unsupported; maybe different handling is necessary there?
http://stackoverflow.com/questions/21279036/what-is-clangs-equivalent-to-rdynamic-gcc-flag/21279573#21279573
That one will be a little bit weird because apparently the flag is different on OS X!

I haven't been able to actually test with clang because of a Debian bug that affects the system I was testing on:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=797917

It would also be cool to test this on FreeBSD but my FreeBSD box is not running right now... :(

rcurtin · 2016-03-03T20:06:41Z

I should add, I'm very excited about this, I think this backtrace is a really nice improvement. 👍

Kirizaki · 2016-03-03T21:06:51Z

Part You pointed in code is done.

Second thing:

I think that this has been tested on OS X and is known to not work, so maybe we disable HAS_BFD_DL in the CMake configuration if we are on OS X? Either way we should be sure that compilation works out-of-the-box on OS X.

Disabling HAS_BFD_DL is temporary but good idea at the moment. Tomorrow I should take care about this.

But in near future it has to be done properly. @zoq did some research about compiling mlpack::backtrace in OS X. There is also problem with dependencies in binutils.

[Friday, December 11, 2015] [09:15:03 PM] Okay, it turns out that if you install binutils using homebrew it doesn't come with libiberty because of 'Homebrew/legacy-homebrew#35881'.
[Friday, December 11, 2015] [09:15:17 PM] Unfortunately there is no brew formula for libiberty. So you need to install it manually. Afterwards, you can link against '-liberty'.
[Friday, December 11, 2015] [09:15:30 PM] Unfortunately there is no brew formula for libiberty. So you need to install it manually. Afterwards, you can link against '-liberty'. But now it turns out you also need to link against libintl which comes with the gettext package. Turns out gettext ships with OS X (default). So my gettext package conflicts with the default version.
[Friday, December 11, 2015] [09:15:44 PM] No problem adjust the path and we are fine. At the end we can link against the lib '-lintl'.
[Friday, December 11, 2015] [09:15:52 PM] Now the source code build fine, but prints an error message 'Failed to open executable! : No such file or directory'. On OS X there is no '/proc/self/exe', any ideas?

So my next step is to prevent building mlpack::backtrace in OS X and FreeBSD. This should help for a while. Later on I will just work on extending this feature for other platforms.

rcurtin · 2016-03-03T21:12:23Z

Awesome, this sounds like a good approach. I'm fine with this only working on Linux for now (because I think what we have implemented only works on Linux, and it doesn't work very well), and we can extend support later. We just have to make sure that it still compiles correctly on other platforms before we merge it in. :)

Kirizaki · 2016-03-03T21:16:04Z

I agree. I forget that mlpack is not only for Linux users. :P

Use CreateNVP from the data namespace.

zoq · 2016-03-03T23:41:13Z

Okay, I tested the code on FreeBSD, and ended up with the same error as for OSX: there is no '/proc/self/exe'. I changed the CMakeLists.txt file so that it doesn't look for the libdl library. libdl is is part of libc so FindLibDL always returns "libdl not found". Anyway, everything builds fine, without changing the CMakeLists.txt file.

Find and compile libbfd & libdl only when OS is Linux.

...find and compile libbfd & libdl only when OS is Linux.

...compile backtrace.hpp only when OS is Linux

Add mlpack::backtrace utility.

zoq · 2016-03-09T01:47:18Z

Since, we tested this on FreeBSD, OSX and windows, yes right Microsoft Windows. I'll go and merge this in. Thanks for the contribution!

rcurtin · 2016-03-14T15:21:04Z

Thanks again, this is great! :)

Kirizaki added 18 commits February 24, 2016 21:22

Add mlpack::backtrace for Ubuntu (C++, Clang).

e671aa8

Commit mlpack::backtrace.

c827d59

Update for mlpack::backtrace

e9ee6bd

Add find_package LibBfd.cmake / LibDL.cmake and inc. dirs.

Delete old backtrace depend.

bce8a34

Add mlpack::backtrace info.

aa35528

Add note about printing a backtrace.

Add mlpack::backtrace header.

5f430cd

Include header file.

Add mlpack::backtrace

8cbd0d5

Include header & source files for backtrace.

Add mlpack::backtrace

1404c1d

Printing backtrace for Log::Assert.

Add mlpack::backtrace

5e2f445

Prints backtrace for Log::Fatal.

Add mlpack::backtrace

6257fad

Add linking to libraries for backtrace.

Merge pull request #1 from mlpack/master

322d384

Test the RMSprop optimizer with logistic regression and a feedforward…

Update log.cpp

876de9c

Dummy constructor

95b9320

Dummy constructor

0a0d68b

Include backtrace.hpp only when libs are found

dc0e553

Work with mlpack::backtrace only when HAS_BFD_DL..

171c35b

... libs

Merge pull request #2 from mlpack/master

170c3a2

Update from master

Update prefixedoutstream_impl.hpp

76e48a5

Merge pull request #3 from mlpack/master

f987048

merge from mlpack

rcurtin reviewed Mar 3, 2016
View reviewed changes

Kirizaki added 2 commits March 3, 2016 21:36

Add -rdynamic if BFD or DL found.

6c4eb24

Change Ubuntu on Linux

0bc16ea

Kirizaki added 2 commits March 3, 2016 21:40

Change order in printing backtrace

e664f3d

Change order in printing backtrace

f8228a5

Merge pull request #4 from mlpack/master

78c60a7

Use CreateNVP from the data namespace.

Kirizaki added 5 commits March 5, 2016 21:02

Linux specific dependencies

cdb5ac3

Find and compile libbfd & libdl only when OS is Linux.

Platform specific dependencies

a0472e4

...find and compile libbfd & libdl only when OS is Linux.

Linux specific dependencies

0aad1ee

...compile backtrace.hpp only when OS is Linux

Update CMakeLists.txt

38c6d82

Update CMakeLists.txt

7ae1355

zoq added a commit that referenced this pull request Mar 9, 2016

Merge pull request #535 from Kirizaki/master

1a9c41a

Add mlpack::backtrace utility.

zoq merged commit 1a9c41a into mlpack:master Mar 9, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add mlpack::backtrace utility. #535

Add mlpack::backtrace utility. #535

Kirizaki commented Mar 1, 2016

rcurtin commented Mar 1, 2016

Kirizaki commented Mar 2, 2016

rcurtin commented Mar 2, 2016

Kirizaki commented Mar 3, 2016

rcurtin Mar 3, 2016

Kirizaki Mar 3, 2016

rcurtin commented Mar 3, 2016

rcurtin commented Mar 3, 2016

Kirizaki commented Mar 3, 2016

rcurtin commented Mar 3, 2016

Kirizaki commented Mar 3, 2016

zoq commented Mar 3, 2016

zoq commented Mar 9, 2016

rcurtin commented Mar 14, 2016

Add mlpack::backtrace utility. #535

Add mlpack::backtrace utility. #535

Conversation

Kirizaki commented Mar 1, 2016

rcurtin commented Mar 1, 2016

Kirizaki commented Mar 2, 2016

rcurtin commented Mar 2, 2016

Kirizaki commented Mar 3, 2016

rcurtin Mar 3, 2016

Choose a reason for hiding this comment

Kirizaki Mar 3, 2016

Choose a reason for hiding this comment

rcurtin commented Mar 3, 2016

rcurtin commented Mar 3, 2016

Kirizaki commented Mar 3, 2016

rcurtin commented Mar 3, 2016

Kirizaki commented Mar 3, 2016

zoq commented Mar 3, 2016

zoq commented Mar 9, 2016

rcurtin commented Mar 14, 2016