Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

boost --with-python required on osx for pycaffe target #465 #1193

Closed
sanjeev87 opened this issue Sep 30, 2014 · 25 comments
Closed

boost --with-python required on osx for pycaffe target #465 #1193

sanjeev87 opened this issue Sep 30, 2014 · 25 comments

Comments

@sanjeev87
Copy link

brew install --build-from-source --with-python --fresh -vd boost is not installing boost_python lib files .. Please refer Homebrew/legacy-homebrew#32704 , that explains that boost no longer installs python .. I am still trying to install boost python , any advice helps as this method is not working

@shelhamer
Copy link
Member

Try boost 1.55 and older homebrew packages instead. boost 1.56 seems to have yet more issues with NVCC and doesn't build for me due to

/usr/local/include/boost/config/suffix.hpp(496): error: identifier "__int128" is undefined
/usr/local/include/boost/config/suffix.hpp(497): error: expected a ";"

@shelhamer
Copy link
Member

git checkout a252214 /usr/local/Library/Formula/boost.rb in the homebrew clone, that is /usr/loca/Cellar or the like, will restore the boost 1.55 formula that works on OSX.

@roseperrone
Copy link

That fixed build errors, but I still get the following runtime error. What can I try to fix it?

(lldb) bt
* thread #1: tid = 0x9437e, 0x0000000000000000, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000
    frame #1: 0x0000000103f3b37d Python`PyEval_GetGlobals + 23
    frame #2: 0x0000000103f49ea5 Python`PyImport_Import + 131
    frame #3: 0x0000000103f48687 Python`PyImport_ImportModule + 31
    frame #4: 0x0000000102ee9d4d _caffe.so`caffe::init_module__caffe() + 3149
    frame #5: 0x0000000103e5f071 libboost_python.dylib`boost::python::handle_exception_impl(boost::function0<void>) + 81
    frame #6: 0x0000000103e601e6 libboost_python.dylib`boost::python::detail::init_module(char const*, void (*)()) + 118
    frame #7: 0x000000010009fbc6 Python`_PyImport_LoadDynamicModule + 140
    frame #8: 0x000000010009f8d3 Python`import_submodule + 274
    frame #9: 0x000000010009f49d Python`load_next + 280
    frame #10: 0x000000010009d454 Python`PyImport_ImportModuleLevel + 589
    frame #11: 0x0000000100082e3b Python`builtin___import__ + 135
    frame #12: 0x000000010000e60f Python`PyObject_Call + 99
    frame #13: 0x000000010008d697 Python`PyEval_CallObjectWithKeywords + 93
    frame #14: 0x0000000100089eb2 Python`PyEval_EvalFrameEx + 9358
    frame #15: 0x0000000100087893 Python`PyEval_EvalCodeEx + 1608
    frame #16: 0x0000000100087245 Python`PyEval_EvalCode + 54
    frame #17: 0x000000010009c141 Python`PyImport_ExecCodeModuleEx + 236
    frame #18: 0x000000010009ecee Python`load_source_module + 813
    frame #19: 0x000000010009f8d3 Python`import_submodule + 274
    frame #20: 0x000000010009f49d Python`load_next + 280
    frame #21: 0x000000010009d454 Python`PyImport_ImportModuleLevel + 589
    frame #22: 0x0000000100082e3b Python`builtin___import__ + 135
    frame #23: 0x000000010000e60f Python`PyObject_Call + 99
    frame #24: 0x000000010008d697 Python`PyEval_CallObjectWithKeywords + 93
    frame #25: 0x0000000100089eb2 Python`PyEval_EvalFrameEx + 9358
    frame #26: 0x0000000100087893 Python`PyEval_EvalCodeEx + 1608
    frame #27: 0x0000000100087245 Python`PyEval_EvalCode + 54
    frame #28: 0x000000010009c141 Python`PyImport_ExecCodeModuleEx + 236
    frame #29: 0x000000010009ecee Python`load_source_module + 813
    frame #30: 0x000000010009f0f3 Python`load_package + 303
    frame #31: 0x000000010009f8d3 Python`import_submodule + 274
    frame #32: 0x000000010009f49d Python`load_next + 280
    frame #33: 0x000000010009d454 Python`PyImport_ImportModuleLevel + 589
    frame #34: 0x0000000100082e3b Python`builtin___import__ + 135
    frame #35: 0x000000010000e60f Python`PyObject_Call + 99
    frame #36: 0x000000010008d697 Python`PyEval_CallObjectWithKeywords + 93
    frame #37: 0x0000000100089eb2 Python`PyEval_EvalFrameEx + 9358
    frame #38: 0x0000000100087893 Python`PyEval_EvalCodeEx + 1608
    frame #39: 0x0000000100087245 Python`PyEval_EvalCode + 54
    frame #40: 0x00000001000a70c2 Python`run_mod + 53
    frame #41: 0x00000001000a7165 Python`PyRun_FileExFlags + 133
    frame #42: 0x00000001000a6d02 Python`PyRun_SimpleFileExFlags + 769
    frame #43: 0x00000001000b8256 Python`Py_Main + 2970
    frame #44: 0x00007fff85b825fd libdyld.dylib`start + 1

I followed the advice of https://github.com/Homebrew/homebrew/wiki/Common-Issues (in the segfaut section) with no luck.

@shuggiefisher
Copy link

brew install boost-python fixed it for me on 1.56

@roseperrone
Copy link

I did that (uninstalled boost 1.55, installed boost 1.56, ran brew install boost-python, but now I see these errors when I run make clean; make all; make runtest:

[----------] Global test environment tear-down
[==========] 457 tests from 98 test cases ran. (17644 ms total)
[  PASSED  ] 454 tests.
[  FAILED  ] 3 tests, listed below:
[  FAILED  ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[  FAILED  ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU
[  FAILED  ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU

 3 FAILED TESTS
make: *** [runtest] Error 1

In my Makefile.config, I've set CPU_ONLY := 1.

@shelhamer
Copy link
Member

There should be more detailed errors where those tests are run. Do
./build/test/test_power_layer.testbin and include the gradient checker
outputs for the listed failures.

On Wed, Oct 8, 2014 at 12:34 PM, Rose Perrone notifications@github.com
wrote:

I did that (uninstalled boost 1.55, installed boost 1.56, ran brew
install boost-python, but now I see these errors when I run make clean;
make all; make runtest:

[----------] Global test environment tear-down
[==========] 457 tests from 98 test cases ran. (17644 ms total)
[ PASSED ] 454 tests.
[ FAILED ] 3 tests, listed below:
[ FAILED ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU

3 FAILED TESTS
make: *** [runtest] Error 1


Reply to this email directly or view it on GitHub
#1193 (comment).

@sanjeev87
Copy link
Author

Hi ,

I am facing the same issue as well.
When I run ./build/test/test_power_layer.testbin i get the following errors,

[ FAILED ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU (3 ms)
[ RUN ] PowerLayerTest/1.TestPowerZero
[ OK ] PowerLayerTest/1.TestPowerZero (0 ms)
[ RUN ] PowerLayerTest/1.TestPowerZeroGradient
[ OK ] PowerLayerTest/1.TestPowerZeroGradient (1 ms)
[ RUN ] PowerLayerTest/1.TestPowerOne
[ OK ] PowerLayerTest/1.TestPowerOne (0 ms)
[ RUN ] PowerLayerTest/1.TestPowerOneGradient
[ OK ] PowerLayerTest/1.TestPowerOneGradient (2 ms)
[ RUN ] PowerLayerTest/1.TestPowerTwo
[ OK ] PowerLayerTest/1.TestPowerTwo (0 ms)
[ RUN ] PowerLayerTest/1.TestPowerTwoGradient
[ OK ] PowerLayerTest/1.TestPowerTwoGradient (15 ms)
[ RUN ] PowerLayerTest/1.TestPowerTwoScaleHalfGradient
[ OK ] PowerLayerTest/1.TestPowerTwoScaleHalfGradient (3 ms)
[----------] 10 tests from PowerLayerTest/1 (27 ms total)

[----------] Global test environment tear-down
[==========] 20 tests from 2 test cases ran. (78 ms total)
[ PASSED ] 17 tests.
[ FAILED ] 3 tests, listed below:
[ FAILED ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU

Please help fix this issue.

@roseperrone
Copy link

I have build/test/test_all.testbin, but not build/test/test_power_layer.testbin. When I ran it, I found this error:

[ RUN      ] BlobSimpleTest/0.TestPointersCPUGPU
F1009 07:15:19.080399 2099200784 syncedmem.cpp:91] CPU-only Mode: cannot make GPU call.
*** Check failure stack trace: ***
    @        0x1074dc88a  google::LogMessage::Fail()
    @        0x1074dbaa8  google::LogMessage::SendToLog()
    @        0x1074dc51a  google::LogMessage::Flush()
    @        0x1074e0008  google::LogMessageFatal::~LogMessageFatal()
    @        0x1074dcd25  google::LogMessageFatal::~LogMessageFatal()
    @        0x1071ec3b6  caffe::SyncedMemory::gpu_data()
    @        0x10717f8a6  caffe::Blob<>::gpu_data()
    @        0x106f9d72c  caffe::BlobSimpleTest_TestPointersCPUGPU_Test<>::TestBody()
    @        0x10713f69c  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @        0x10712f6ba  testing::Test::Run()
    @        0x107130602  testing::TestInfo::Run()
    @        0x107130cd0  testing::TestCase::Run()
    @        0x1071361e5  testing::internal::UnitTestImpl::RunAllTests()
    @        0x10713ff84  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @        0x107135f09  testing::UnitTest::Run()
    @        0x106f75c0a  main
    @     0x7fff8fa795fd  start
Abort trap: 6

@shelhamer
Copy link
Member

@roseperrone

[ RUN ] BlobSimpleTest/0.TestPointersCPUGPU
F1009 07:15:19.080399 2099200784 syncedmem.cpp:91] CPU-only Mode: cannot make GPU call.

This shouldn't happen -- it indicates that a GPU test is being called from CPU mode. Did you make all once before setting the CPU_ONLY flag? I just did a quick test clone on OS X 10.9 and everything went right so perhaps try a make clean and make && make test && make runtest.

@shelhamer
Copy link
Member

The installation instructions for have been updated for the boost and boost-python split in cb60a41 with a warning about NVCC and boost 1.56 at least on OS X.

Note to self: check out -D__STRICT_ANSI__ for clang++.

@BradNeuberg
Copy link

I'm still seeing these same problems, even with the updates you made to the README.md file. I'm on Mac OS X 10.9.5 using brew and anaconda to set everything up.

@BradNeuberg
Copy link

Note that I could only get it working by setting CPU_ONLY := 1 in the Makefile. I also had the Nvidia CUDA package installed.

@BradNeuberg
Copy link

All the unit tests passed except these in my setup:

[  FAILED  ] 3 tests, listed below:
[  FAILED  ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[  FAILED  ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU
[  FAILED  ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU

@BradNeuberg
Copy link

I lied; it doesn't actually work, even with the GPU support turned off. When I tried to train using the MNIST data it failed as follows:

bradneuberg: caffe (master) ./examples/mnist/train_lenet.sh
I1012 11:44:40.661694 2007946000 caffe.cpp:99] Use GPU with device ID 0
F1012 11:44:40.662529 2007946000 common.cpp:53] CPU-only Mode: cannot make GPU call.
*** Check failure stack trace: ***
@ 0x100d8e88a google::LogMessage::Fail()
@ 0x100d8daa8 google::LogMessage::SendToLog()
@ 0x100d8e51a google::LogMessage::Flush()
@ 0x100d92008 google::LogMessageFatal::~LogMessageFatal()
@ 0x100d8ed25 google::LogMessageFatal::~LogMessageFatal()
@ 0x100c0c516 caffe::Caffe::SetDevice()
@ 0x100bca683 train()
@ 0x100bcc871 main
@ 0x7fff892345fd start
@ 0x3 (unknown)
./examples/mnist/train_lenet.sh: line 3: 46528 Abort trap: 6 ./build/tools/caffe train --solver=examples/mnist/lenet_solver.prototxt

@shelhamer
Copy link
Member

@BradNeuberg there are a few conflated issues in your comments:

Please follow-up with usage questions on the caffe-users mailing list.

@sanjeev87
Copy link
Author

The issue of the three tests failing has been reported by multiple users now and it is now being followed in multiple threads..

This is the place in the mailing list that the discussion is on https://groups.google.com/forum/#!searchin/caffe-users/making$20runtest/caffe-users/TcCx-qjTLp8/2WfQHSOrlmUJ

[ FAILED ] 3 tests, listed below:
[ FAILED ] PowerLayerTest/0.TestPowerGradientShiftZero, where TypeParam = caffe::FloatCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradient, where TypeParam = caffe::DoubleCPU
[ FAILED ] PowerLayerTest/1.TestPowerGradientShiftZero, where TypeParam = caffe::DoubleCPU

@BradNeuberg
Copy link

Thanks for responding so quickly! I've installed Boost and boost-python 1.56 and made sure that libstdc++ is used; note that in the README it doesn't say to edit the brew config file for boost-python. Not sure if that is a README bug. I've tried editing that file to ensure boost-python doesn't use clang and it doesn't seem to work.

I'll try the other things you've mentioned to see if I can get it working. Thanks for your help!

@mprat
Copy link
Contributor

mprat commented Nov 14, 2014

@BradNeuberg I just made it work on my machine using MKL and Boost 1.55. Make sure you are really careful about your BLAS_LIBS and BLAS_INCLUDE in the Makefile.config when switching to MKL.

@beejisbrigit
Copy link

I am also running into the same issues @BradNeuberg when doing a fresh "newbie" install of caffe. Is there a cleaner way to revert to boost 1.55, other than running "brew edit boost" and changing the url/sha1 for the package version? "brew install boost 1.55" and some variations of 1.55.## didn't work. Switching the package url does appear to be working however.

@mprat
Copy link
Contributor

mprat commented Nov 19, 2014

cd /usr/local
git checkout a252214 /usr/local/Library/Formula/boost.rb
brew install --build-from-source --with-python --fresh -vd boost

(I put it here: https://github.com/BVLC/caffe/wiki/Installation-(OSX) for the future)

@svanschalkwyk
Copy link

Comments about Boost version are red herrings. I ran into the same issues. They disappeared when I included Python into the build.
Runtest now executes cleanly.

@houxianxu
Copy link

When I run git checkout a252214 /usr/local/Library/Formula/boost.rb, there is an error.
error: pathspec 'a252214' did not match any file(s) known to git.

Any idea about it, thanks!

@mprat
Copy link
Contributor

mprat commented Jul 9, 2015

I actually had this problem myself recently - it looks like someone went through the homebrew commit history and the commit has changed. If you go through and look for the commit hash for boost.rb that is for version 1.55, you can find the right hash. Alternatively, you can do the following:

brew edit boost

And change the first two lines (the url and sha256 ones) to:

url "https://downloads.sourceforge.net/project/boost/boost/1.57.0/boost_1_57_0.tar.bz2"
sha1 "e151557ae47afd1b43dc3fac46f8b04a8fe51c12"

The newest version of caffe should work with boost 1.57 in this configuration - no need to revert back to boost 1.55.

@ghost
Copy link

ghost commented Oct 16, 2019

how to get rid of this problem
LD -o .build_release/lib/libcaffe.so.1.0.0
ld: library not found for -lboost_python
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [.build_release/lib/libcaffe.so.1.0.0] Error 1
please someone help, I did everything..I am on Mac Mojave

@zimenglan-sysu-512
Copy link

hi @mekong0404
did u solve it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants