-
Notifications
You must be signed in to change notification settings - Fork 18.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problem with dynamic loading and caffe.proto #1917
Comments
There is a known issue with protobuf in loading dynamically linked libraries that all link to protobuf: https://code.google.com/p/protobuf/issues/detail?id=128. @longjon may be able to comment from his experience on this. A possible workaround is to combine your separate library-classifier combinations into a single library-classifiers arrangement with different calls for each model -- but in my own work I've only worked with a single |
Well, the point is, that it works fine until I've updated the caffe version yesterday and checked out the latest master (because of the modification of the net-constructor!). Even with the same protobuf version our implementation works with an older version of caffe on other 14.04 ubuntu systems we have. But this is a caffe version, we checked out by means of a snapshot (zip-file) more than half a year ago, so unfortunately we don't have any version number! But I already used a newer version, that also works in our framework with dynamic loading. Unfortunately I removed this trunk (before I checked out the current version) and thus I don't know its exact version. We found out, that in the new caffe version, the CMakeList files and structure changed for building the protobuf files and part, respectively (in comparison to our old caffe snapshot!). So it seems, that these changes cause our trouble. I would like to test the last 2-3 caffe master versions. Is there a way to check out these older versions? One more issue: Handling different classifiers in different libraries is a perfect way to allow for a flexible system structure, which is essential for a modular system concept. But even in this dynamic framework we only load "libcaffe.so" once which is a part of the main application. This is the usual way to handle shared objects. |
You can check out any version of the project since it is versioned through git. You can also look at our releases and pick a favorite.
#1667 overhauled the CMake build.
Right, that's sensible and fine -- what I was trying to say and why I linked the protobuf issue is that multiple libraries linked to protobuf (like Caffe classifiers) can conflict if they have a shared message. The protobuf issue suggests statically linking your classifier modules to libprotobuf. Earlier the whole Caffe project was static linked but we've switched to dynamic linking. |
You may want to try the The error you're getting does suggest that If you can produce a minimal non-working example using the |
Thanks for you fast replies. We solved our problem, which was caused by linking the "libproto.a" to our dynamic libraries. This causes the above error and is not necessary anymore with the new caffe version. Now everything is fine and works. :-) Thank you for your great support & best regards, Tom |
Yep, linking caffe against libprotobuf.a instead of libprotobuf.so could solve this issue. |
@tianzhi0549 Can you show in details how to link caffe against libprotobuf.a instead of libprotobuf.so. Thank you so much. |
@denny1108 I changed caffe's Makefile. Specifically, I added I have uploaded my Makefile to gist(https://gist.github.com/tianzhi0549/773c8dbc383c0cb80e7b). You could check it out to see what changes I made (Line 172 and 369). |
@tianzhi0549 Changing the Makefile gives: LD -o .build_debug/lib/libcaffe.so ANy ideas on how to solve this? |
@nbubis Try to run |
This is after make clean.
|
@nbubis Sorry for that. |
@tianzhi0549 thank you so much. At the end, I uninstall the old protobuf and install a new version, which solve my problem. It seems that the old protobuf on my machine can only generate one network instance for caffe. |
@denny1108 Can you please tell me which version of protobuf worked for you. I am also facing this issue. Basically when I run MATLAB + caffe code for this first time, it works fine. But rerunning the same code crashes. If I restart MATLAB, the same code runs fine again. Its painful to restart MATLAB everytime I need to rerun. (will be helpful if you could share URL/details what exactly you did to fix this issue). |
@puneetdabulya & whoever else runs into this issue: The issue seems to be memory deallocation by the caffe code. Using a static version of protobuf therefore solves the issue, since the protobuf lib is loaded separately by each instance of caffe. To solve: Uninstall any protobuf compilers you currently may have on your system. Good luck! |
@nbubis Thank you for the instructions. Those who are stuck at after following the above instructions. LD -o .build_debug/lib/libcaffe.so While installing protobuf, edit src/Makefile, in CXXFLAGS add -fPIC and recompile. It would fix this error. |
I have do as the instruction as above, but a new problem is: when I do "make matcaffe", there is something wrong, make: *** [matlab/+caffe/private/caffe_.mexa64] Error 255 |
Hi, |
Hello, After downloading protobuf C++ source code, configure, make and install, and recompiling Caffe, everything worked fine in Matlab (i.e. matcaffe), but pycaffe does not! Does anyone had a problem like this? What do you thing I can do to fix it? Thanks. |
I installed pycaffe recently. Most protobuf related issues can be solved by Hope it works. Thanks On Tuesday, March 8, 2016, NoaArbel notifications@github.com wrote:
--Puneet |
It worked! |
I'm facing with this problem. Even though I reinstalled the latest protobuf, my matlab program also crashes in second time. Please help me to resolve this error. Thank you very much! |
The root cause is some 3rd party library, such as opencv, are built with caffe.proto The opencv-contrib-dnn module should be disabled for my case. |
@raingo |
@KentChun33333 |
@raingo works for me, thanks! @KentChun33333 you could remove dnn directory from opencv_contrib/modules, and re-compile opencv with opencv_contrib again. |
@raingo workaround worked also for me, thanks! So, is this a problem on protobuf? on caffe? or in opencv? |
@agilmor It's a problem on the third part module of opencv named dnn, I think. |
Hello, The exact error is: It does work correctly if I build against a static libprotobuf.a. |
I run "make clean", after that run "make". This problem is solved. |
another possibility is that you messed up the bashrc file (when you have multiple caffe versions) |
I find the above instruction not worked and I configure protobuf with ./configure --enable-shared --with-pic, then it works. |
@dtmoodie, i have the same error, can you please elaborate me the solution. Where i have to used LD_DEBUG=libs?? |
@TomKae please guide me to solve the error. |
Sorry, we do not use caffe anymore and I do not remember how we solved
the problem. It was a long time ago!
Best regards,
Thomas
Am 24.08.2017 um 12:57 schrieb sainisanjay:
…
@TomKae <https://github.com/tomkae> please guide me to solve the error.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1917 (comment)>, or
mute the thread
<https://github.com/notifications/unsubscribe-auth/AKYAEpgK8bS_doDeG0cUUlxUiw0Hivb3ks5sbVcVgaJpZM4DjT5Y>.
--
Dr. Thomas Käster
Deep Learning Laboratory
Institut für Neuro- und Bioinformatik
Universität zu Lübeck
Ratzeburger Allee 160 (Geb. 64, Raum 18)
D-23562 Lübeck
Tel.: +49 451 8836818
Fax: +49 451 8836819
|
My solution can be found here: https://github.com/dtmoodie/docker_scripts/blob/master/deploy/Dockerfile I compile protobuf from source with the correct flags and then compile Cafe against a static protobuf. |
Thanks @TomKae @dtmoodie For your response. However, my problem is exactly solved by using following link solution. |
My solution way is that: |
After switching to the current caffe version today, dynamic loading of several libraries, which contain different classifiers for specific tasks, does not work anymore. We've got the following error message:
[libprotobuf ERROR google/protobuf/descriptor_database.cc:57] File already exists in database: caffe.proto
Furthermore we are using Ubuntu 14.04 and the following version of libprotobuf:
/usr/lib/x86_64-linux-gnu/libprotobuf.so.8
/usr/lib/x86_64-linux-gnu/libprotobuf.so.8.0.0
Before updating caffe everything was fine and allows for loading of more than one library containing a caffe net. Seems as if the part of building the caffe specific proto header and cc-file has changed. Is there any conncetion to our problem? And how can we fix our problem?
Best, Tom
The text was updated successfully, but these errors were encountered: