Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This program requires version 3.3.0 of the Protocol Buffer runtime library #5711

Open
thomy800 opened this issue Jun 23, 2017 · 33 comments
Open

Comments

@thomy800
Copy link

Hi. I installed caffe as described on https://github.com/BVLC/caffe/wiki/Ubuntu-16.04-or-15.10-Installation-Guide. I set WITH_PYTHON_LAYER := 1. Caffe runs fine except when I am trying to load a python layer. It then complains:
I0621 13:38:52.346406 21139 layer_factory.hpp:77] Creating layer data
2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609]
[libprotobuf FATAL google/protobuf/stubs/common.cc:61] This program requires version 3.3.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "google/protobuf/descriptor.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): This program requires version 3.3.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "google/protobuf/descriptor.pb.cc".)
*** Aborted at 1498045132 (unix time) try "date -d @1498045132" if you are using GNU date ***
PC: @ 0x7f48153ec428 gsignal
*** SIGABRT (@0x3e800005293) received by PID 21139 (TID 0x7f4818061ac0) from PID 21139; stack trace: ***
@ 0x7f48153ec4b0 (unknown)
@ 0x7f48153ec428 gsignal
@ 0x7f48153ee02a abort
@ 0x7f481620084d __gnu_cxx::__verbose_terminate_handler()
@ 0x7f48161fe6b6 (unknown)
@ 0x7f48161fe701 std::terminate()
@ 0x7f48161fe919 __cxa_throw
@ 0x7f4816744647 google::protobuf::internal::LogMessage::Finish()
@ 0x7f481674487d google::protobuf::internal::VerifyVersion()
@ 0x7f47e177de34 google::protobuf::protobuf_google_2fprotobuf_2fdescriptor_2eproto::TableStruct::InitDefaultsImpl()
@ 0x7f4816744f75 google::protobuf::GoogleOnceInitImpl()
@ 0x7f47e1779475 google::protobuf::protobuf_google_2fprotobuf_2fdescriptor_2eproto::InitDefaults()
@ 0x7f47e17794a9 google::protobuf::protobuf_google_2fprotobuf_2fdescriptor_2eproto::AddDescriptorsImpl()
@ 0x7f4816744f75 google::protobuf::GoogleOnceInitImpl()
@ 0x7f47e1779525 google::protobuf::protobuf_google_2fprotobuf_2fdescriptor_2eproto::AddDescriptors()
@ 0x7f4817ed26ba (unknown)
@ 0x7f4817ed27cb (unknown)
@ 0x7f4817ed78e2 (unknown)
@ 0x7f4817ed2564 (unknown)
@ 0x7f4817ed6da9 (unknown)
@ 0x7f48032abf09 (unknown)
@ 0x7f4817ed2564 (unknown)
@ 0x7f48032ac571 (unknown)
@ 0x7f48032abfa1 dlopen
@ 0x7f4815a7488d _PyImport_GetDynLoadFunc
@ 0x7f4815ae34be _PyImport_LoadDynamicModule
@ 0x7f4815ae4300 (unknown)
@ 0x7f4815ae45c8 (unknown)
@ 0x7f4815ae56db PyImport_ImportModuleLevel
@ 0x7f4815a5c698 (unknown)
@ 0x7f4815aa81e3 PyObject_Call
@ 0x7f4815b7e447 PyEval_CallObjectWithKeywords
./train.sh: line 1: 21139 Aborted

I investigated three possible packages:
1] pips protobuf: originally 2.6 was installed. I changed to 3.3.0, recompiled caffe, still the same problem
2] libprotobuf-dev: it claims to be libprotobuf10 amd64 3.1.0
3] protobuf-compiler: it claims to be 2.6.1-1.3, a web search suggested there is currently a version maximum of 3.1.0

Any advice is highly appreciated.
Cheers,
Thomy800

@ZhipengLiu6
Copy link

Have you solved this problem?

@thomy800
Copy link
Author

thomy800 commented Jul 3, 2017

Unfortunately not...

@aalennku
Copy link

This problem seems not caused by caffe, since i used to run caffe on protoc 2.6.1

I got this problem after install tensorflow, ant not solved

@aalennku
Copy link

aalennku commented Jul 11, 2017

Hey, the problem is solved by uninstalling the protobuf (version 3.3) :
pip2 uninstall protobuf
pip3 uninstall protobuf

至于 这么干以后tensorflow会不会崩,今朝有酒今朝醉吧。

@thomy800
Copy link
Author

Uninstalling didn't work for me:

I0711 09:52:17.679932 10729 layer_factory.hpp:77] Creating layer data
Traceback (most recent call last):
File "/home/.../test.py", line 1, in
import caffe
File ".../caffe/init.py", line 1, in
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
File ".../caffe/pycaffe.py", line 15, in
import caffe.io
File ".../caffe/io.py", line 8, in
from caffe.proto import caffe_pb2
File ".../caffe/proto/caffe_pb2.py", line 6, in
from google.protobuf.internal import enum_type_wrapper
ImportError: No module named protobuf.internal

@jasjuang
Copy link
Contributor

sudo pip install protobuf solves it for me

@VitoChien
Copy link

VitoChien commented Jul 30, 2017

This problem should be caused by the old version of lib installed by apt-get. Try to do
sudo apt-get remove --purge libprotobuf-dev

@thomy800
Copy link
Author

What I did:

sudo apt-get remove --purge libprotobuf-dev
sudo apt-get install libprotobuf-dev
cd caffe-root-path
make clean
make all -j8
make pycaffe

The warning changed a bit:

This program requires version 3.3.0 of the Protocol Buffer runtime library, but the installed version is 3.1.0. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "google/protobuf/descriptor.pb.cc".)

So instead of 2.6.1 I now have 3.1.0 installed. However protobuf does not seem to be satisfied. I was looking for 3.3.0 and found this. But I could not install it:

sudo add-apt-repository ppa:maarten-fonville/protobuf
sudo apt-get update
sudo apt-get install libprotobuf-dev=3.3.0

E: Version ‘3.3.0’ for ‘libprotobuf-dev’ was not found

So.. still no protobuf 3.3.0...

@VitoChien
Copy link

VitoChien commented Jul 31, 2017

@thomy800
What happened is that the repository that your apt-get gets things from doesn't have the protobuf 3.3.0. I advise you to build protobuf from source.Here is the link.
https://github.com/google/protobuf/tree/master/src
It seems that click on the url doesn't guide to the page.Just copy it and paste it on the browser address bar and open it.

@thomy800
Copy link
Author

Well I did as you suggested. I installed from source, however it seems to be version 3.3.2. I am not sure, though.
Anyway, when I am trying to compile caffe again, caffe failes to handle the new protobuf:

sudo make all
PROTOC src/caffe/proto/caffe.proto
protoc: error while loading shared libraries: libprotoc.so.13: cannot open shared object file: No such file or directory
Makefile:638: recipe for target '.build_release/src/caffe/proto/caffe.pb.cc' failed
make: *** [.build_release/src/caffe/proto/caffe.pb.cc] Error 127

@VitoChien
Copy link

VitoChien commented Jul 31, 2017

@thomy800
If you followed all steps in the Installation and did the last step in the it.which is
sudo ldconfig # refresh shared library cache

Then the problem should be that the path where the lib is is not included in the Makefile.config. Follow these steps:

  1. open a terminal and input:
    sudo find / -name "libprotoc.so.13"

this will list the libprotoc.so.13 file existing in your computer.Ignore the one that in the path that built the source, I got this in my computer:

/usr/local/lib/libprotoc.so.13

  1. Edit Makefile.config. Add the path that you got in the previous step in the LIBRARY_DIRS. For me , I added the '/usr/local/lib/'

3.Try to make it again.

@thomy800
Copy link
Author

sudo ldconfig
was missing. I rebuild caffe. Next error:

I0731 13:51:12.636873 17780 layer_factory.hpp:77] Creating layer loss
*** Aborted at 1501501873 (unix time) try "date -d @1501501873" if you are using GNU date ***
PC: @ 0x7f2d417d3b73 std::_Hashtable<>::clear()
*** SIGSEGV (@0x9) received by PID 17780 (TID 0x7f2d78255ac0) from PID 9; stack trace: ***
@ 0x7f2d754964b0 (unknown)
@ 0x7f2d417d3b73 std::_Hashtable<>::clear()
@ 0x7f2d417c4ca6 google::protobuf::DescriptorPool::FindFileByName()
@ 0x7f2d417a1dc8 google::protobuf::python::cdescriptor_pool::AddSerializedFile()
@ 0x7f2d75b007d0 PyEval_EvalFrameEx
@ 0x7f2d75c2901c PyEval_EvalCodeEx
@ 0x7f2d75b7f3dd (unknown)
@ 0x7f2d75b521e3 PyObject_Call
@ 0x7f2d75b72ae5 (unknown)
@ 0x7f2d75b09123 (unknown)
@ 0x7f2d75b521e3 PyObject_Call
@ 0x7f2d75afd13c PyEval_EvalFrameEx
@ 0x7f2d75c2901c PyEval_EvalCodeEx
@ 0x7f2d75af7b89 PyEval_EvalCode
@ 0x7f2d75b8c1b4 PyImport_ExecCodeModuleEx
@ 0x7f2d75b8cb8f (unknown)
@ 0x7f2d75b8e300 (unknown)
@ 0x7f2d75b8e5c8 (unknown)
@ 0x7f2d75b8f6db PyImport_ImportModuleLevel
@ 0x7f2d75b06698 (unknown)
@ 0x7f2d75b521e3 PyObject_Call
@ 0x7f2d75c28447 PyEval_CallObjectWithKeywords
@ 0x7f2d75afb5c6 PyEval_EvalFrameEx
@ 0x7f2d75c2901c PyEval_EvalCodeEx
@ 0x7f2d75af7b89 PyEval_EvalCode
@ 0x7f2d75b8c1b4 PyImport_ExecCodeModuleEx
@ 0x7f2d75b8cb8f (unknown)
@ 0x7f2d75b8e300 (unknown)
@ 0x7f2d75b8e5c8 (unknown)
@ 0x7f2d75b8f6db PyImport_ImportModuleLevel
@ 0x7f2d75b06698 (unknown)
@ 0x7f2d75b521e3 PyObject_Call
./train.sh: line 8: 17780 Segmentation fault (core dumped)

@VitoChien
Copy link

VitoChien commented Jul 31, 2017

@thomy800 I've never met the problem before. I guess it may be caused by the higher version of protobuf. Try to build the v3.3.0 version. There are steps that you can switch the version:
git clone --recursive https://github.com/google/protobuf
cd ./protobuf
git checkout v3.3.0
then redo the steps in the installation.

May you need wait for another solution or google it if this help nothing.

@thomy800
Copy link
Author

Well, I did so:

protoc --version
libprotoc 3.3.0

However, still the same error:

I0731 15:34:01.122179 19043 layer_factory.hpp:77] Creating layer loss
*** Aborted at 1501508041 (unix time) try "date -d @1501508041" if you are using GNU date ***
PC: @ 0x7f0b345a4b73 std::_Hashtable<>::clear()
*** SIGSEGV (@0x9) received by PID 19043 (TID 0x7f0b6b021ac0) from PID 9; stack trace: ***
@ 0x7f0b682674b0 (unknown)
@ 0x7f0b345a4b73 std::_Hashtable<>::clear()
@ 0x7f0b34595ca6 google::protobuf::DescriptorPool::FindFileByName()
@ 0x7f0b34572dc8 google::protobuf::python::cdescriptor_pool::AddSerializedFile()
@ 0x7f0b688d17d0 PyEval_EvalFrameEx
@ 0x7f0b689fa01c PyEval_EvalCodeEx
...

It really looked promising, but now it's getting annoying...

@moustaphakaraki
Copy link

Your tips led me in the wrong direction. If you check Caffe, it doesn't really need any protobuf version above 2.6.1, but only started asking for it after TensorFlow was installed. It turned out that it was because TensorFlow somehow upgraded my python protobuf. How I solved the problem was:
pip install -Iv protobuf==2.6.1
I moved TensorFlow to its own virtual environment. Now I have protobuf 2.5 via apt-get, and 2.6.1 via pip. Everything is back to normal.

@angleboy8
Copy link

pip install --user --upgrade protobuf==3.1.0.post1
For above problem,I solved it by "#5357". I hope these will be helpful for you.

@hieudt-pixelz
Copy link

Problem is that when building Caffe, the C++ version of google protobuff is 2.6.1 with this command:
sudo apt-get install libprotobuf-dev
But when install libraries for python with requirements.txt, has this line:
protobuf>=2.5.0
So the latest version of protobuf (3.3 as of now) is installed.
Just remove the protobuf 3.3.0, change the above line to
protobuf=2.6.1
in the requirement.txt and install protobuf again.

P/S: I did as mkaraki48 but not able to fix this error.
But as the command above suggests, my solution is the same as his (not sure about the -Iv option in his command).

@gerald4
Copy link

gerald4 commented Aug 19, 2017

I have the same problem. I installed caffe, it was working fine but a day, due to some installations i go an error

import caffe
[libprotobuf FATAL google/protobuf/stubs/common.cc:61] This program requires version 3.2.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "google/protobuf/descriptor.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): This program requires version 3.2.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "google/protobuf/descriptor.pb.cc".)
Abandon (core dumped)

@ginobilinie
Copy link

I think this is because tensorflow ask you to install protobuf 3.3.0, but caffe uses protobuf 2.x (both 2.5.0 or 2.6.1 are okay).
Solution, uninstall prootbuf 3.3.0, and then install protobuf 3.0a3 or 3.1.x
pip uninstall protobuf
sudo pip install -Iv protobuf==3.0a3

@RezaFCM
Copy link

RezaFCM commented Sep 11, 2017

It worked for me after installing "libprotobuf-dev":
pkcon install libprotobuf-dev

@yungs2017
Copy link

yungs2017 commented Sep 12, 2017

The problem may cause by the version installed by 'sudo get-apt' is not as same as the 'pip install' version.
in my env. the apt install version is 2.6.1, but the pip install version is 3.4.0, so i uninstall the pip version use
pip uninstall protobuf
and then install the 2.6.1 version
pip uninstall protobuf==2.6.1
it works well, mybe helpful
ps. if you install the protobuf by 'sudo pip protobuf' you will also need to remove it as
sudo pip uninstall protobuf

@Spbohai
Copy link

Spbohai commented Oct 27, 2017

so this problem not solved yet? still don't know how to fix it

@feipanir
Copy link

feipanir commented Oct 30, 2017

@Spbohai I repaired this problem according to @yungs2017(there is a small typo on the 5th line).
In my situation, I am not root in my linux system. So every time I need use pip install --user to install python package. Because I install caffe & tensorflow in the same home path, this problem happened to me. My "apt install" version is 2.5.0, and my 'pip install' version on my .local is 3.3.0. It is solved by:

  1. pip uninstall protobuf==3.3.0 (uninstall ver3.3.0 from my .local path)
  2. pip install protobuf==2.5.0 --user (install ver2.5.0 onmy .local path)

@Spbohai
Copy link

Spbohai commented Oct 30, 2017

@feipan664 Thanks a lot, I solved by:
pip install -Iv protobuf==2.6.1

@cciao
Copy link

cciao commented Nov 2, 2017

yep, I have the same trouble.

  • libprotoc 3.4.0
  • ubuntu 16.04

I use gdb to get the backtrace, and it shows:

[New LWP 5046]
[New LWP 5070]
[New LWP 5067]
[New LWP 5068]
[New LWP 5071]
[New LWP 5066]
[New LWP 5072]
[New LWP 5069]
[New LWP 5074]
[New LWP 5073]
[New LWP 5076]
[New LWP 5075]
[New LWP 5080]
[New LWP 5077]
[New LWP 5078]
[New LWP 5079]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `python tf.py'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f0a010e8516 in google::protobuf::internal::Mutex::Lock() ()
from /usr/lib/x86_64-linux-gnu/libprotobuf-lite.so.9
[Current thread is 1 (Thread 0x7f0add9b3700 (LWP 5046))]

In another machine with libprotoc 2.6, it works well.
And one more thing, in the machine with 3.4.0, it's ok sometimes.

I want to use caffe and tensorboard (required proto3). So what can I do?
Any idea would be appreciated.

@ShangxuanWu
Copy link

I followed what @hieudt-pixelz said and it totally works! Thanks!!

@double-vane
Copy link

@ShangxuanWu 你好,你按照hieudt-pixelz做的有效,是指tensorflow和caffe都可以正常运行吗?还是只是解决了caffe这个问题,tensorflow还能正常运行吗?谢谢!

@ruizcrp
Copy link

ruizcrp commented May 18, 2018

I'm having a very similar issue in this triangle between caffe, tensorflow and python. But it is currently saying that version 3.5 is required and that 3.1 is installed. I tried all of the above but it didn't work. When I do proto --version it says 3.5.1. And on pip I installed 3.5.2. I tried to downgrade without that it affected this message of 3.1.

@chensonglu
Copy link

Thanks. @hieudt-pixelz @mkaraki48 It works. However, for tensorflow 1.8.0, it needs protobuf>=3.4.0. So i have to downgrade the tensorflow or other ways. Anyway, good answer.

@ujsyehao
Copy link

ujsyehao commented Jun 29, 2018

Can caffe support protobuf 3.6 version?
It can support, please refer to #6359

@Shivesh4680
Copy link

Hi,

I have written a simple python script which has only one line "import tensorflow as tf".

I am calling the same script from my c++ code (Using Python.h header to call python script from c++ code).

I am getting following error related to libprotobuf.

[libprotobuf FATAL google/protobuf/stubs/common.cc:61] This program requires version 3.5.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "external/protobuf_archive/src/google/protobuf/any.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): This program requires version 3.5.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "external/protobuf_archive/src/google/protobuf/any.pb.cc".)
Aborted (core dumped)

I have tried following things but still it didn't solve my problem.

  1. sudo pip install protobuf==3.6.1
  2. Installing libprotobuf-dev
    Refer link : "https://launchpad.net/ubuntu/disco/arm64/libprotobuf-dev/3.6.1.3-1"
  3. Installing libprotobuf library for c++ from its source code (I installed version 3.5.0)
    Refer link : "https://stackoverflow.com/questions/43937682/upgrading-protobuf-tensorflow-on-jetson-tx2"

For your information, I am using Nvidia Jetson TX2 and I have installed tensorflow using command "pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp33 tensorflow-gpu".

Please help me solve the above issue. Thanks in advance.

@eightmillimeter
Copy link

Hi all,

When I run the following script:
tensorflowjs_converter --input_format=tf_frozen_model --output_node_names=final_result output/saved_model.pb web_model

I get the same error as you:
[libprotobuf FATAL external/protobuf_archive/src/google/protobuf/stubs/common.cc:68] This program requires version 3.10.0 of the Protocol Buffer runtime library, but the installed version is 3.4.0. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "google/protobuf/descriptor.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): This program requires version 3.10.0 of the Protocol Buffer runtime library, but the installed version is 3.4.0. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "google/protobuf/descriptor.pb.cc".)
Aborted (core dumped)

I uninstall protobuf and reinstalled. After installation I can see that my version is 3.10.0:
Installing collected packages: six, setuptools, protobuf
Successfully installed protobuf-3.10.0 setuptools-41.6.0 six-1.12.0

However, when I run my script I still get the same error.

Any idea to solve this?

Regards

@SWHL
Copy link

SWHL commented Oct 28, 2020

I have met the same question. I solved it by uninstall protobuf, and then install it by conda install protobuf.
I found the conda can better solve the dependency problem between packages.
The output log:

sh-4.2$ conda install protobuf
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.4.10
  latest version: 4.9.1

Please update conda by running

    $ conda update -n base conda



## Package Plan ##

  environment location: /home/xxxx/anaconda3/envs/tf_py

  added / updated specs: 
    - protobuf


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    protobuf-3.4.0             |           py36_0         475 KB  https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
    libprotobuf-3.4.0          |                0         4.3 MB  https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
    six-1.10.0                 |           py36_0          19 KB  https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
    ------------------------------------------------------------
                                           Total:         4.8 MB

The following NEW packages will be INSTALLED:

    libprotobuf: 3.4.0-0       https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
    protobuf:    3.4.0-py36_0  https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
    six:         1.10.0-py36_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free

Proceed ([y]/n)? y


Downloading and Extracting Packages
protobuf 3.4.0: ############################################################################################################################################################ | 100% 
libprotobuf 3.4.0: ######################################################################################################################################################### | 100% 
six 1.10.0: ################################################################################################################################################################ | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests