New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python3 support #293

Closed
shiquanwang opened this Issue Apr 4, 2014 · 9 comments

Comments

Projects
None yet
8 participants
@shiquanwang

shiquanwang commented Apr 4, 2014

As listed in requirements.txt:

Cython>=0.19.2
h5py>=2.2.0
ipython>=1.1.0
leveldb>=0.191
matplotlib>=1.3.1
networkx>=1.8.1
nose>=1.3.0
numpy>=1.7.1
pandas>=0.12.0
protobuf>=2.5.0
python-gflags>=2.0
scikit-image>=0.9.3
scikit-learn>=0.14.1
scipy>=0.13.2

With my limited search, Python3 support of protobuf and python-gflags are not known.

It is promising, anyone got an idea or a successful build.

@shelhamer shelhamer added the wontfix label Apr 19, 2014

@shelhamer shelhamer changed the title from Any idea about Python3 support to Python3 support Apr 19, 2014

@shelhamer

This comment has been minimized.

Show comment
Hide comment
@shelhamer

shelhamer Apr 19, 2014

Member

We have no official plan to support python3. However, any adventurous souls are welcome to comment with their installation attempts.

Member

shelhamer commented Apr 19, 2014

We have no official plan to support python3. However, any adventurous souls are welcome to comment with their installation attempts.

@shelhamer shelhamer closed this Apr 19, 2014

@inferrna

This comment has been minimized.

Show comment
Hide comment
@inferrna

inferrna Oct 6, 2015

@shiquanwang
Ubuntu 15.04, got it working. Steps:

  1. Install alpha version of python3-protobuf:
    ( protocolbuffers/protobuf#9 )
    sudo pip3 install --pre protobuf
  2. Install other requirements. In my case:
    python3-gflags python3-skimage
  3. Select python version in cmake:
    cmake-gui ../
    then set "Advanced" checkbox to see and modify python related parameters.

inferrna commented Oct 6, 2015

@shiquanwang
Ubuntu 15.04, got it working. Steps:

  1. Install alpha version of python3-protobuf:
    ( protocolbuffers/protobuf#9 )
    sudo pip3 install --pre protobuf
  2. Install other requirements. In my case:
    python3-gflags python3-skimage
  3. Select python version in cmake:
    cmake-gui ../
    then set "Advanced" checkbox to see and modify python related parameters.
@adnan15110

This comment has been minimized.

Show comment
Hide comment
@adnan15110

adnan15110 Oct 29, 2015

@inferrna
I am following your instruction to install it python3. everything compiled successfully but when I tried to import caffe it thows following error.

Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
File "", line 1, in
File "/library_install_dir/caffe/python/caffe/init.py", line 3, in
from .proto.caffe_pb2 import TRAIN, TEST
File "/library_install_dir/caffe/python/caffe/proto/caffe_pb2.py", line 977, in
has_default_value=True, default_value=unicode("constant", "utf-8"),
NameError: name 'unicode' is not defined

It looks like some python2 and python3 unicode compitability error. anyone having similar issues?

adnan15110 commented Oct 29, 2015

@inferrna
I am following your instruction to install it python3. everything compiled successfully but when I tried to import caffe it thows following error.

Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
File "", line 1, in
File "/library_install_dir/caffe/python/caffe/init.py", line 3, in
from .proto.caffe_pb2 import TRAIN, TEST
File "/library_install_dir/caffe/python/caffe/proto/caffe_pb2.py", line 977, in
has_default_value=True, default_value=unicode("constant", "utf-8"),
NameError: name 'unicode' is not defined

It looks like some python2 and python3 unicode compitability error. anyone having similar issues?

@inferrna

This comment has been minimized.

Show comment
Hide comment
@inferrna

inferrna Oct 30, 2015

Check your cmake options. The file caffe/proto/caffe_pb2.py is generated by build process and my version haven't any call to unicode().

inferrna commented Oct 30, 2015

Check your cmake options. The file caffe/proto/caffe_pb2.py is generated by build process and my version haven't any call to unicode().

@CyanVladimirS

This comment has been minimized.

Show comment
Hide comment
@CyanVladimirS

CyanVladimirS Feb 25, 2016

@inferrna

which version of protobuf did you use? Because current version doesn't work with Python 3, I got this error:
Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/google/protobuf/internal/python_message.py", line 1091, in MergeFromString
if self._InternalParse(serialized, 0, length) != length:
File "/usr/local/lib/python3.4/dist-packages/google/protobuf/internal/python_message.py", line 1113, in InternalParse
(tag_bytes, new_pos) = local_ReadTag(buffer, pos)
File "/usr/local/lib/python3.4/dist-packages/google/protobuf/internal/decoder.py", line 181, in ReadTag
while six.indexbytes(buffer, pos) & 0x80:
TypeError: unsupported operand type(s) for &: 'str' and 'int'

when I use protobuf version: 3.0.0b2

CyanVladimirS commented Feb 25, 2016

@inferrna

which version of protobuf did you use? Because current version doesn't work with Python 3, I got this error:
Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/google/protobuf/internal/python_message.py", line 1091, in MergeFromString
if self._InternalParse(serialized, 0, length) != length:
File "/usr/local/lib/python3.4/dist-packages/google/protobuf/internal/python_message.py", line 1113, in InternalParse
(tag_bytes, new_pos) = local_ReadTag(buffer, pos)
File "/usr/local/lib/python3.4/dist-packages/google/protobuf/internal/decoder.py", line 181, in ReadTag
while six.indexbytes(buffer, pos) & 0x80:
TypeError: unsupported operand type(s) for &: 'str' and 'int'

when I use protobuf version: 3.0.0b2

@inferrna

This comment has been minimized.

Show comment
Hide comment
@inferrna

inferrna Feb 26, 2016

First try to import it in python console:

$ python3.4
Python 3.4.3+ (default, Oct 14 2015, 16:03:50) 
[GCC 5.2.1 20151010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from google import protobuf
>>> protobuf.__version__
'3.0.0b2'

My opinion it's more like caffe problem than protobuf. Actually I'm on this fork https://github.com/naibaf7/caffe, it also might affect.

inferrna commented Feb 26, 2016

First try to import it in python console:

$ python3.4
Python 3.4.3+ (default, Oct 14 2015, 16:03:50) 
[GCC 5.2.1 20151010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from google import protobuf
>>> protobuf.__version__
'3.0.0b2'

My opinion it's more like caffe problem than protobuf. Actually I'm on this fork https://github.com/naibaf7/caffe, it also might affect.

@Algomorph

This comment has been minimized.

Show comment
Hide comment
@Algomorph

Algomorph Jun 24, 2016

Just tried it on Ubuntu 16.04, succeeded with CMake compile & import in python 3 shell.

Installed the latest protobuf bindings, which was 3.0.0b3 at time of this post. I'm not sure everything will work out, since Caffe on my platform still links against the Ubuntu-standard protobuf 2.6.1 binaries, and I'm hesitant to install 3.0.0b3.

Command to get the bindings:
sudo pip3 install protobuf==3.0.0b3

Relevant CMake options:
Boost_PYTHON_LIBRARY_DEBUG=/usr/lib/x86_64-linux-gnu/libboost_python-py35.so
Boost_PYTHON_LIBRARY_RELEASE=/usr/lib/x86_64-linux-gnu/libboost_python-py35.so
PYTHON_EXECUTABLE=/usr/bin/python3
PYTHON_INCLUDE_DIR=/usr/include/python3.5
PYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.5m.so

Algomorph commented Jun 24, 2016

Just tried it on Ubuntu 16.04, succeeded with CMake compile & import in python 3 shell.

Installed the latest protobuf bindings, which was 3.0.0b3 at time of this post. I'm not sure everything will work out, since Caffe on my platform still links against the Ubuntu-standard protobuf 2.6.1 binaries, and I'm hesitant to install 3.0.0b3.

Command to get the bindings:
sudo pip3 install protobuf==3.0.0b3

Relevant CMake options:
Boost_PYTHON_LIBRARY_DEBUG=/usr/lib/x86_64-linux-gnu/libboost_python-py35.so
Boost_PYTHON_LIBRARY_RELEASE=/usr/lib/x86_64-linux-gnu/libboost_python-py35.so
PYTHON_EXECUTABLE=/usr/bin/python3
PYTHON_INCLUDE_DIR=/usr/include/python3.5
PYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.5m.so

@CSJDeveloper

This comment has been minimized.

Show comment
Hide comment
@CSJDeveloper

CSJDeveloper Aug 15, 2016

@adnan15110
Hey, I have a same problem as you. "NameError: name 'unicode' is not defined"....
Have you solved your problem in the end, please help me out.
Thanks a lot.
And I have already checked my Makefile.config. It is correctly setted.
ANACONDA_HOME := $(HOME)/languageCode/anaconda3
PYTHON_LIBRARIES := boost_python-py34 python3.4m
PYTHON_INCLUDE := $(ANACONDA_HOME)/envs/py34/include
$(ANACONDA_HOME)/envs/py34/include/python3.4m
$(ANACONDA_HOME)/envs/py34/lib/python3.4/site-packages/numpy/core/include ....

CSJDeveloper commented Aug 15, 2016

@adnan15110
Hey, I have a same problem as you. "NameError: name 'unicode' is not defined"....
Have you solved your problem in the end, please help me out.
Thanks a lot.
And I have already checked my Makefile.config. It is correctly setted.
ANACONDA_HOME := $(HOME)/languageCode/anaconda3
PYTHON_LIBRARIES := boost_python-py34 python3.4m
PYTHON_INCLUDE := $(ANACONDA_HOME)/envs/py34/include
$(ANACONDA_HOME)/envs/py34/include/python3.4m
$(ANACONDA_HOME)/envs/py34/lib/python3.4/site-packages/numpy/core/include ....

@Laowai01

This comment has been minimized.

Show comment
Hide comment
@Laowai01

Laowai01 Feb 9, 2018

i have compiled caffe on Ubuntu 16.04 LTS both with python 2 and python, it is really a hectic process for beginners, for caffe with python 2.7 make sure that your /.local/lib/python2.7/site-packages only contains a version same as you protoc version. protoc version can be checked by entering

protoc --version

for me i got problems with python 2.7 because by mistake i installed protbuf 3.0 alpha in python 2.7 site packages, so there was a protobuf error after removing it from python 2.7 site packages everything worked fine
also install python-dev version

apt-get install the python3.x-dev

and for python 3.5 make sure you have python 3.0 alpha, if you have both python 2.7 and python 3.x in the system use pip3 otherwise use pip

pip3 install protobuf==3.0.0-alpha-3

or another protobuf such as 3.5 version of protobuf.
also do this

pip3 install python-dateutil --upgrade

after that do

make clean

make all -j8
make runtest -j8

make pycaffe

after performing make all make runtest steps don't forget to do that

sudo ldconfig

hope everything works OK for you.

Laowai01 commented Feb 9, 2018

i have compiled caffe on Ubuntu 16.04 LTS both with python 2 and python, it is really a hectic process for beginners, for caffe with python 2.7 make sure that your /.local/lib/python2.7/site-packages only contains a version same as you protoc version. protoc version can be checked by entering

protoc --version

for me i got problems with python 2.7 because by mistake i installed protbuf 3.0 alpha in python 2.7 site packages, so there was a protobuf error after removing it from python 2.7 site packages everything worked fine
also install python-dev version

apt-get install the python3.x-dev

and for python 3.5 make sure you have python 3.0 alpha, if you have both python 2.7 and python 3.x in the system use pip3 otherwise use pip

pip3 install protobuf==3.0.0-alpha-3

or another protobuf such as 3.5 version of protobuf.
also do this

pip3 install python-dateutil --upgrade

after that do

make clean

make all -j8
make runtest -j8

make pycaffe

after performing make all make runtest steps don't forget to do that

sudo ldconfig

hope everything works OK for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment