grpc_python_plugin is not included with pip install grpcio #5738

Closed
spf2 opened this Issue Mar 14, 2016 · 21 comments

Comments

Projects
None yet
@spf2

spf2 commented Mar 14, 2016

I ran pip install grpcio successfully, but grpc_python_plugin is not installed. Is it the case that grpc needs to be built from source to get this plugin?

@soltanmm

This comment has been minimized.

Show comment
Hide comment
@soltanmm

soltanmm Mar 14, 2016

Contributor

We have plans to package both protoc and grpc_python_plugin via PyPI by GA (maybe sooner!). For now, though, you do indeed need to get the plugin from source (and possibly, too, protoc due to our version requirements).

Contributor

soltanmm commented Mar 14, 2016

We have plans to package both protoc and grpc_python_plugin via PyPI by GA (maybe sooner!). For now, though, you do indeed need to get the plugin from source (and possibly, too, protoc due to our version requirements).

@spf2

This comment has been minimized.

Show comment
Hide comment
@spf2

spf2 Mar 14, 2016

Got it.

Well, in case anyone else runs into this, here's how to build the plugin:

$ git submodule update --init
$ make grpc_python_plugin

spf2 commented Mar 14, 2016

Got it.

Well, in case anyone else runs into this, here's how to build the plugin:

$ git submodule update --init
$ make grpc_python_plugin

@soltanmm soltanmm self-assigned this Mar 14, 2016

@soltanmm soltanmm added this to the GA milestone Mar 14, 2016

@titusz

This comment has been minimized.

Show comment
Hide comment
@titusz

titusz Apr 13, 2016

Had some fun following the grpc tutorials until I reached this problem. Are there any plans for a win32 wheel including the grpc_python_plugin?

titusz commented Apr 13, 2016

Had some fun following the grpc tutorials until I reached this problem. Are there any plans for a win32 wheel including the grpc_python_plugin?

@revantk

This comment has been minimized.

Show comment
Hide comment
@revantk

revantk Apr 14, 2016

The make grpc_python_plugin command doesn't work for me.

The error i get is:
[HOSTCXX] Compiling src/compiler/csharp_generator.cc src/compiler/csharp_generator.cc:47:43: error: no member named 'GetReflectionClassName' in namespace 'google::protobuf::compiler::csharp' using google::protobuf::compiler::csharp::GetReflectionClassName; ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ src/compiler/csharp_generator.cc:245:26: error: use of undeclared identifier 'GetReflectionClassName' "umbrella", GetReflectionClassName(service->file()), "index", ^ 2 errors generated. make: *** [/Users/revant/lib/grpc/objs/opt/src/compiler/csharp_generator.o] Error 1

I also tried installing grpc from source by calling sudo -H GRPC_PYTHON_BUILD_WITH_CYTHON=1 pip install and get an error:

`
duplicate symbol _grpc_global_wakeup_fd in:
python_build/temp.macosx-10.10-x86_64-2.7/src/core/lib/iomgr/ev_poll_and_epoll_posix.o
python_build/temp.macosx-10.10-x86_64-2.7/src/core/lib/iomgr/ev_posix.o
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

....

  We could not diagnose your build failure. Please file an issue at http://www.github.com/grpc/grpc with [Python install] in the title.`

revantk commented Apr 14, 2016

The make grpc_python_plugin command doesn't work for me.

The error i get is:
[HOSTCXX] Compiling src/compiler/csharp_generator.cc src/compiler/csharp_generator.cc:47:43: error: no member named 'GetReflectionClassName' in namespace 'google::protobuf::compiler::csharp' using google::protobuf::compiler::csharp::GetReflectionClassName; ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ src/compiler/csharp_generator.cc:245:26: error: use of undeclared identifier 'GetReflectionClassName' "umbrella", GetReflectionClassName(service->file()), "index", ^ 2 errors generated. make: *** [/Users/revant/lib/grpc/objs/opt/src/compiler/csharp_generator.o] Error 1

I also tried installing grpc from source by calling sudo -H GRPC_PYTHON_BUILD_WITH_CYTHON=1 pip install and get an error:

`
duplicate symbol _grpc_global_wakeup_fd in:
python_build/temp.macosx-10.10-x86_64-2.7/src/core/lib/iomgr/ev_poll_and_epoll_posix.o
python_build/temp.macosx-10.10-x86_64-2.7/src/core/lib/iomgr/ev_posix.o
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

....

  We could not diagnose your build failure. Please file an issue at http://www.github.com/grpc/grpc with [Python install] in the title.`
@revantk

This comment has been minimized.

Show comment
Hide comment
@revantk

revantk Apr 15, 2016

It turns out that even though the third_party folder has the correct version of protobuf, the code is using the shared protobuf library on my computer (which was an older version and had the GetUmbrellaClassName instead of GetReflectionClassName).

It was using the protobuf include found in /usr/local/include/google/protobuf instead of the third_party/protobuf/google/protobuf.

revantk commented Apr 15, 2016

It turns out that even though the third_party folder has the correct version of protobuf, the code is using the shared protobuf library on my computer (which was an older version and had the GetUmbrellaClassName instead of GetReflectionClassName).

It was using the protobuf include found in /usr/local/include/google/protobuf instead of the third_party/protobuf/google/protobuf.

@bothwellw

This comment has been minimized.

Show comment
Hide comment
@bothwellw

bothwellw Apr 20, 2016

@revantk how did you get it working ? I am getting this error trying to make the python plugin

bothwell@Ubuntu64:/opt/grpc$ make grpc_python_plugin
[HOSTLD] Linking /opt/grpc/bins/opt/grpc_python_plugin
/usr/bin/ld: warning: libprotobuf.so.10, needed by /usr/local/lib/libprotoc.so, may conflict with libprotobuf.so.9
/usr/bin/ld: /opt/grpc/libs/opt/libgrpc_plugin_support.a(python_generator.o): undefined reference to symbol 'ZN6google8protobuf2io7Printer5PrintEPKcS4_RKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES4_SC_S4_SC_S4_SC'
/usr/local/lib/libprotobuf.so.10: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:10535: recipe for target '/opt/grpc/bins/opt/grpc_python_plugin' failed
make: *** [/opt/grpc/bins/opt/grpc_python_plugin] Error 1
bothwell@Ubuntu64:/opt/grpc$

bothwellw commented Apr 20, 2016

@revantk how did you get it working ? I am getting this error trying to make the python plugin

bothwell@Ubuntu64:/opt/grpc$ make grpc_python_plugin
[HOSTLD] Linking /opt/grpc/bins/opt/grpc_python_plugin
/usr/bin/ld: warning: libprotobuf.so.10, needed by /usr/local/lib/libprotoc.so, may conflict with libprotobuf.so.9
/usr/bin/ld: /opt/grpc/libs/opt/libgrpc_plugin_support.a(python_generator.o): undefined reference to symbol 'ZN6google8protobuf2io7Printer5PrintEPKcS4_RKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES4_SC_S4_SC_S4_SC'
/usr/local/lib/libprotobuf.so.10: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:10535: recipe for target '/opt/grpc/bins/opt/grpc_python_plugin' failed
make: *** [/opt/grpc/bins/opt/grpc_python_plugin] Error 1
bothwell@Ubuntu64:/opt/grpc$

@revantk

This comment has been minimized.

Show comment
Hide comment
@revantk

revantk Apr 21, 2016

@bothwellw I didn't spend too much time on it. I just changed the grpc code to use GetUmbrellaClass instead of GetReflectionClass instead of trying to link the right libraries. Specifically src/compiler/csharp_generator.cc.

But in your case it seems like there are 2 libprotobuf.so libraries which might be a different issue. I'm not very familiar with c++ builds and dll linking but i would say try to reinstall protobuf from scratch and then try re building the plugin.

revantk commented Apr 21, 2016

@bothwellw I didn't spend too much time on it. I just changed the grpc code to use GetUmbrellaClass instead of GetReflectionClass instead of trying to link the right libraries. Specifically src/compiler/csharp_generator.cc.

But in your case it seems like there are 2 libprotobuf.so libraries which might be a different issue. I'm not very familiar with c++ builds and dll linking but i would say try to reinstall protobuf from scratch and then try re building the plugin.

@bothwellw

This comment has been minimized.

Show comment
Hide comment
@bothwellw

bothwellw Apr 22, 2016

Thanks re installing worked for me
On 21 Apr 2016 5:56 PM, "Revant" notifications@github.com wrote:

@bothwellw https://github.com/bothwellw I didn't spend too much time on
it. I just changed the grpc code to use GetUmbrellaClass instead of
GetReflectionClass instead of trying to link the right libraries.
Specifically src/compiler/csharp_generator.cc.

But in your case it seems like there are 2 libprotobuf.so libraries which
might be a different issue. I'm not very familiar with c++ builds and dll
linking but i would say try to reinstall protobuf from scratch and then try
re building the plugin.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#5738 (comment)

Thanks re installing worked for me
On 21 Apr 2016 5:56 PM, "Revant" notifications@github.com wrote:

@bothwellw https://github.com/bothwellw I didn't spend too much time on
it. I just changed the grpc code to use GetUmbrellaClass instead of
GetReflectionClass instead of trying to link the right libraries.
Specifically src/compiler/csharp_generator.cc.

But in your case it seems like there are 2 libprotobuf.so libraries which
might be a different issue. I'm not very familiar with c++ builds and dll
linking but i would say try to reinstall protobuf from scratch and then try
re building the plugin.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#5738 (comment)

@parkan

This comment has been minimized.

Show comment
Hide comment
@parkan

parkan Apr 25, 2016

Any recommendations for redistributing/packaging the plugin as a dependency with our code?

parkan commented Apr 25, 2016

Any recommendations for redistributing/packaging the plugin as a dependency with our code?

@soltanmm

This comment has been minimized.

Show comment
Hide comment
@soltanmm

soltanmm Apr 27, 2016

Contributor

@parkan Not at the moment. We'll have a new package on PyPI soon (the plan is for the next minor release).

As this is a subtask tightly coupled to the rest of #5980, I'm closing it as a duplicate.

Contributor

soltanmm commented Apr 27, 2016

@parkan Not at the moment. We'll have a new package on PyPI soon (the plan is for the next minor release).

As this is a subtask tightly coupled to the rest of #5980, I'm closing it as a duplicate.

@soltanmm soltanmm closed this Apr 27, 2016

@jaytaylor

This comment has been minimized.

Show comment
Hide comment
@jaytaylor

jaytaylor Jun 14, 2016

In hopes this will help someone else in the future! ..

If you're missing the grpc_python_plugin binary on macOS (Mac OS X?):

git clone https://github.com/google/protobuf.git
cd protobuf
./autogen.sh
./configure
make
make install
cd ..

Then:

git clone https://github.com/grpc/grpc
cd grpc
git submodule update --init --recursive
make grpc_python_plugin
cp bins/opt/grpc_python_plugin /usr/local/bin/

After this I was good to go!

jaytaylor commented Jun 14, 2016

In hopes this will help someone else in the future! ..

If you're missing the grpc_python_plugin binary on macOS (Mac OS X?):

git clone https://github.com/google/protobuf.git
cd protobuf
./autogen.sh
./configure
make
make install
cd ..

Then:

git clone https://github.com/grpc/grpc
cd grpc
git submodule update --init --recursive
make grpc_python_plugin
cp bins/opt/grpc_python_plugin /usr/local/bin/

After this I was good to go!

@parkan

This comment has been minimized.

Show comment
Hide comment
@parkan

parkan Jun 15, 2016

@jaytaylor FWIW I would avoid installing from master, so I'd revise your instructions to

git clone https://github.com/grpc/grpc
git co release-0_14_1 # this bit
cd grpc
git submodule update --init --recursive
make grpc_python_plugin
cp bins/opt/grpc_python_plugin /usr/local/bin/

grpc also builds protobuf as a dependency so you probably don't need to build it beforehand/separately (it's in third_party)

parkan commented Jun 15, 2016

@jaytaylor FWIW I would avoid installing from master, so I'd revise your instructions to

git clone https://github.com/grpc/grpc
git co release-0_14_1 # this bit
cd grpc
git submodule update --init --recursive
make grpc_python_plugin
cp bins/opt/grpc_python_plugin /usr/local/bin/

grpc also builds protobuf as a dependency so you probably don't need to build it beforehand/separately (it's in third_party)

@soltanmm

This comment has been minimized.

Show comment
Hide comment
@soltanmm

soltanmm Jun 15, 2016

Contributor

For anyone still trying to build from source, there's an easier way:
pip install grpcio-tools
python -m grpc.tools.protoc --python_out=$OUT --grpc_python_out=$OUT $YOUR_PROTOC_FLAGS_HERE

Contributor

soltanmm commented Jun 15, 2016

For anyone still trying to build from source, there's an easier way:
pip install grpcio-tools
python -m grpc.tools.protoc --python_out=$OUT --grpc_python_out=$OUT $YOUR_PROTOC_FLAGS_HERE

@parkan

This comment has been minimized.

Show comment
Hide comment
@parkan

parkan Jun 15, 2016

@soltanmm trying that w/o protobuf installed gives grpc/tools/main.cc:30:61: fatal error: google/protobuf/compiler/command_line_interface.h: No such file or directory

parkan commented Jun 15, 2016

@soltanmm trying that w/o protobuf installed gives grpc/tools/main.cc:30:61: fatal error: google/protobuf/compiler/command_line_interface.h: No such file or directory

@jaytaylor

This comment has been minimized.

Show comment
Hide comment
@jaytaylor

jaytaylor Jun 15, 2016

@parkan Thanks for the good advice!

Now when I try running the route guide python example I get:

grpc.framework.interfaces.face.face.LocalError: LocalError(code=StatusCode.UNIMPLEMENTED, details="b'Method "b\'GetFeature\'" of service "b\'routeguide.RouteGuide\'" not implemented!'")

Not sure where things have gone wrong.

edit

Still getting the same error on macOS after rebuilding from 0.14.1. And the same thing on Linux.

I've narrowed it down to python3. When I run the examples with python2 it all works fine.

Will file a separate issue for this breakage.

#6918

jaytaylor commented Jun 15, 2016

@parkan Thanks for the good advice!

Now when I try running the route guide python example I get:

grpc.framework.interfaces.face.face.LocalError: LocalError(code=StatusCode.UNIMPLEMENTED, details="b'Method "b\'GetFeature\'" of service "b\'routeguide.RouteGuide\'" not implemented!'")

Not sure where things have gone wrong.

edit

Still getting the same error on macOS after rebuilding from 0.14.1. And the same thing on Linux.

I've narrowed it down to python3. When I run the examples with python2 it all works fine.

Will file a separate issue for this breakage.

#6918

@soltanmm

This comment has been minimized.

Show comment
Hide comment
@soltanmm

soltanmm Jun 16, 2016

Contributor

@parkan did you upgrade pip? To get the binary distribution (and dodge compilation) you need to upgrade pip to at least 8.1.

Contributor

soltanmm commented Jun 16, 2016

@parkan did you upgrade pip? To get the binary distribution (and dodge compilation) you need to upgrade pip to at least 8.1.

@parkan

This comment has been minimized.

Show comment
Hide comment
@parkan

parkan Jun 16, 2016

@soltanmm thanks, seems like an ancient system pip was breaking the install on that particular machine

parkan commented Jun 16, 2016

@soltanmm thanks, seems like an ancient system pip was breaking the install on that particular machine

@soltanmm

This comment has been minimized.

Show comment
Hide comment
@soltanmm

soltanmm Jun 16, 2016

Contributor

@parkan though, the earlier error is arguably still a problem - the source distribution for grpcio-tools should compile...

Contributor

soltanmm commented Jun 16, 2016

@parkan though, the earlier error is arguably still a problem - the source distribution for grpcio-tools should compile...

@parkan

This comment has been minimized.

Show comment
Hide comment
@parkan

parkan Jun 16, 2016

Building from source now dies w/a different error:

(venv) [~]% pip install grpcio-tools -I --no-binary grpcio-tools                                 (arkadiy@molybdenum:~)
Collecting grpcio-tools
  Using cached grpcio_tools-0.14.0.tar.gz
Collecting protobuf>=3.0.0a3 (from grpcio-tools)
  Using cached protobuf-3.0.0b3-py2-none-any.whl
Collecting six>=1.9 (from protobuf>=3.0.0a3->grpcio-tools)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting setuptools (from protobuf>=3.0.0a3->grpcio-tools)
  Using cached setuptools-23.0.0-py2.py3-none-any.whl
Skipping bdist_wheel for grpcio-tools, due to binaries being disabled for it.
Installing collected packages: six, setuptools, protobuf, grpcio-tools
  Running setup.py install for grpcio-tools ... error
    Complete output from command /private/tmp/venv/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-build-9gWX7w/grpcio-tools/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-sJSg_7-record/install-record.txt --single-version-externally-managed --compile --install-headers /private/tmp/venv/include/site/python2.7/grpcio-tools:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.11-intel-2.7
    creating build/lib.macosx-10.11-intel-2.7/grpc
    copying grpc/__init__.py -> build/lib.macosx-10.11-intel-2.7/grpc
    creating build/lib.macosx-10.11-intel-2.7/grpc/tools
    copying grpc/tools/__init__.py -> build/lib.macosx-10.11-intel-2.7/grpc/tools
    copying grpc/tools/protoc.py -> build/lib.macosx-10.11-intel-2.7/grpc/tools
    running build_ext
    building 'grpc.tools.protoc_compiler' extension
    creating build/temp.macosx-10.11-intel-2.7
    creating build/temp.macosx-10.11-intel-2.7/grpc
    creating build/temp.macosx-10.11-intel-2.7/grpc/tools
    creating build/temp.macosx-10.11-intel-2.7/grpc_root
    creating build/temp.macosx-10.11-intel-2.7/grpc_root/src
    creating build/temp.macosx-10.11-intel-2.7/grpc_root/src/compiler
    creating build/temp.macosx-10.11-intel-2.7/third_party
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/ruby
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/python
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/objectivec
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/js
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/javanano
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/java
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/csharp
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/cpp
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/util
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/util/internal
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/stubs
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/io
    cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c grpc/tools/protoc_compiler.cpp -o build/temp.macosx-10.11-intel-2.7/grpc/tools/protoc_compiler.o -frtti -std=c++11
    grpc/tools/protoc_compiler.cpp:1810:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                                   ^
    grpc/tools/protoc_compiler.cpp:391:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
    static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                                ^
    grpc/tools/protoc_compiler.cpp:1875:26: warning: unused function '__Pyx_PyObject_IsTrue' [-Wunused-function]
    static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) {
                             ^
    grpc/tools/protoc_compiler.cpp:1925:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
    static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                    ^
    grpc/tools/protoc_compiler.cpp:1987:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
    static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                    ^
    grpc/tools/protoc_compiler.cpp:1098:32: warning: unused function '__Pyx_GetItemInt_Tuple_Fast' [-Wunused-function]
    static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i,
                                   ^
    grpc/tools/protoc_compiler.cpp:1113:32: warning: unused function '__Pyx_GetItemInt_Fast' [-Wunused-function]
    static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list,
                                   ^
    grpc/tools/protoc_compiler.cpp:1345:32: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                   ^
    grpc/tools/protoc_compiler.cpp:1394:27: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                              ^
    grpc/tools/protoc_compiler.cpp:1579:26: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                             ^
    10 warnings generated.
    grpc/tools/protoc_compiler.cpp:785:48: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      __pyx_t_3 = __Pyx_PyInt_From_int(protoc_main(__pyx_t_1, __pyx_v_argv)); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 39, __pyx_L1_error)
                                       ~~~~~~~~~~~ ^~~~~~~~~
    grpc/tools/protoc_compiler.cpp:1810:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                                   ^
    grpc/tools/protoc_compiler.cpp:391:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
    static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                                ^
    grpc/tools/protoc_compiler.cpp:1875:26: warning: unused function '__Pyx_PyObject_IsTrue' [-Wunused-function]
    static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) {
                             ^
    grpc/tools/protoc_compiler.cpp:1925:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
    static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                    ^
    grpc/tools/protoc_compiler.cpp:1987:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
    static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                    ^
    grpc/tools/protoc_compiler.cpp:1098:32: warning: unused function '__Pyx_GetItemInt_Tuple_Fast' [-Wunused-function]
    static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i,
                                   ^
    grpc/tools/protoc_compiler.cpp:1113:32: warning: unused function '__Pyx_GetItemInt_Fast' [-Wunused-function]
    static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list,
                                   ^
    grpc/tools/protoc_compiler.cpp:1345:32: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                   ^
    grpc/tools/protoc_compiler.cpp:1394:27: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                              ^
    grpc/tools/protoc_compiler.cpp:1579:26: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                             ^
    11 warnings generated.
    cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c grpc/tools/main.cc -o build/temp.macosx-10.11-intel-2.7/grpc/tools/main.o -frtti -std=c++11
    grpc/tools/main.cc:33:10: fatal error: 'src/compiler/python_generator.h' file not found
    #include "src/compiler/python_generator.h"
             ^
    1 error generated.
    error: command 'cc' failed with exit status 1

    ----------------------------------------
Command "/private/tmp/venv/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-build-9gWX7w/grpcio-tools/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-sJSg_7-record/install-record.txt --single-version-externally-managed --compile --install-headers /private/tmp/venv/include/site/python2.7/grpcio-tools" failed with error code 1 in /private/var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-build-9gWX7w/grpcio-tools/

parkan commented Jun 16, 2016

Building from source now dies w/a different error:

(venv) [~]% pip install grpcio-tools -I --no-binary grpcio-tools                                 (arkadiy@molybdenum:~)
Collecting grpcio-tools
  Using cached grpcio_tools-0.14.0.tar.gz
Collecting protobuf>=3.0.0a3 (from grpcio-tools)
  Using cached protobuf-3.0.0b3-py2-none-any.whl
Collecting six>=1.9 (from protobuf>=3.0.0a3->grpcio-tools)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting setuptools (from protobuf>=3.0.0a3->grpcio-tools)
  Using cached setuptools-23.0.0-py2.py3-none-any.whl
Skipping bdist_wheel for grpcio-tools, due to binaries being disabled for it.
Installing collected packages: six, setuptools, protobuf, grpcio-tools
  Running setup.py install for grpcio-tools ... error
    Complete output from command /private/tmp/venv/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-build-9gWX7w/grpcio-tools/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-sJSg_7-record/install-record.txt --single-version-externally-managed --compile --install-headers /private/tmp/venv/include/site/python2.7/grpcio-tools:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.11-intel-2.7
    creating build/lib.macosx-10.11-intel-2.7/grpc
    copying grpc/__init__.py -> build/lib.macosx-10.11-intel-2.7/grpc
    creating build/lib.macosx-10.11-intel-2.7/grpc/tools
    copying grpc/tools/__init__.py -> build/lib.macosx-10.11-intel-2.7/grpc/tools
    copying grpc/tools/protoc.py -> build/lib.macosx-10.11-intel-2.7/grpc/tools
    running build_ext
    building 'grpc.tools.protoc_compiler' extension
    creating build/temp.macosx-10.11-intel-2.7
    creating build/temp.macosx-10.11-intel-2.7/grpc
    creating build/temp.macosx-10.11-intel-2.7/grpc/tools
    creating build/temp.macosx-10.11-intel-2.7/grpc_root
    creating build/temp.macosx-10.11-intel-2.7/grpc_root/src
    creating build/temp.macosx-10.11-intel-2.7/grpc_root/src/compiler
    creating build/temp.macosx-10.11-intel-2.7/third_party
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/ruby
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/python
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/objectivec
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/js
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/javanano
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/java
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/csharp
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/compiler/cpp
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/util
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/util/internal
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/stubs
    creating build/temp.macosx-10.11-intel-2.7/third_party/protobuf/src/google/protobuf/io
    cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c grpc/tools/protoc_compiler.cpp -o build/temp.macosx-10.11-intel-2.7/grpc/tools/protoc_compiler.o -frtti -std=c++11
    grpc/tools/protoc_compiler.cpp:1810:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                                   ^
    grpc/tools/protoc_compiler.cpp:391:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
    static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                                ^
    grpc/tools/protoc_compiler.cpp:1875:26: warning: unused function '__Pyx_PyObject_IsTrue' [-Wunused-function]
    static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) {
                             ^
    grpc/tools/protoc_compiler.cpp:1925:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
    static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                    ^
    grpc/tools/protoc_compiler.cpp:1987:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
    static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                    ^
    grpc/tools/protoc_compiler.cpp:1098:32: warning: unused function '__Pyx_GetItemInt_Tuple_Fast' [-Wunused-function]
    static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i,
                                   ^
    grpc/tools/protoc_compiler.cpp:1113:32: warning: unused function '__Pyx_GetItemInt_Fast' [-Wunused-function]
    static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list,
                                   ^
    grpc/tools/protoc_compiler.cpp:1345:32: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                   ^
    grpc/tools/protoc_compiler.cpp:1394:27: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                              ^
    grpc/tools/protoc_compiler.cpp:1579:26: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                             ^
    10 warnings generated.
    grpc/tools/protoc_compiler.cpp:785:48: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
      __pyx_t_3 = __Pyx_PyInt_From_int(protoc_main(__pyx_t_1, __pyx_v_argv)); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 39, __pyx_L1_error)
                                       ~~~~~~~~~~~ ^~~~~~~~~
    grpc/tools/protoc_compiler.cpp:1810:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                                   ^
    grpc/tools/protoc_compiler.cpp:391:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
    static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                                ^
    grpc/tools/protoc_compiler.cpp:1875:26: warning: unused function '__Pyx_PyObject_IsTrue' [-Wunused-function]
    static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) {
                             ^
    grpc/tools/protoc_compiler.cpp:1925:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
    static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                    ^
    grpc/tools/protoc_compiler.cpp:1987:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
    static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                    ^
    grpc/tools/protoc_compiler.cpp:1098:32: warning: unused function '__Pyx_GetItemInt_Tuple_Fast' [-Wunused-function]
    static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i,
                                   ^
    grpc/tools/protoc_compiler.cpp:1113:32: warning: unused function '__Pyx_GetItemInt_Fast' [-Wunused-function]
    static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list,
                                   ^
    grpc/tools/protoc_compiler.cpp:1345:32: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                   ^
    grpc/tools/protoc_compiler.cpp:1394:27: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                              ^
    grpc/tools/protoc_compiler.cpp:1579:26: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                             ^
    11 warnings generated.
    cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c grpc/tools/main.cc -o build/temp.macosx-10.11-intel-2.7/grpc/tools/main.o -frtti -std=c++11
    grpc/tools/main.cc:33:10: fatal error: 'src/compiler/python_generator.h' file not found
    #include "src/compiler/python_generator.h"
             ^
    1 error generated.
    error: command 'cc' failed with exit status 1

    ----------------------------------------
Command "/private/tmp/venv/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-build-9gWX7w/grpcio-tools/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-sJSg_7-record/install-record.txt --single-version-externally-managed --compile --install-headers /private/tmp/venv/include/site/python2.7/grpcio-tools" failed with error code 1 in /private/var/folders/sm/38kkrt9n5z5524hmxqn3h8yh0000gq/T/pip-build-9gWX7w/grpcio-tools/
@evmin

This comment has been minimized.

Show comment
Hide comment
@evmin

evmin Jun 22, 2016

Fails on Atomic, too.

evmin commented Jun 22, 2016

Fails on Atomic, too.

@vikash009

This comment has been minimized.

Show comment
Hide comment
@vikash009

vikash009 Aug 25, 2016

I have used following command.

<<
-git clone https://github.com/grpc/grpc.git
-git submodule update --init
-make install prefix=/home/ubuntu/GRPC_LOCAL/latest_grpc/grpc/build/

Seeing Following Error:

<<
[CXX] Compiling src/cpp/util/time.cc
[HOSTCXX] Compiling src/compiler/cpp_generator.cc
[HOSTCXX] Compiling src/compiler/csharp_generator.cc
In file included from src/compiler/csharp_generator.cc:40:0:
./src/compiler/csharp_generator.h:39:58: fatal error: google/protobuf/compiler/csharp/csharp_names.h: No such file or directory
#include <google/protobuf/compiler/csharp/csharp_names.h>
^
compilation terminated.
make: *** [/home/ubuntu/GRPC_LOCAL/latest_grpc/grpc/objs/opt/src/compiler/csharp_generator.o] Error 1
ubuntu@ip-10-10-10-199:~/GRPC_LOCAL/latest_grpc/grpc$

After running the run_deps command

<<
ubuntu@ip-10-10-10-199:/GRPC_LOCAL/latest_grpc/grpc$ make run_dep_checks
pkg-config --atleast-version=1.0.2 openssl || true
pkg-config --atleast-version=1.0.1 openssl || true
pkg-config --exists zlib || true
cc -pthread -I/usr/local/include -Ithird_party/googletest/include -Ithird_party/boringssl/include -g -Wall -Wextra -Werror -Wno-long-long -Wno-unused-parameter -O2 -fPIC -I. -Iinclude -I/home/ubuntu/GRPC_LOCAL/latest_grpc/grpc/gens -DNDEBUG -DINSTALL_PREFIX="/usr/local" -DGRPC_HAVE_PERFTOOLS -std=c99 -Wsign-conversion -Wconversion -Wshadow -o /dev/null test/build/perftools.c -lprofiler -g -fPIC -Llibs/opt -pthread || true
pkg-config --atleast-version=3.0.0 protobuf || true
protoc --version | grep -q libprotoc.3 || true
ubuntu@ip-10-10-10-199:
/GRPC_LOCAL/latest_grpc/grpc$

one more thing::
<<
after doing locate
locate -i csharp_names.h
o/p
./grpc/third_party/protobuf/src/google/protobuf/compiler/csharp/csharp_names.h

Is any one have faced this issue ??

vikash009 commented Aug 25, 2016

I have used following command.

<<
-git clone https://github.com/grpc/grpc.git
-git submodule update --init
-make install prefix=/home/ubuntu/GRPC_LOCAL/latest_grpc/grpc/build/

Seeing Following Error:

<<
[CXX] Compiling src/cpp/util/time.cc
[HOSTCXX] Compiling src/compiler/cpp_generator.cc
[HOSTCXX] Compiling src/compiler/csharp_generator.cc
In file included from src/compiler/csharp_generator.cc:40:0:
./src/compiler/csharp_generator.h:39:58: fatal error: google/protobuf/compiler/csharp/csharp_names.h: No such file or directory
#include <google/protobuf/compiler/csharp/csharp_names.h>
^
compilation terminated.
make: *** [/home/ubuntu/GRPC_LOCAL/latest_grpc/grpc/objs/opt/src/compiler/csharp_generator.o] Error 1
ubuntu@ip-10-10-10-199:~/GRPC_LOCAL/latest_grpc/grpc$

After running the run_deps command

<<
ubuntu@ip-10-10-10-199:/GRPC_LOCAL/latest_grpc/grpc$ make run_dep_checks
pkg-config --atleast-version=1.0.2 openssl || true
pkg-config --atleast-version=1.0.1 openssl || true
pkg-config --exists zlib || true
cc -pthread -I/usr/local/include -Ithird_party/googletest/include -Ithird_party/boringssl/include -g -Wall -Wextra -Werror -Wno-long-long -Wno-unused-parameter -O2 -fPIC -I. -Iinclude -I/home/ubuntu/GRPC_LOCAL/latest_grpc/grpc/gens -DNDEBUG -DINSTALL_PREFIX="/usr/local" -DGRPC_HAVE_PERFTOOLS -std=c99 -Wsign-conversion -Wconversion -Wshadow -o /dev/null test/build/perftools.c -lprofiler -g -fPIC -Llibs/opt -pthread || true
pkg-config --atleast-version=3.0.0 protobuf || true
protoc --version | grep -q libprotoc.3 || true
ubuntu@ip-10-10-10-199:
/GRPC_LOCAL/latest_grpc/grpc$

one more thing::
<<
after doing locate
locate -i csharp_names.h
o/p
./grpc/third_party/protobuf/src/google/protobuf/compiler/csharp/csharp_names.h

Is any one have faced this issue ??

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment