ImportError: No module named etcdrpc

[durzo]

Firstly, thanks for taking the time to make this module. lots of people have been waiting for a v3 implementation.

Using Ubuntu 12.04 LTS with python 2.7.6 and pip 1.5.4. After installing via pip install etcd3 and getting many, many warning messages (but successful installation):

$ cat test3.py
#!/usr/bin/python

import etcd3
conn = etcd3.client(host='127.0.0.1')
conn.get('/test')

$ ./test3.py
Traceback (most recent call last):
  File "./test3.py", line 3, in <module>
    import etcd3
  File "/usr/local/lib/python2.7/dist-packages/etcd3/__init__.py", line 3, in <module>
    from etcd3.client import Etcd3Client
  File "/usr/local/lib/python2.7/dist-packages/etcd3/client.py", line 5, in <module>
    import etcd3.transactions as transactions
  File "/usr/local/lib/python2.7/dist-packages/etcd3/transactions.py", line 1, in <module>
    from etcd3.etcdrpc import rpc_pb2 as etcdrpc
ImportError: No module named etcdrpc

$ ls /usr/local/lib/python2.7/dist-packages/etcd3
client.py  client.pyc  exceptions.py  exceptions.pyc  __init__.py  __init__.pyc  members.py  members.pyc  transactions.py  transactions.pyc  utils.py  utils.pyc

[durzo]

After reading the docs I guess I need to run the "generating protobuf stuff" which isn't clear this is a requirement for install ?

$ python -m grpc.tools.protoc -Ietcd3/proto --python_out=etcd3/etcdrpc/ --grpc_python_out=etcd3/etcdrpc/ etcd3/proto/*.proto
/usr/bin/python: No module named grpc.tools

Seems I'm missing some stuff.. eventually figured out I had to install cython and grpcio-tools:

$ pip install cython
$ pip install grpcio-tools

$ python -m grpc.tools.protoc -Ietcd3/proto --python_out=etcd3/etcdrpc/ --grpc_python_out=etcd3/etcdrpc/ etcd3/proto/*.proto
Import grpc:No module named gevent.socket
etcd3/proto: warning: directory does not exist.
etcd3/proto/*.proto: No such file or directory

$ cd /usr/local/lib/python2.7/dist-packages/etcd3

$ python -m grpc.tools.protoc -Ietcd3/proto --python_out=etcd3/etcdrpc/ --grpc_python_out=etcd3/etcdrpc/ etcd3/proto/*.proto
Import grpc:No module named gevent.socket
etcd3/proto: warning: directory does not exist.
etcd3/proto/*.proto: No such file or directory

Now at a loss :/

[durzo]

Getting further...

Fix missing gevent.socket errors:

$ pip install gevent

Figured out I have to run the protobuf generate code from the source repo directory, not the module install dir.

$ git clone git://github.com/kragniz/python-etcd3
$ cd python-etcd3
$ python -m grpc.tools.protoc -Ietcd3/proto --python_out=etcd3/etcdrpc/ --grpc_python_out=etcd3/etcdrpc/ etcd3/proto/*.proto
$ git status
    modified:   etcd3/etcdrpc/rpc_pb2.py

What am I supposed to do with this file? where does it go? Lets try:

$ cp -r etcd3/etcdrpc /usr/local/lib/python2.7/dist-packages/etcd3/
$ ./test3.py
Traceback (most recent call last):
  File "./test3.py", line 3, in <module>
    import etcd3
  File "/usr/local/lib/python2.7/dist-packages/etcd3/__init__.py", line 3, in <module>
    from etcd3.client import Etcd3Client
  File "/usr/local/lib/python2.7/dist-packages/etcd3/client.py", line 5, in <module>
    import etcd3.transactions as transactions
  File "/usr/local/lib/python2.7/dist-packages/etcd3/transactions.py", line 1, in <module>
    from etcd3.etcdrpc import rpc_pb2 as etcdrpc
  File "/usr/local/lib/python2.7/dist-packages/etcd3/etcdrpc/rpc_pb2.py", line 17, in <module>
    import kv_pb2 as kv__pb2
  File "/usr/local/lib/python2.7/dist-packages/etcd3/etcdrpc/kv_pb2.py", line 187, in <module>
    from grpc.beta import implementations as beta_implementations
  File "/usr/local/lib/python2.7/dist-packages/grpc/beta/implementations.py", line 39, in <module>
    from grpc import _auth
  File "/usr/local/lib/python2.7/dist-packages/grpc/_auth.py", line 43, in <module>
    class GoogleCallCredentials(grpc.AuthMetadataPlugin):
AttributeError: 'module' object has no attribute 'AuthMetadataPlugin'

This is painfull :/

[durzo]

Solved!

A bit of googling informed me I had conflicts with 'grpc' and 'grpcio'. So i cleaned everything and started over:

$ rm -rf /usr/local/lib/python2.7/dist-packages/grpc*
$ rm -rf /usr/local/lib/python2.7/dist-packages/google* 
$ rm -rf /usr/local/lib/python2.7/dist-packages/cython*
$ rm -rf /usr/local/lib/python2.7/dist-packages/Cython*
$ rm -rf /usr/local/lib/python2.7/dist-packages/gevent*
$ rm -rf /usr/local/lib/python2.7/dist-packages/protobuf*

$ pip install cython
$ pip install grpcio
$ pip install grpcio-tools

$ git clone git://github.com/kragniz/python-etcd3
$ cd python-etcd3
$ python -m grpc.tools.protoc -Ietcd3/proto --python_out=etcd3/etcdrpc/ --grpc_python_out=etcd3/etcdrpc/ etcd3/proto/*.proto
$ git status
    modified:   etcd3/etcdrpc/rpc_pb2.py
$ cp -r etcd3/etcdrpc /usr/local/lib/python2.7/dist-packages/etcd3/
$ cd ~/
$ cat test3.py
#!/usr/bin/python

import etcd3
conn = etcd3.client(host='127.0.0.1')
print conn.get('/test')

$ ./test3.py
foo

Wow, all this should really be documented better.

[kragniz]

Hey, thanks for trying this out! As you can tell, it's in a very early state. You shouldn't have to generate the protobuf stubs yourself, since these should be bundled with the rest of the code.

I hadn't actually tested installing it with pip, my bad. It works okay from a git clone, but not from pypi:

(etcd3-test) ~/git $ git clone https://github.com/kragniz/python-etcd3 python-etcd-test
Cloning into 'python-etcd-test'...
remote: Counting objects: 657, done.
remote: Compressing objects: 100% (149/149), done.
remote: Total 657 (delta 88), reused 0 (delta 0), pack-reused 507
Receiving objects: 100% (657/657), 188.01 KiB | 178.00 KiB/s, done.
Resolving deltas: 100% (359/359), done.
(etcd3-test) ~/git $ pip install -e python-etcd-test/
Obtaining file:///home/louis/git/python-etcd-test
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0.0 in /tmp/etcd3-test/lib/python3.5/site-packages (from etcd3==0.2.0)
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /tmp/etcd3-test/lib/python3.5/site-packages (from grpcio>=1.0.0->etcd3==0.2.0)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.0.0 in /tmp/etcd3-test/lib/python3.5/site-packages (from grpcio>=1.0.0->etcd3==0.2.0)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /tmp/etcd3-test/lib/python3.5/site-packages (from grpcio>=1.0.0->etcd3==0.2.0)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /tmp/etcd3-test/lib/python3.5/site-packages (from grpcio>=1.0.0->etcd3==0.2.0)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /tmp/etcd3-test/lib/python3.5/site-packages (from protobuf>=3.0.0->grpcio>=1.0.0->etcd3==0.2.0)
Installing collected packages: etcd3
  Running setup.py develop for etcd3
Successfully installed etcd3
(etcd3-test) ~/git $ python
Python 3.5.2 (default, Sep 10 2016, 08:21:44) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import etcd3
>>> 

I'll have a look and see what's wrong with the version pushed to pypi.

[kragniz]

This should now be fixed with 4abdd63. I've pushed 0.2.2 to pypi, which seems to work okay.

[durzo]

Tested on a clean install, works fine now. thanks!