New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken upgrade to 0.20 #2493

Closed
jlowin opened this Issue Oct 4, 2016 · 10 comments

Comments

Projects
None yet
5 participants
@jlowin

jlowin commented Oct 4, 2016

(Continuing from #2464 (comment))

I also just upgraded from 0.19 to 0.20 with pip install --upgrade google-cloud and now can't import anything:

In [2]: from google.cloud import datastore
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-46f71c99c111> in <module>()
----> 1 from google.cloud import datastore

ImportError: cannot import name 'datastore'

I went in to site-packages and deleted every google package, then reinstalled google-cloud. Now I get this error:

In [1]: from google.cloud import datastore
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-46f71c99c111> in <module>()
----> 1 from google.cloud import datastore

/Users/jlowin/anaconda3/lib/python3.5/site-packages/google/cloud/datastore/__init__.py in <module>()
     52
     53
---> 54 from google.cloud.datastore.batch import Batch
     55 from google.cloud.datastore.connection import Connection
     56 from google.cloud.datastore.client import Client

/Users/jlowin/anaconda3/lib/python3.5/site-packages/google/cloud/datastore/batch.py in <module>()
     22 """
     23
---> 24 from google.cloud.datastore import helpers
     25 from google.cloud.datastore._generated import datastore_pb2 as _datastore_pb2
     26

/Users/jlowin/anaconda3/lib/python3.5/site-packages/google/cloud/datastore/helpers.py in <module>()
     21 import itertools
     22
---> 23 from google.protobuf import struct_pb2
     24 from google.type import latlng_pb2
     25 import six

ImportError: No module named 'google.protobuf'
@tseaver

This comment has been minimized.

Show comment
Hide comment
@tseaver

tseaver Oct 4, 2016

Member

As a baseline, installing into a fresh virtualenv works:

$ /opt/Python-2.7.11/bin/virtualenv /tmp/gcp-2493
New python executable in /tmp/gcp-2493/bin/python
Installing setuptools, pip, wheel...done.
$ /tmp/gcp-2493/bin/pip install google-cloud
Collecting google-cloud
  Downloading google_cloud-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-resource-manager>=0.20.0 (from google-cloud)
  Downloading google_cloud_resource_manager-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-dns>=0.20.0 (from google-cloud)
  Downloading google_cloud_dns-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-bigquery>=0.20.0 (from google-cloud)
  Using cached google_cloud_bigquery-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-translate>=0.20.0 (from google-cloud)
  Downloading google_cloud_translate-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-language>=0.20.0 (from google-cloud)
  Downloading google_cloud_language-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-datastore>=0.20.0 (from google-cloud)
  Downloading google_cloud_datastore-0.20.0-py2.py3-none-any.whl (91kB)
    100% |████████████████████████████████| 92kB 1.3MB/s 
Collecting google-cloud-monitoring>=0.20.0 (from google-cloud)
  Downloading google_cloud_monitoring-0.20.0-py2.py3-none-any.whl (57kB)
    100% |████████████████████████████████| 61kB 3.5MB/s 
Collecting google-cloud-vision>=0.20.0 (from google-cloud)
  Downloading google_cloud_vision-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-storage>=0.20.0 (from google-cloud)
  Downloading google_cloud_storage-0.20.0-py2.py3-none-any.whl (62kB)
    100% |████████████████████████████████| 71kB 2.7MB/s 
Collecting google-cloud-pubsub>=0.20.0 (from google-cloud)
  Downloading google_cloud_pubsub-0.20.0-py2.py3-none-any.whl (50kB)
    100% |████████████████████████████████| 51kB 1.9MB/s 
Collecting google-cloud-core>=0.20.0 (from google-cloud)
  Using cached google_cloud_core-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-error-reporting>=0.20.0 (from google-cloud)
  Downloading google_cloud_error_reporting-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-logging>=0.20.0 (from google-cloud)
  Downloading google_cloud_logging-0.20.0-py2.py3-none-any.whl (58kB)
    100% |████████████████████████████████| 61kB 2.5MB/s 
Collecting google-cloud-bigtable>=0.20.0 (from google-cloud)
  Downloading google_cloud_bigtable-0.20.0-py2.py3-none-any.whl (117kB)
    100% |████████████████████████████████| 122kB 1.2MB/s 
Collecting grpcio<2.0dev,>=1.0.0 (from google-cloud-datastore>=0.20.0->google-cloud)
  Using cached grpcio-1.0.0-cp27-cp27m-manylinux1_x86_64.whl
Collecting google-gax<0.15dev,>=0.14.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting gapic-google-pubsub-v1<0.11dev,>=0.10.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting grpc-google-pubsub-v1<0.11dev,>=0.10.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting httplib2>=0.9.1 (from google-cloud-core>=0.20.0->google-cloud)
Collecting six (from google-cloud-core>=0.20.0->google-cloud)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting protobuf>=3.0.0 (from google-cloud-core>=0.20.0->google-cloud)
  Using cached protobuf-3.1.0.post1-py2.py3-none-any.whl
Collecting googleapis-common-protos>=1.3.4 (from google-cloud-core>=0.20.0->google-cloud)
Collecting oauth2client<4.0.0dev,>=3.0.0 (from google-cloud-core>=0.20.0->google-cloud)
Collecting gapic-google-logging-v2<0.11dev,>=0.10.1 (from google-cloud-logging>=0.20.0->google-cloud)
Collecting grpc-google-logging-v2<0.11dev,>=0.10.1 (from google-cloud-logging>=0.20.0->google-cloud)
Collecting enum34>=1.0.4 (from grpcio<2.0dev,>=1.0.0->google-cloud-datastore>=0.20.0->google-cloud)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting futures>=2.2.0 (from grpcio<2.0dev,>=1.0.0->google-cloud-datastore>=0.20.0->google-cloud)
  Using cached futures-3.0.5-py2-none-any.whl
Collecting ply==3.8 (from google-gax<0.15dev,>=0.14.1->google-cloud-pubsub>=0.20.0->google-cloud)
Collecting future>=0.15.2 (from google-gax<0.15dev,>=0.14.1->google-cloud-pubsub>=0.20.0->google-cloud)
Collecting grpc-google-iam-v1<0.11.0dev,>=0.10.0 (from gapic-google-pubsub-v1<0.11dev,>=0.10.1->google-cloud-pubsub>=0.20.0->google-cloud)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /tmp/gcp-2493/lib/python2.7/site-packages (from protobuf>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Collecting rsa>=3.1.4 (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Installing collected packages: httplib2, six, protobuf, googleapis-common-protos, pyasn1, rsa, pyasn1-modules, oauth2client, google-cloud-core, google-cloud-resource-manager, google-cloud-dns, google-cloud-bigquery, google-cloud-translate, google-cloud-language, enum34, futures, grpcio, google-cloud-datastore, google-cloud-monitoring, google-cloud-vision, google-cloud-storage, ply, future, google-gax, grpc-google-pubsub-v1, grpc-google-iam-v1, gapic-google-pubsub-v1, google-cloud-pubsub, grpc-google-logging-v2, gapic-google-logging-v2, google-cloud-logging, google-cloud-error-reporting, google-cloud-bigtable, google-cloud
Successfully installed enum34-1.1.6 future-0.15.2 futures-3.0.5 gapic-google-logging-v2-0.10.1 gapic-google-pubsub-v1-0.10.1 google-cloud-0.20.0 google-cloud-bigquery-0.20.0 google-cloud-bigtable-0.20.0 google-cloud-core-0.20.0 google-cloud-datastore-0.20.0 google-cloud-dns-0.20.0 google-cloud-error-reporting-0.20.0 google-cloud-language-0.20.0 google-cloud-logging-0.20.0 google-cloud-monitoring-0.20.0 google-cloud-pubsub-0.20.0 google-cloud-resource-manager-0.20.0 google-cloud-storage-0.20.0 google-cloud-translate-0.20.0 google-cloud-vision-0.20.0 google-gax-0.14.1 googleapis-common-protos-1.3.5 grpc-google-iam-v1-0.10.1 grpc-google-logging-v2-0.10.1 grpc-google-pubsub-v1-0.10.1 grpcio-1.0.0 httplib2-0.9.2 oauth2client-3.0.0 ply-3.8 protobuf-3.1.0.post1 pyasn1-0.1.9 pyasn1-modules-0.0.8 rsa-3.4.2 six-1.10.0
$ /tmp/gcp-2493/bin/pip freeze
-f file:///<path-to-homedir>/.pip/wheels
enum34==1.1.6
future==0.15.2
futures==3.0.5
gapic-google-logging-v2==0.10.1
gapic-google-pubsub-v1==0.10.1
google-cloud==0.20.0
google-cloud-bigquery==0.20.0
google-cloud-bigtable==0.20.0
google-cloud-core==0.20.0
google-cloud-datastore==0.20.0
google-cloud-dns==0.20.0
google-cloud-error-reporting==0.20.0
google-cloud-language==0.20.0
google-cloud-logging==0.20.0
google-cloud-monitoring==0.20.0
google-cloud-pubsub==0.20.0
google-cloud-resource-manager==0.20.0
google-cloud-storage==0.20.0
google-cloud-translate==0.20.0
google-cloud-vision==0.20.0
google-gax==0.14.1
googleapis-common-protos==1.3.5
grpc-google-iam-v1==0.10.1
grpc-google-logging-v2==0.10.1
grpc-google-pubsub-v1==0.10.1
grpcio==1.0.0
httplib2==0.9.2
oauth2client==3.0.0
ply==3.8
protobuf==3.1.0.post1
pyasn1==0.1.9
pyasn1-modules==0.0.8
rsa==3.4.2
six==1.10.0
$ /tmp/gcp-2493/bin/python
Python 2.7.11 (default, Feb 22 2016, 07:44:44) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.cloud import datastore
>>> 

but an upgrade fails:

$ rm -r /tmp/gcp-2493/
$ /opt/Python-2.7.11/bin/virtualenv /tmp/gcp-2493
New python executable in /tmp/gcp-2493/bin/python
Installing setuptools, pip, wheel...done.
$ /tmp/gcp-2493/bin/pip install google-cloud==0.19.0
Collecting google-cloud==0.19.0
  Downloading google-cloud-0.19.0.tar.gz (456kB)
    100% |████████████████████████████████| 460kB 396kB/s 
Collecting httplib2>=0.9.1 (from google-cloud==0.19.0)
Collecting googleapis-common-protos (from google-cloud==0.19.0)
Collecting oauth2client>=2.0.1 (from google-cloud==0.19.0)
Collecting protobuf>=3.0.0 (from google-cloud==0.19.0)
  Using cached protobuf-3.1.0.post1-py2.py3-none-any.whl
Collecting six (from google-cloud==0.19.0)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting grpcio>=1.0.0 (from google-cloud==0.19.0)
  Using cached grpcio-1.0.0-cp27-cp27m-manylinux1_x86_64.whl
Collecting google-gax<0.14dev,>=0.13.0 (from google-cloud==0.19.0)
Collecting gapic-google-pubsub-v1<0.10dev,>=0.9.0 (from google-cloud==0.19.0)
Collecting grpc-google-pubsub-v1<0.10dev,>=0.9.0 (from google-cloud==0.19.0)
Collecting gapic-google-logging-v2<0.10dev,>=0.9.0 (from google-cloud==0.19.0)
Collecting grpc-google-logging-v2<0.10dev,>=0.9.0 (from google-cloud==0.19.0)
Collecting rsa>=3.1.4 (from oauth2client>=2.0.1->google-cloud==0.19.0)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client>=2.0.1->google-cloud==0.19.0)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client>=2.0.1->google-cloud==0.19.0)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /tmp/gcp-2493/lib/python2.7/site-packages (from protobuf>=3.0.0->google-cloud==0.19.0)
Collecting enum34>=1.0.4 (from grpcio>=1.0.0->google-cloud==0.19.0)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting futures>=2.2.0 (from grpcio>=1.0.0->google-cloud==0.19.0)
  Using cached futures-3.0.5-py2-none-any.whl
Collecting ply==3.8 (from google-gax<0.14dev,>=0.13.0->google-cloud==0.19.0)
Collecting future>=0.15.2 (from google-gax<0.14dev,>=0.13.0->google-cloud==0.19.0)
Building wheels for collected packages: google-cloud
  Running setup.py bdist_wheel for google-cloud ... done
Successfully built google-cloud
Installing collected packages: httplib2, six, protobuf, googleapis-common-protos, pyasn1, rsa, pyasn1-modules, oauth2client, enum34, futures, grpcio, ply, future, google-gax, grpc-google-pubsub-v1, gapic-google-pubsub-v1, grpc-google-logging-v2, gapic-google-logging-v2, google-cloud
Successfully installed enum34-1.1.6 future-0.15.2 futures-3.0.5 gapic-google-logging-v2-0.9.3 gapic-google-pubsub-v1-0.9.3 google-cloud-0.19.0 google-gax-0.13.0 googleapis-common-protos-1.3.5 grpc-google-logging-v2-0.9.3 grpc-google-pubsub-v1-0.9.3 grpcio-1.0.0 httplib2-0.9.2 oauth2client-3.0.0 ply-3.8 protobuf-3.1.0.post1 pyasn1-0.1.9 pyasn1-modules-0.0.8 rsa-3.4.2 six-1.10.0
$ /tmp/gcp-2493/bin/pip freeze
-f file:///<path-to-homedir>/.pip/wheels
enum34==1.1.6
future==0.15.2
futures==3.0.5
gapic-google-logging-v2==0.9.3
gapic-google-pubsub-v1==0.9.3
google-cloud==0.19.0
google-gax==0.13.0
googleapis-common-protos==1.3.5
grpc-google-logging-v2==0.9.3
grpc-google-pubsub-v1==0.9.3
grpcio==1.0.0
httplib2==0.9.2
oauth2client==3.0.0
ply==3.8
protobuf==3.1.0.post1
pyasn1==0.1.9
pyasn1-modules==0.0.8
rsa==3.4.2
six==1.10.0
$ /tmp/gcp-2493/bin/pip install --upgrade google-cloud
Collecting google-cloud
  Using cached google_cloud-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-resource-manager>=0.20.0 (from google-cloud)
  Using cached google_cloud_resource_manager-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-dns>=0.20.0 (from google-cloud)
  Using cached google_cloud_dns-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-bigquery>=0.20.0 (from google-cloud)
  Using cached google_cloud_bigquery-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-translate>=0.20.0 (from google-cloud)
  Using cached google_cloud_translate-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-language>=0.20.0 (from google-cloud)
  Using cached google_cloud_language-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-datastore>=0.20.0 (from google-cloud)
  Using cached google_cloud_datastore-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-monitoring>=0.20.0 (from google-cloud)
  Using cached google_cloud_monitoring-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-vision>=0.20.0 (from google-cloud)
  Using cached google_cloud_vision-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-storage>=0.20.0 (from google-cloud)
  Using cached google_cloud_storage-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-pubsub>=0.20.0 (from google-cloud)
  Using cached google_cloud_pubsub-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-core>=0.20.0 (from google-cloud)
  Using cached google_cloud_core-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-error-reporting>=0.20.0 (from google-cloud)
  Using cached google_cloud_error_reporting-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-logging>=0.20.0 (from google-cloud)
  Using cached google_cloud_logging-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-bigtable>=0.20.0 (from google-cloud)
  Using cached google_cloud_bigtable-0.20.0-py2.py3-none-any.whl
Requirement already up-to-date: grpcio<2.0dev,>=1.0.0 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-datastore>=0.20.0->google-cloud)
Collecting google-gax<0.15dev,>=0.14.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting gapic-google-pubsub-v1<0.11dev,>=0.10.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting grpc-google-pubsub-v1<0.11dev,>=0.10.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Requirement already up-to-date: httplib2>=0.9.1 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: six in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: protobuf>=3.0.0 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: googleapis-common-protos>=1.3.4 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: oauth2client<4.0.0dev,>=3.0.0 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Collecting gapic-google-logging-v2<0.11dev,>=0.10.1 (from google-cloud-logging>=0.20.0->google-cloud)
Collecting grpc-google-logging-v2<0.11dev,>=0.10.1 (from google-cloud-logging>=0.20.0->google-cloud)
Requirement already up-to-date: enum34>=1.0.4 in /tmp/gcp-2493/lib/python2.7/site-packages (from grpcio<2.0dev,>=1.0.0->google-cloud-datastore>=0.20.0->google-cloud)
Requirement already up-to-date: futures>=2.2.0 in /tmp/gcp-2493/lib/python2.7/site-packages (from grpcio<2.0dev,>=1.0.0->google-cloud-datastore>=0.20.0->google-cloud)
Requirement already up-to-date: ply==3.8 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-gax<0.15dev,>=0.14.1->google-cloud-pubsub>=0.20.0->google-cloud)
Requirement already up-to-date: future>=0.15.2 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-gax<0.15dev,>=0.14.1->google-cloud-pubsub>=0.20.0->google-cloud)
Collecting grpc-google-iam-v1<0.11.0dev,>=0.10.0 (from gapic-google-pubsub-v1<0.11dev,>=0.10.1->google-cloud-pubsub>=0.20.0->google-cloud)
Requirement already up-to-date: setuptools in /tmp/gcp-2493/lib/python2.7/site-packages (from protobuf>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: rsa>=3.1.4 in /tmp/gcp-2493/lib/python2.7/site-packages (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: pyasn1-modules>=0.0.5 in /tmp/gcp-2493/lib/python2.7/site-packages (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: pyasn1>=0.1.7 in /tmp/gcp-2493/lib/python2.7/site-packages (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Installing collected packages: google-cloud-core, google-cloud-resource-manager, google-cloud-dns, google-cloud-bigquery, google-cloud-translate, google-cloud-language, google-cloud-datastore, google-cloud-monitoring, google-cloud-vision, google-cloud-storage, google-gax, grpc-google-pubsub-v1, grpc-google-iam-v1, gapic-google-pubsub-v1, google-cloud-pubsub, grpc-google-logging-v2, gapic-google-logging-v2, google-cloud-logging, google-cloud-error-reporting, google-cloud-bigtable, google-cloud
  Found existing installation: google-gax 0.13.0
    Uninstalling google-gax-0.13.0:
      Successfully uninstalled google-gax-0.13.0
  Found existing installation: grpc-google-pubsub-v1 0.9.3
    Uninstalling grpc-google-pubsub-v1-0.9.3:
      Successfully uninstalled grpc-google-pubsub-v1-0.9.3
  Found existing installation: gapic-google-pubsub-v1 0.9.3
    Uninstalling gapic-google-pubsub-v1-0.9.3:
      Successfully uninstalled gapic-google-pubsub-v1-0.9.3
  Found existing installation: grpc-google-logging-v2 0.9.3
    Uninstalling grpc-google-logging-v2-0.9.3:
      Successfully uninstalled grpc-google-logging-v2-0.9.3
  Found existing installation: gapic-google-logging-v2 0.9.3
    Uninstalling gapic-google-logging-v2-0.9.3:
      Successfully uninstalled gapic-google-logging-v2-0.9.3
  Found existing installation: google-cloud 0.19.0
    Uninstalling google-cloud-0.19.0:
      Successfully uninstalled google-cloud-0.19.0
Successfully installed gapic-google-logging-v2-0.10.1 gapic-google-pubsub-v1-0.10.1 google-cloud-0.20.0 google-cloud-bigquery-0.20.0 google-cloud-bigtable-0.20.0 google-cloud-core-0.20.0 google-cloud-datastore-0.20.0 google-cloud-dns-0.20.0 google-cloud-error-reporting-0.20.0 google-cloud-language-0.20.0 google-cloud-logging-0.20.0 google-cloud-monitoring-0.20.0 google-cloud-pubsub-0.20.0 google-cloud-resource-manager-0.20.0 google-cloud-storage-0.20.0 google-cloud-translate-0.20.0 google-cloud-vision-0.20.0 google-gax-0.14.1 grpc-google-iam-v1-0.10.1 grpc-google-logging-v2-0.10.1 grpc-google-pubsub-v1-0.10.1
$ /tmp/gcp-2493/bin/python
Python 2.7.11 (default, Feb 22 2016, 07:44:44) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.cloud import datastore
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name datastore
>>> 

The culprit seems to be tied into upgrades to the gapic-* packages:

>>> import sys
>>> sys.modules['google'].__path__
['/tmp/gcp-2493/lib/python2.7/site-packages/google']
>>> sys.modules['google.cloud'].__path__
['/tmp/gcp-2493/lib/python2.7/site-packages/google/cloud']
>>> 
[/home/tseaver] 
$ ls -laF /tmp/gcp-2493/lib/python2.7/site-packages/google/cloud
total 24
drwxrwxr-x  3 tseaver tseaver 4096 Oct  4 12:51 ./
drwxrwxr-x 12 tseaver tseaver 4096 Oct  4 12:51 ../
drwxrwxr-x  4 tseaver tseaver 4096 Oct  4 12:51 gapic/
-rw-rw-r--  1 tseaver tseaver 2547 Oct  4 12:51 _testing.py
-rw-rw-r--  1 tseaver tseaver 4500 Oct  4 12:51 _testing.pyc

Which appear to have clobbered the google.cloud namespace package.

Member

tseaver commented Oct 4, 2016

As a baseline, installing into a fresh virtualenv works:

$ /opt/Python-2.7.11/bin/virtualenv /tmp/gcp-2493
New python executable in /tmp/gcp-2493/bin/python
Installing setuptools, pip, wheel...done.
$ /tmp/gcp-2493/bin/pip install google-cloud
Collecting google-cloud
  Downloading google_cloud-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-resource-manager>=0.20.0 (from google-cloud)
  Downloading google_cloud_resource_manager-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-dns>=0.20.0 (from google-cloud)
  Downloading google_cloud_dns-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-bigquery>=0.20.0 (from google-cloud)
  Using cached google_cloud_bigquery-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-translate>=0.20.0 (from google-cloud)
  Downloading google_cloud_translate-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-language>=0.20.0 (from google-cloud)
  Downloading google_cloud_language-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-datastore>=0.20.0 (from google-cloud)
  Downloading google_cloud_datastore-0.20.0-py2.py3-none-any.whl (91kB)
    100% |████████████████████████████████| 92kB 1.3MB/s 
Collecting google-cloud-monitoring>=0.20.0 (from google-cloud)
  Downloading google_cloud_monitoring-0.20.0-py2.py3-none-any.whl (57kB)
    100% |████████████████████████████████| 61kB 3.5MB/s 
Collecting google-cloud-vision>=0.20.0 (from google-cloud)
  Downloading google_cloud_vision-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-storage>=0.20.0 (from google-cloud)
  Downloading google_cloud_storage-0.20.0-py2.py3-none-any.whl (62kB)
    100% |████████████████████████████████| 71kB 2.7MB/s 
Collecting google-cloud-pubsub>=0.20.0 (from google-cloud)
  Downloading google_cloud_pubsub-0.20.0-py2.py3-none-any.whl (50kB)
    100% |████████████████████████████████| 51kB 1.9MB/s 
Collecting google-cloud-core>=0.20.0 (from google-cloud)
  Using cached google_cloud_core-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-error-reporting>=0.20.0 (from google-cloud)
  Downloading google_cloud_error_reporting-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-logging>=0.20.0 (from google-cloud)
  Downloading google_cloud_logging-0.20.0-py2.py3-none-any.whl (58kB)
    100% |████████████████████████████████| 61kB 2.5MB/s 
Collecting google-cloud-bigtable>=0.20.0 (from google-cloud)
  Downloading google_cloud_bigtable-0.20.0-py2.py3-none-any.whl (117kB)
    100% |████████████████████████████████| 122kB 1.2MB/s 
Collecting grpcio<2.0dev,>=1.0.0 (from google-cloud-datastore>=0.20.0->google-cloud)
  Using cached grpcio-1.0.0-cp27-cp27m-manylinux1_x86_64.whl
Collecting google-gax<0.15dev,>=0.14.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting gapic-google-pubsub-v1<0.11dev,>=0.10.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting grpc-google-pubsub-v1<0.11dev,>=0.10.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting httplib2>=0.9.1 (from google-cloud-core>=0.20.0->google-cloud)
Collecting six (from google-cloud-core>=0.20.0->google-cloud)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting protobuf>=3.0.0 (from google-cloud-core>=0.20.0->google-cloud)
  Using cached protobuf-3.1.0.post1-py2.py3-none-any.whl
Collecting googleapis-common-protos>=1.3.4 (from google-cloud-core>=0.20.0->google-cloud)
Collecting oauth2client<4.0.0dev,>=3.0.0 (from google-cloud-core>=0.20.0->google-cloud)
Collecting gapic-google-logging-v2<0.11dev,>=0.10.1 (from google-cloud-logging>=0.20.0->google-cloud)
Collecting grpc-google-logging-v2<0.11dev,>=0.10.1 (from google-cloud-logging>=0.20.0->google-cloud)
Collecting enum34>=1.0.4 (from grpcio<2.0dev,>=1.0.0->google-cloud-datastore>=0.20.0->google-cloud)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting futures>=2.2.0 (from grpcio<2.0dev,>=1.0.0->google-cloud-datastore>=0.20.0->google-cloud)
  Using cached futures-3.0.5-py2-none-any.whl
Collecting ply==3.8 (from google-gax<0.15dev,>=0.14.1->google-cloud-pubsub>=0.20.0->google-cloud)
Collecting future>=0.15.2 (from google-gax<0.15dev,>=0.14.1->google-cloud-pubsub>=0.20.0->google-cloud)
Collecting grpc-google-iam-v1<0.11.0dev,>=0.10.0 (from gapic-google-pubsub-v1<0.11dev,>=0.10.1->google-cloud-pubsub>=0.20.0->google-cloud)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /tmp/gcp-2493/lib/python2.7/site-packages (from protobuf>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Collecting rsa>=3.1.4 (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Installing collected packages: httplib2, six, protobuf, googleapis-common-protos, pyasn1, rsa, pyasn1-modules, oauth2client, google-cloud-core, google-cloud-resource-manager, google-cloud-dns, google-cloud-bigquery, google-cloud-translate, google-cloud-language, enum34, futures, grpcio, google-cloud-datastore, google-cloud-monitoring, google-cloud-vision, google-cloud-storage, ply, future, google-gax, grpc-google-pubsub-v1, grpc-google-iam-v1, gapic-google-pubsub-v1, google-cloud-pubsub, grpc-google-logging-v2, gapic-google-logging-v2, google-cloud-logging, google-cloud-error-reporting, google-cloud-bigtable, google-cloud
Successfully installed enum34-1.1.6 future-0.15.2 futures-3.0.5 gapic-google-logging-v2-0.10.1 gapic-google-pubsub-v1-0.10.1 google-cloud-0.20.0 google-cloud-bigquery-0.20.0 google-cloud-bigtable-0.20.0 google-cloud-core-0.20.0 google-cloud-datastore-0.20.0 google-cloud-dns-0.20.0 google-cloud-error-reporting-0.20.0 google-cloud-language-0.20.0 google-cloud-logging-0.20.0 google-cloud-monitoring-0.20.0 google-cloud-pubsub-0.20.0 google-cloud-resource-manager-0.20.0 google-cloud-storage-0.20.0 google-cloud-translate-0.20.0 google-cloud-vision-0.20.0 google-gax-0.14.1 googleapis-common-protos-1.3.5 grpc-google-iam-v1-0.10.1 grpc-google-logging-v2-0.10.1 grpc-google-pubsub-v1-0.10.1 grpcio-1.0.0 httplib2-0.9.2 oauth2client-3.0.0 ply-3.8 protobuf-3.1.0.post1 pyasn1-0.1.9 pyasn1-modules-0.0.8 rsa-3.4.2 six-1.10.0
$ /tmp/gcp-2493/bin/pip freeze
-f file:///<path-to-homedir>/.pip/wheels
enum34==1.1.6
future==0.15.2
futures==3.0.5
gapic-google-logging-v2==0.10.1
gapic-google-pubsub-v1==0.10.1
google-cloud==0.20.0
google-cloud-bigquery==0.20.0
google-cloud-bigtable==0.20.0
google-cloud-core==0.20.0
google-cloud-datastore==0.20.0
google-cloud-dns==0.20.0
google-cloud-error-reporting==0.20.0
google-cloud-language==0.20.0
google-cloud-logging==0.20.0
google-cloud-monitoring==0.20.0
google-cloud-pubsub==0.20.0
google-cloud-resource-manager==0.20.0
google-cloud-storage==0.20.0
google-cloud-translate==0.20.0
google-cloud-vision==0.20.0
google-gax==0.14.1
googleapis-common-protos==1.3.5
grpc-google-iam-v1==0.10.1
grpc-google-logging-v2==0.10.1
grpc-google-pubsub-v1==0.10.1
grpcio==1.0.0
httplib2==0.9.2
oauth2client==3.0.0
ply==3.8
protobuf==3.1.0.post1
pyasn1==0.1.9
pyasn1-modules==0.0.8
rsa==3.4.2
six==1.10.0
$ /tmp/gcp-2493/bin/python
Python 2.7.11 (default, Feb 22 2016, 07:44:44) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.cloud import datastore
>>> 

but an upgrade fails:

$ rm -r /tmp/gcp-2493/
$ /opt/Python-2.7.11/bin/virtualenv /tmp/gcp-2493
New python executable in /tmp/gcp-2493/bin/python
Installing setuptools, pip, wheel...done.
$ /tmp/gcp-2493/bin/pip install google-cloud==0.19.0
Collecting google-cloud==0.19.0
  Downloading google-cloud-0.19.0.tar.gz (456kB)
    100% |████████████████████████████████| 460kB 396kB/s 
Collecting httplib2>=0.9.1 (from google-cloud==0.19.0)
Collecting googleapis-common-protos (from google-cloud==0.19.0)
Collecting oauth2client>=2.0.1 (from google-cloud==0.19.0)
Collecting protobuf>=3.0.0 (from google-cloud==0.19.0)
  Using cached protobuf-3.1.0.post1-py2.py3-none-any.whl
Collecting six (from google-cloud==0.19.0)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting grpcio>=1.0.0 (from google-cloud==0.19.0)
  Using cached grpcio-1.0.0-cp27-cp27m-manylinux1_x86_64.whl
Collecting google-gax<0.14dev,>=0.13.0 (from google-cloud==0.19.0)
Collecting gapic-google-pubsub-v1<0.10dev,>=0.9.0 (from google-cloud==0.19.0)
Collecting grpc-google-pubsub-v1<0.10dev,>=0.9.0 (from google-cloud==0.19.0)
Collecting gapic-google-logging-v2<0.10dev,>=0.9.0 (from google-cloud==0.19.0)
Collecting grpc-google-logging-v2<0.10dev,>=0.9.0 (from google-cloud==0.19.0)
Collecting rsa>=3.1.4 (from oauth2client>=2.0.1->google-cloud==0.19.0)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client>=2.0.1->google-cloud==0.19.0)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client>=2.0.1->google-cloud==0.19.0)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /tmp/gcp-2493/lib/python2.7/site-packages (from protobuf>=3.0.0->google-cloud==0.19.0)
Collecting enum34>=1.0.4 (from grpcio>=1.0.0->google-cloud==0.19.0)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting futures>=2.2.0 (from grpcio>=1.0.0->google-cloud==0.19.0)
  Using cached futures-3.0.5-py2-none-any.whl
Collecting ply==3.8 (from google-gax<0.14dev,>=0.13.0->google-cloud==0.19.0)
Collecting future>=0.15.2 (from google-gax<0.14dev,>=0.13.0->google-cloud==0.19.0)
Building wheels for collected packages: google-cloud
  Running setup.py bdist_wheel for google-cloud ... done
Successfully built google-cloud
Installing collected packages: httplib2, six, protobuf, googleapis-common-protos, pyasn1, rsa, pyasn1-modules, oauth2client, enum34, futures, grpcio, ply, future, google-gax, grpc-google-pubsub-v1, gapic-google-pubsub-v1, grpc-google-logging-v2, gapic-google-logging-v2, google-cloud
Successfully installed enum34-1.1.6 future-0.15.2 futures-3.0.5 gapic-google-logging-v2-0.9.3 gapic-google-pubsub-v1-0.9.3 google-cloud-0.19.0 google-gax-0.13.0 googleapis-common-protos-1.3.5 grpc-google-logging-v2-0.9.3 grpc-google-pubsub-v1-0.9.3 grpcio-1.0.0 httplib2-0.9.2 oauth2client-3.0.0 ply-3.8 protobuf-3.1.0.post1 pyasn1-0.1.9 pyasn1-modules-0.0.8 rsa-3.4.2 six-1.10.0
$ /tmp/gcp-2493/bin/pip freeze
-f file:///<path-to-homedir>/.pip/wheels
enum34==1.1.6
future==0.15.2
futures==3.0.5
gapic-google-logging-v2==0.9.3
gapic-google-pubsub-v1==0.9.3
google-cloud==0.19.0
google-gax==0.13.0
googleapis-common-protos==1.3.5
grpc-google-logging-v2==0.9.3
grpc-google-pubsub-v1==0.9.3
grpcio==1.0.0
httplib2==0.9.2
oauth2client==3.0.0
ply==3.8
protobuf==3.1.0.post1
pyasn1==0.1.9
pyasn1-modules==0.0.8
rsa==3.4.2
six==1.10.0
$ /tmp/gcp-2493/bin/pip install --upgrade google-cloud
Collecting google-cloud
  Using cached google_cloud-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-resource-manager>=0.20.0 (from google-cloud)
  Using cached google_cloud_resource_manager-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-dns>=0.20.0 (from google-cloud)
  Using cached google_cloud_dns-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-bigquery>=0.20.0 (from google-cloud)
  Using cached google_cloud_bigquery-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-translate>=0.20.0 (from google-cloud)
  Using cached google_cloud_translate-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-language>=0.20.0 (from google-cloud)
  Using cached google_cloud_language-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-datastore>=0.20.0 (from google-cloud)
  Using cached google_cloud_datastore-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-monitoring>=0.20.0 (from google-cloud)
  Using cached google_cloud_monitoring-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-vision>=0.20.0 (from google-cloud)
  Using cached google_cloud_vision-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-storage>=0.20.0 (from google-cloud)
  Using cached google_cloud_storage-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-pubsub>=0.20.0 (from google-cloud)
  Using cached google_cloud_pubsub-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-core>=0.20.0 (from google-cloud)
  Using cached google_cloud_core-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-error-reporting>=0.20.0 (from google-cloud)
  Using cached google_cloud_error_reporting-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-logging>=0.20.0 (from google-cloud)
  Using cached google_cloud_logging-0.20.0-py2.py3-none-any.whl
Collecting google-cloud-bigtable>=0.20.0 (from google-cloud)
  Using cached google_cloud_bigtable-0.20.0-py2.py3-none-any.whl
Requirement already up-to-date: grpcio<2.0dev,>=1.0.0 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-datastore>=0.20.0->google-cloud)
Collecting google-gax<0.15dev,>=0.14.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting gapic-google-pubsub-v1<0.11dev,>=0.10.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Collecting grpc-google-pubsub-v1<0.11dev,>=0.10.1 (from google-cloud-pubsub>=0.20.0->google-cloud)
Requirement already up-to-date: httplib2>=0.9.1 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: six in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: protobuf>=3.0.0 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: googleapis-common-protos>=1.3.4 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: oauth2client<4.0.0dev,>=3.0.0 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-cloud-core>=0.20.0->google-cloud)
Collecting gapic-google-logging-v2<0.11dev,>=0.10.1 (from google-cloud-logging>=0.20.0->google-cloud)
Collecting grpc-google-logging-v2<0.11dev,>=0.10.1 (from google-cloud-logging>=0.20.0->google-cloud)
Requirement already up-to-date: enum34>=1.0.4 in /tmp/gcp-2493/lib/python2.7/site-packages (from grpcio<2.0dev,>=1.0.0->google-cloud-datastore>=0.20.0->google-cloud)
Requirement already up-to-date: futures>=2.2.0 in /tmp/gcp-2493/lib/python2.7/site-packages (from grpcio<2.0dev,>=1.0.0->google-cloud-datastore>=0.20.0->google-cloud)
Requirement already up-to-date: ply==3.8 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-gax<0.15dev,>=0.14.1->google-cloud-pubsub>=0.20.0->google-cloud)
Requirement already up-to-date: future>=0.15.2 in /tmp/gcp-2493/lib/python2.7/site-packages (from google-gax<0.15dev,>=0.14.1->google-cloud-pubsub>=0.20.0->google-cloud)
Collecting grpc-google-iam-v1<0.11.0dev,>=0.10.0 (from gapic-google-pubsub-v1<0.11dev,>=0.10.1->google-cloud-pubsub>=0.20.0->google-cloud)
Requirement already up-to-date: setuptools in /tmp/gcp-2493/lib/python2.7/site-packages (from protobuf>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: rsa>=3.1.4 in /tmp/gcp-2493/lib/python2.7/site-packages (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: pyasn1-modules>=0.0.5 in /tmp/gcp-2493/lib/python2.7/site-packages (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Requirement already up-to-date: pyasn1>=0.1.7 in /tmp/gcp-2493/lib/python2.7/site-packages (from oauth2client<4.0.0dev,>=3.0.0->google-cloud-core>=0.20.0->google-cloud)
Installing collected packages: google-cloud-core, google-cloud-resource-manager, google-cloud-dns, google-cloud-bigquery, google-cloud-translate, google-cloud-language, google-cloud-datastore, google-cloud-monitoring, google-cloud-vision, google-cloud-storage, google-gax, grpc-google-pubsub-v1, grpc-google-iam-v1, gapic-google-pubsub-v1, google-cloud-pubsub, grpc-google-logging-v2, gapic-google-logging-v2, google-cloud-logging, google-cloud-error-reporting, google-cloud-bigtable, google-cloud
  Found existing installation: google-gax 0.13.0
    Uninstalling google-gax-0.13.0:
      Successfully uninstalled google-gax-0.13.0
  Found existing installation: grpc-google-pubsub-v1 0.9.3
    Uninstalling grpc-google-pubsub-v1-0.9.3:
      Successfully uninstalled grpc-google-pubsub-v1-0.9.3
  Found existing installation: gapic-google-pubsub-v1 0.9.3
    Uninstalling gapic-google-pubsub-v1-0.9.3:
      Successfully uninstalled gapic-google-pubsub-v1-0.9.3
  Found existing installation: grpc-google-logging-v2 0.9.3
    Uninstalling grpc-google-logging-v2-0.9.3:
      Successfully uninstalled grpc-google-logging-v2-0.9.3
  Found existing installation: gapic-google-logging-v2 0.9.3
    Uninstalling gapic-google-logging-v2-0.9.3:
      Successfully uninstalled gapic-google-logging-v2-0.9.3
  Found existing installation: google-cloud 0.19.0
    Uninstalling google-cloud-0.19.0:
      Successfully uninstalled google-cloud-0.19.0
Successfully installed gapic-google-logging-v2-0.10.1 gapic-google-pubsub-v1-0.10.1 google-cloud-0.20.0 google-cloud-bigquery-0.20.0 google-cloud-bigtable-0.20.0 google-cloud-core-0.20.0 google-cloud-datastore-0.20.0 google-cloud-dns-0.20.0 google-cloud-error-reporting-0.20.0 google-cloud-language-0.20.0 google-cloud-logging-0.20.0 google-cloud-monitoring-0.20.0 google-cloud-pubsub-0.20.0 google-cloud-resource-manager-0.20.0 google-cloud-storage-0.20.0 google-cloud-translate-0.20.0 google-cloud-vision-0.20.0 google-gax-0.14.1 grpc-google-iam-v1-0.10.1 grpc-google-logging-v2-0.10.1 grpc-google-pubsub-v1-0.10.1
$ /tmp/gcp-2493/bin/python
Python 2.7.11 (default, Feb 22 2016, 07:44:44) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.cloud import datastore
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name datastore
>>> 

The culprit seems to be tied into upgrades to the gapic-* packages:

>>> import sys
>>> sys.modules['google'].__path__
['/tmp/gcp-2493/lib/python2.7/site-packages/google']
>>> sys.modules['google.cloud'].__path__
['/tmp/gcp-2493/lib/python2.7/site-packages/google/cloud']
>>> 
[/home/tseaver] 
$ ls -laF /tmp/gcp-2493/lib/python2.7/site-packages/google/cloud
total 24
drwxrwxr-x  3 tseaver tseaver 4096 Oct  4 12:51 ./
drwxrwxr-x 12 tseaver tseaver 4096 Oct  4 12:51 ../
drwxrwxr-x  4 tseaver tseaver 4096 Oct  4 12:51 gapic/
-rw-rw-r--  1 tseaver tseaver 2547 Oct  4 12:51 _testing.py
-rw-rw-r--  1 tseaver tseaver 4500 Oct  4 12:51 _testing.pyc

Which appear to have clobbered the google.cloud namespace package.

@tseaver tseaver added the GAX label Oct 4, 2016

@tseaver

This comment has been minimized.

Show comment
Hide comment
@tseaver

tseaver Oct 4, 2016

Member

@bjwatson Any thoughts?

Member

tseaver commented Oct 4, 2016

@bjwatson Any thoughts?

@jlowin

This comment has been minimized.

Show comment
Hide comment
@jlowin

jlowin Oct 4, 2016

Indeed, installing a fresh conda environment solved this. And I was able to solve it in the old environment by uninstalling all google packages and reinstalling -- protobuf was the holdout that I didn't think of at first. It wasn't enough to --upgrade google-cloud, I had to explicitly uninstall protobuf.

jlowin commented Oct 4, 2016

Indeed, installing a fresh conda environment solved this. And I was able to solve it in the old environment by uninstalling all google packages and reinstalling -- protobuf was the holdout that I didn't think of at first. It wasn't enough to --upgrade google-cloud, I had to explicitly uninstall protobuf.

@dhermes

This comment has been minimized.

Show comment
Hide comment
@dhermes

dhermes Oct 4, 2016

Member

@tseaver See #2464 (comment) and #2464 (comment)

AFAICT it's out of our hands. The packages installed are identical after the upgrade but the directory structure is b0rked somehow by pip / setuptools.

UPDATE: I confirm your analysis about the clobbered google/cloud directory

Member

dhermes commented Oct 4, 2016

@tseaver See #2464 (comment) and #2464 (comment)

AFAICT it's out of our hands. The packages installed are identical after the upgrade but the directory structure is b0rked somehow by pip / setuptools.

UPDATE: I confirm your analysis about the clobbered google/cloud directory

@tseaver

This comment has been minimized.

Show comment
Hide comment
@tseaver

tseaver Oct 4, 2016

Member

@dhermes I'm suspicious that one or more of our dependencies (protobuf, gapic-*, grpc-*) as installed via google-cloud==0.19.0 b0rk the namespace-package bookkeeping, confusing pip on the upgrade

Member

tseaver commented Oct 4, 2016

@dhermes I'm suspicious that one or more of our dependencies (protobuf, gapic-*, grpc-*) as installed via google-cloud==0.19.0 b0rk the namespace-package bookkeeping, confusing pip on the upgrade

@jlowin

This comment has been minimized.

Show comment
Hide comment
@jlowin

jlowin Oct 4, 2016

For what it's worth, after upgrading to 0.20, I run the following commands to get it working (doubtless some of the uninstalls are false positives but I haven't gone one by one to figure it out):

pip install "google-cloud==0.19"
python -c "from google.cloud import datastore" # works

pip install google-cloud --upgrade # broken upgrade from 0.19
python -c "from google.cloud import datastore" # fails

pip uninstall -y google-cloud-bigquery google-cloud-bigtable google-cloud-core google-cloud-datastore google-cloud-dns google-cloud-error-reporting google-cloud-language google-cloud-logging google-cloud-monitoring google-cloud-pubsub google-cloud-resource-manager google-cloud-storage google-cloud-translate google-cloud-vision google-gax protobuf

pip install google-cloud --upgrade
python -c "from google.cloud import datastore" # works

jlowin commented Oct 4, 2016

For what it's worth, after upgrading to 0.20, I run the following commands to get it working (doubtless some of the uninstalls are false positives but I haven't gone one by one to figure it out):

pip install "google-cloud==0.19"
python -c "from google.cloud import datastore" # works

pip install google-cloud --upgrade # broken upgrade from 0.19
python -c "from google.cloud import datastore" # fails

pip uninstall -y google-cloud-bigquery google-cloud-bigtable google-cloud-core google-cloud-datastore google-cloud-dns google-cloud-error-reporting google-cloud-language google-cloud-logging google-cloud-monitoring google-cloud-pubsub google-cloud-resource-manager google-cloud-storage google-cloud-translate google-cloud-vision google-gax protobuf

pip install google-cloud --upgrade
python -c "from google.cloud import datastore" # works

@bjwatson

This comment has been minimized.

Show comment
Hide comment
@bjwatson

bjwatson Oct 4, 2016

Contributor

@tseaver I followed @jlowin steps above, and don't see any issues with the GAPIC or gRPC packages after running pip install google-cloud --upgrade.

I tried these alternate instructions and had no problems:

pip install "google-cloud==0.19"
python -c "from google.cloud import datastore" # works

pip uninstall google-cloud
pip install google-cloud --upgrade
python -c "from google.cloud import datastore" # works

There appears to be an issue with the google-cloud upgrade itself, likely due to the modularization between 0.19.0 and 0.20.0. For some reason, packages like google-cloud-datastore 0.20.0 are not properly installed when doing an --upgrade (e.g. the venv/lib/python2.7/site-packages/google/cloud/datastore is missing).

Any ideas @jonparrott?

Contributor

bjwatson commented Oct 4, 2016

@tseaver I followed @jlowin steps above, and don't see any issues with the GAPIC or gRPC packages after running pip install google-cloud --upgrade.

I tried these alternate instructions and had no problems:

pip install "google-cloud==0.19"
python -c "from google.cloud import datastore" # works

pip uninstall google-cloud
pip install google-cloud --upgrade
python -c "from google.cloud import datastore" # works

There appears to be an issue with the google-cloud upgrade itself, likely due to the modularization between 0.19.0 and 0.20.0. For some reason, packages like google-cloud-datastore 0.20.0 are not properly installed when doing an --upgrade (e.g. the venv/lib/python2.7/site-packages/google/cloud/datastore is missing).

Any ideas @jonparrott?

@dhermes

This comment has been minimized.

Show comment
Hide comment
@dhermes

dhermes Jan 31, 2017

Member

Closing this out as it seems to have gone stale

Member

dhermes commented Jan 31, 2017

Closing this out as it seems to have gone stale

@dhermes dhermes closed this Jan 31, 2017

@npetitclerc

This comment has been minimized.

Show comment
Hide comment
@npetitclerc

npetitclerc Feb 1, 2017

I had similar issues upgrading to 0.22.0 from 0.19.0 @jlowin solution worked for me:

pip uninstall -y google-cloud-bigquery google-cloud-bigtable google-cloud-core google-cloud-datastore google-cloud-dns google-cloud-error-reporting google-cloud-language google-cloud-logging google-cloud-monitoring google-cloud-pubsub google-cloud-resource-manager google-cloud-storage google-cloud-translate google-cloud-vision google-gax protobuf
 
pip install google-cloud --upgrade

@bjwatson FYI uninstalling google-cloud only didn't work for me.

npetitclerc commented Feb 1, 2017

I had similar issues upgrading to 0.22.0 from 0.19.0 @jlowin solution worked for me:

pip uninstall -y google-cloud-bigquery google-cloud-bigtable google-cloud-core google-cloud-datastore google-cloud-dns google-cloud-error-reporting google-cloud-language google-cloud-logging google-cloud-monitoring google-cloud-pubsub google-cloud-resource-manager google-cloud-storage google-cloud-translate google-cloud-vision google-gax protobuf
 
pip install google-cloud --upgrade

@bjwatson FYI uninstalling google-cloud only didn't work for me.

@dhermes

This comment has been minimized.

Show comment
Hide comment
@dhermes

dhermes Feb 1, 2017

Member

Yes it seems 0.19.0 is the problem, but it's unclear if anything can be done about it (other than having this issue here as a reference).

Member

dhermes commented Feb 1, 2017

Yes it seems 0.19.0 is the problem, but it's unclear if anything can be done about it (other than having this issue here as a reference).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment