Install fails with poetry/python 3.10

[indigoviolet]

Setting up a clean virtual environment with Poetry, Python 3.10 and google-cloud-compute fails with an error about google.api_core.

This error goes away if I use Python 3.8.1 instead.

I realize that poetry might not be an officially supported package manager, but I wanted to report anyway for other users who might encounter such an issue:

❯ poetry --version
Poetry (version 1.2.0a2)

Environment details

• OS type and version: Ubuntu 21.04

• Python version: python --version Python 3.10.1

• pip version: pip --version pip 21.0.1 from /home/venky/dev/gcloud-test/.venv/lib/python3.10/site-packages/pip (python 3.10)

• google-cloud-compute version: pip show google-cloud-compute

Name: google-cloud-compute
Version: 1.0.0
Summary: UNKNOWN
Home-page: https://github.com/googleapis/python-compute
Author: Google LLC
Author-email: googleapis-packages@google.com
License: Apache 2.0
Location: /home/venky/dev/gcloud-test/.venv/lib/python3.10/site-packages
Requires: proto-plus, google-api-core
Required-by: 

Steps to reproduce

❯ cat pyproject.toml            
[tool.poetry]
name = "gcloud-test"
version = "0.0.0"
description = ""
authors = ["Venky Iyer <indigoviolet@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.10"
google-cloud-compute = "^1.0.0"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

❯ poetry show --tree     
google-cloud-compute 1.0.0
├── google-api-core >=2.2.0,<3.0.0dev
│   ├── google-auth >=1.25.0,<3.0dev 
│   │   ├── cachetools >=2.0.0,<6.0 
│   │   ├── pyasn1-modules >=0.2.1 
│   │   │   └── pyasn1 >=0.4.6,<0.5.0 
│   │   ├── rsa >=3.1.4,<5 
│   │   │   └── pyasn1 >=0.1.3 (circular dependency aborted here)
│   │   └── six >=1.9.0 
│   ├── googleapis-common-protos >=1.52.0,<2.0dev 
│   │   └── protobuf >=3.12.0 
│   ├── grpcio >=1.33.2,<2.0dev 
│   │   └── six >=1.5.2 (circular dependency aborted here)
│   ├── grpcio-status >=1.33.2,<2.0dev 
│   │   ├── googleapis-common-protos >=1.5.5 (circular dependency aborted here)
│   │   ├── grpcio >=1.43.0 (circular dependency aborted here)
│   │   └── protobuf >=3.6.0 (circular dependency aborted here)
│   ├── protobuf >=3.12.0 (circular dependency aborted here)
│   └── requests >=2.18.0,<3.0.0dev 
│       ├── certifi >=2017.4.17 
│       ├── charset-normalizer >=2.0.0,<2.1.0 
│       ├── idna >=2.5,<4 
│       └── urllib3 >=1.21.1,<1.27 
└── proto-plus >=1.19.7
    └── protobuf >=3.19.0 

❯ python      
Python 3.10.1 (main, Jan 13 2022, 12:25:55) [GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.cloud import compute_v1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/venky/dev/gcloud-test/.venv/lib/python3.10/site-packages/google/cloud/compute_v1/__init__.py", line 17, in <module>
    from .services.accelerator_types import AcceleratorTypesClient
  File "/home/venky/dev/gcloud-test/.venv/lib/python3.10/site-packages/google/cloud/compute_v1/services/accelerator_types/__init__.py", line 16, in <module>
    from .client import AcceleratorTypesClient
  File "/home/venky/dev/gcloud-test/.venv/lib/python3.10/site-packages/google/cloud/compute_v1/services/accelerator_types/client.py", line 22, in <module>
    from google.api_core import client_options as client_options_lib
ModuleNotFoundError: No module named 'google.api_core'
>>> 

I'm happy to collect other debugging info if it is helpful.

[parthea]

Hi @indigoviolet,

I was able to reproduce the issue when installing poetry using the get-poetry.py script mentioned here. Once I switched to the install-poetry.py script mentioned here I was no longer able to produce the problem. There appears to be a known issue with the get-poetry.py script as described here. The get-poetry.py script is deprecated as mentioned here.

I had to run poetry remove google-cloud-compute and poetry add google-cloud-compute after re-installing poetry in order for the issue to be resolved.

(test) partheniou@partheniou:~/compute$ curl -sSL https://install.python-poetry.org | python3 -
Retrieving Poetry metadata

The latest version (1.1.12) is already installed.
(test) partheniou@partheniou:~/compute$ poetry shell
Virtual environment already activated: /usr/local/google/home/partheniou/.pyenv/versions/3.10.2/envs/test
(test) partheniou@partheniou:~/compute$ python3
Python 3.10.2 (main, Feb  7 2022, 22:43:09) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.cloud import compute_v1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'google'
>>> exit()
(test) partheniou@partheniou:~/compute$ poetry remove google-cloud-compute
Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

No dependencies to install or update
(test) partheniou@partheniou:~/compute$ poetry add google-cloud-compute
Using version ^1.0.0 for google-cloud-compute

Updating dependencies
Resolving dependencies... (0.3s)

Writing lock file

Package operations: 18 installs, 0 updates, 0 removals

  • Installing protobuf (3.19.4)
  • Installing pyasn1 (0.4.8)
  • Installing six (1.16.0)
  • Installing cachetools (5.0.0)
  • Installing certifi (2021.10.8)
  • Installing charset-normalizer (2.0.11)
  • Installing googleapis-common-protos (1.54.0)
  • Installing grpcio (1.43.0)
  • Installing idna (3.3)
  • Installing pyasn1-modules (0.2.8)
  • Installing rsa (4.8)
  • Installing urllib3 (1.26.8)
  • Installing google-auth (2.6.0)
  • Installing grpcio-status (1.43.0)
  • Installing requests (2.27.1)
  • Installing google-api-core (2.5.0)
  • Installing proto-plus (1.20.0)
  • Installing google-cloud-compute (1.0.0)
(test) partheniou@partheniou:~/compute$ python3
Python 3.10.2 (main, Feb  7 2022, 22:43:09) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.cloud import compute_v1
>>>

I'm going to close this issue but please feel free to re-open it if you have additional questions.

[indigoviolet]

Thank you.

It turned out that I was using install-poetry as recommended, but there is some other issue with Poetry 1.2.0a2 (which I was using in the first place so that the install-poetry installer was default, doh).

Forcing the 1.1.9 Poetry version with the install-poetry installer made everything work.