Need the Correct Versions of google-auth and google-api-python-client for Python 3.9 Runtime to Resolve MTLS Missing Issue

[sarsivas]

Description

We are utilizing the specified package versions listed in requirement.txt for the App Engine Python 3.9 runtime, and we are experiencing the following error when running local development tests with pytest using the Google Cloud SDK
google-auth==2.23.4
google-api-python-client==2.52.0

Error

Traceback:
/usr/lib/python3.9/importlib/init.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
big_query/tests/TestBQConnection.py:2: in
from big_query import bq_connection
big_query/bq_connection.py:6: in
from apiclient.errors import HttpError
/usr/local/lib/python3.9/dist-packages/apiclient/init.py:3: in
from googleapiclient import channel, discovery, errors, http, mimeparse, model
/usr/local/lib/python3.9/dist-packages/googleapiclient/discovery.py:44: in
from google.auth.transport import mtls
E ImportError: cannot import name 'mtls' from 'google.auth.transport' (/usr/lib/google-cloud-sdk/platform/google_appengine/google/auth/transport/init.py)

Notes

• We have attempted the latest version below and encountered the the above error.
  pip install google-api-python-client==2.122.0
pip install google-auth==2.28.2
• Here is the Google SDK version we are using in the App Engine local development appserver.

root@b50aa326faf4:/editor# gcloud --version
Google Cloud SDK 449.0.0
alpha 2023.10.02
app-engine-go 1.9.75
app-engine-java 2.0.19.1
app-engine-python 1.9.106
app-engine-python-extras 1.9.102
beta 2023.10.02
bigtable 
bq 2.0.98
bundled-python3-unix 3.9.16
cbt 0.16.1
cloud-datastore-emulator 2.3.1
cloud-firestore-emulator 1.18.2
cloud-spanner-emulator 1.5.10
core 2023.10.02
gcloud-crc32c 1.0.0
gke-gcloud-auth-plugin 0.5.6
gsutil 5.26
kpt 1.0.0-beta.44
local-extract 1.5.9
pubsub-emulator 0.8.8

[ohmayr]

@sarsivas thanks for reporting this issue.

I tried running a simple script locally with google-api-python-client==2.122.0 and google-auth==2.28.2 and did not get the same error as you.

from googleapiclient.discovery import build


def main():
   build("", "")

if __name__ == '__main__':
    main()

However, I found out that mtls was added to google.auth.transport in v1.14.0.

Therefore, a version of google-auth <= 1.14.0 results in the error that you're experiencing.

With pip install google-auth==1.13.1, I get the following error:

from google.auth.transport import mtls
ImportError: cannot import name 'mtls' from 'google.auth.transport' (/usr/local/google/home/omairn/.pyenv/versions/3.9.5/envs/py39/lib/python3.9/site-packages/google/auth/transport/__init__.py)

It may be that in your case, an older version of google-auth is being installed as a transitive dependency.

Can you confirm this behaviour by running pip show google-auth?

If that's not the case, we will need more information from you regarding what exactly you're running and if this occurs only when using the Python SDK in App engine.

[sarsivas]

Thank you, Ohmayr. We are utilizing the 'google/cloud-sdk:449.0.0' Docker image to execute pytest. The error pertains to the Google Cloud SDK library path where it does not contain the 'mtls.py' module.

Below is the output:

# ls /usr/lib/google-cloud-sdk/platform/google_appengine/google/auth/transport/
__init__.py      _http_client.py  _mtls_helper.py  requests.py 

 
Not referring to google-auth in this path.

# ls /usr/local/lib/python3.9/dist-packages/google/auth/transport/
__init__.py  __pycache__  _aiohttp_requests.py  _custom_tls_signer.py  _http_client.py  _mtls_helper.py  grpc.py  mtls.py  requests.py  urllib3.py

 
After adding the PYTHONPATH as below, it appears to be working
/usr/local/lib/python3.9/dist-packages:/usr/lib/google-cloud-sdk/platform/google_appengine/

Not an issue with the library, it seems that the old version of the library, which doesn't have 'mtls', is present in the Google Cloud SDK version 449. We can close this support ticket.

[vchudnov-g]

Glad you were able to resolve the issue! Closing.