Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImportError: No module named _ssl on App Engine since 1.5.0 #191

Closed
nabla-c0d3 opened this issue Feb 23, 2016 · 19 comments

Comments

@nabla-c0d3
Copy link

commented Feb 23, 2016

After updating to 1.5.0, my App running on App Engine stopped working:

  File "<...>/base_api_client.py", line 4, in <module>
    from googleapiclient.discovery import build
  File "<...>/lib/googleapiclient/discovery.py", line 63, in <module>
    from googleapiclient.http import BatchHttpRequest
  File "<...>/lib/googleapiclient/http.py", line 39, in <module>
    import ssl
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 97, in <module>
    import _ssl             # if we can't import it, let the error propagate
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 948, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named _ssl

The error goes away if I add the ssl module to my app.yaml:

- name: ssl
  version: latest

However, googleapiclient worked fine previously without using/importing the ssl module, so I am not sure this is intended.

Thanks,

@theacodes

This comment has been minimized.

Copy link
Member

commented Feb 23, 2016

@dhermes any ideas on this one? Seems like oauth2client now requires openssl instead of it being optional.

@dhermes

This comment has been minimized.

Copy link
Contributor

commented Feb 23, 2016

I don't see any noticeable changes in setup.py history

@theacodes

This comment has been minimized.

Copy link
Member

commented Feb 23, 2016

I think it's more that maybe some ``try... import... except` clauses were removed?

@dhermes

This comment has been minimized.

Copy link
Contributor

commented Feb 23, 2016

None of them were "removed" (that I know of), just made more explicit (with actual code definitions moved into their own modules). From the stack trace it looks like googleapiclient/http.py is importing ssl.

@theacodes

This comment has been minimized.

Copy link
Member

commented Feb 23, 2016

Arg, thanks! Sorry for assuming it was on oauth2client.

@theacodes theacodes self-assigned this Feb 23, 2016

@dhermes

This comment has been minimized.

Copy link
Contributor

commented Feb 23, 2016

No worries. Cheers.

@nabla-c0d3

This comment has been minimized.

Copy link
Author

commented Feb 23, 2016

Here's the commit that, I think, introduced the error by importing ssl: 703c92c

@nathanielmanistaatgoogle

This comment has been minimized.

Copy link
Contributor

commented Feb 23, 2016

(+@rryk, author of the cited commit)

Let's imagine for a moment that I were a boorish project maintainer incapable of grasping nuance: what's the problem being reported here? The ssl module is part of the Python standard library, right? Why would it be a problem that our library uses the language's standard library?

@nabla-c0d3

This comment has been minimized.

Copy link
Author

commented Feb 23, 2016

The problem is that the App Engine documentation states that you can use googleapiclient on App Engine ( https://developers.google.com/api-client-library/python/guide/google_app_engine ) and this is no longer true unless you make special, non-intuitive changes to your app.yaml file, in order to work around some code that may or may not work on App engine anyway (as the commit did not take this use case into account).

@theacodes

This comment has been minimized.

Copy link
Member

commented Feb 23, 2016

@nathanielmanistaatgoogle App Engine used a heavily modified version of Python, which does not have ssl included by default. Users have to make a change to their app.yaml to opt-in to ssl. We can document this, but IMO it should remain optional as it was before.

@nathanielmanistaatgoogle

This comment has been minimized.

Copy link
Contributor

commented Feb 24, 2016

@nabla-c0d3: thank you for your strong and clear response.

@jonparrott: elaborate a bit more on your opinion? Aside from gumbling in App Engine's general direction I'm not yet sure what the right thing to do here is. I'm reluctant to take on a special requirement for the library like "must not import ssl" that would be awkward or weird to verify in tests.

@theacodes

This comment has been minimized.

Copy link
Member

commented Feb 24, 2016

@nathanielmanistaatgoogle grumbling in App Engine's direction is deserved, I think. I think we have two options here:

  1. Be okay with the fact that we broke users in an environment that previously worked without additional config, and thus shift the burden onto our users.
  2. Add import guards and unbreak users in that environment.

We can grumble at App Engine either way, it's just what we want to do between now and whenever I can convince the runtime team to turn on ssl by default.

@nathanielmanistaatgoogle

This comment has been minimized.

Copy link
Contributor

commented Feb 24, 2016

A guarded import would wound my pride but I think I might survive. Is there an issue in a public issue tracker about turning on ssl by default to which we might link our TODO?

@rryk, any opinion on any of this?

@theacodes

This comment has been minimized.

Copy link
Member

commented Feb 24, 2016

Public issue tracker for App engine is in limbo. I'm part of the runtime team, so I will bring it up in our meeting tomorrow.

@nabla-c0d3

This comment has been minimized.

Copy link
Author

commented Feb 24, 2016

Sorry for insisting but I just want to re-iterate something: I don't think the newly introduced code that is using ssl would even work on App Engine.
App Engine's HTTP stack (httplib, urllib2 and all use it) is heavily customized and I would not expect it to throw an ssl.SSLError ever. I am not 100% sure and the App Engine team would know better of course, but I don't this issue is about enabling ssl on App Engine. It's about Cython-specific code (ie. code that doesn't work on App Engine) that was introduced in the commit.

@nathanielmanistaatgoogle

This comment has been minimized.

Copy link
Contributor

commented Feb 24, 2016

@nabla-c0d3: are you sure you're not conflating CPython and Cython? They are very different things.

I'm not following your point, though - are you saying that the version of the ssl module made available on App Engine does not have the same API as the standard library's ssl module?

@nabla-c0d3

This comment has been minimized.

Copy link
Author

commented Feb 24, 2016

@nathanielmanistaatgoogle yeah I meant CPython.

The ssl modules are probably the same but when making HTTP requests on App Engine, the URL Fetch service is used (https://cloud.google.com/appengine/docs/python/urlfetch/) behind the scenes even when using standard libraries like httplib or urllib2. It is, I think, a completely different implementation which would never throw an ssl.SSLError (what the commit tries to handle) since it probably doesn't even use Python's ssl module at all.

The ssl module is also a recent addition to App Engine and because it is optional, I would not expect the URL Fetch service to throw an ssl.SSLError when something fails at the SSL level; hence the code that was pushed most likely does not work on App Engine.

@rryk

This comment has been minimized.

Copy link
Contributor

commented Feb 24, 2016

Hi, I am on vacation and don't have a good internet access or a computer to
make any changes. Import guard sounds like an acceptable solution to me
though.

On Wed, Feb 24, 2016, 14:51 Alban Diquet notifications@github.com wrote:

@nathanielmanistaatgoogle https://github.com/nathanielmanistaatgoogle
yeah I meant CPython.

The ssl modules are probably the same but when making HTTP requests on
App Engine, the URL Fetch service is used (
https://cloud.google.com/appengine/docs/python/urlfetch/) behind the
scenes (even when using standard libraries like httplib or urllib2). It is,
I think, a completely different implementation which would never throw an
ssl.SSLError (what the commit tries to handle) since it probably doesn't
even use Python's ssl module at all.


Reply to this email directly or view it on GitHub
#191 (comment)
.

rctay added a commit to rctay/google-api-python-client that referenced this issue Apr 20, 2016

googleapiclient.http: guard when importing ssl
Fix googleapis#191 by adding a guard when importing `ssl`, and replace all direct
references to `SSLError` (the sole member of `ssl` being used) with a
shim, `_ssl_SSLError`. It is prefixed with a `_` to caution users
against relying on the name when importing `googleclient.http`.

`httplib2` also takes the same approach (see `ssl_SSLError` in
`httplib2/__init__.py`), but instead of using `None` when faking
`SSLError`, we provide a 'real' class, so that code can continue to
trigger the exception handler for `SSLError` in `googleclient.http` (eg.
in tests, `SSLError`'s are raised). If the shim `_ssl_SSLError` were
`None`, an exception handler for it would not catch anything, as `None`
is not a class and we could never throw anything that could be
"compatible" with `None` [1}

[1] https://docs.python.org/2/reference/compound_stmts.html#except

rctay added a commit to rctay/google-api-python-client that referenced this issue Apr 20, 2016

googleapiclient.http: guard when importing ssl
Fix googleapis#191 by adding a guard when importing `ssl`, and replace all direct
references to `SSLError` (the sole member of `ssl` being used) with a
shim, `_ssl_SSLError`. It is prefixed with a '_' to caution users
against relying on the name when importing `googleclient.http`.

`httplib2` also takes the same approach (see `ssl_SSLError` in
`httplib2/__init__.py`), but instead of using `None` when faking
`SSLError`, we provide a 'real' class, so that code can continue to
trigger the exception handler for `SSLError` in `googleclient.http` (eg.
in tests, `SSLError`'s are raised). If the shim `_ssl_SSLError` were
`None`, an exception handler for it would not catch anything, as `None`
is not a class and we could never throw anything that could be
"compatible" with `None` [1]

[1] https://docs.python.org/2/reference/compound_stmts.html#except
@rctay

This comment has been minimized.

Copy link
Contributor

commented Apr 20, 2016

I ran into this issue while on App Engine and saw this issue. Since there appears to be a consensus on import guards, I have created a pull request #220 that guardedly imports (hurhur) ssl.

rctay added a commit to rctay/google-api-python-client that referenced this issue Apr 26, 2016

googleapiclient.http: guard when importing ssl
Fix googleapis#191 by adding a guard when importing `ssl`, and replace all direct
references to `SSLError` (the sole member of `ssl` being used) with a
shim, `_ssl_SSLError`. It is prefixed with a '_' to caution users
against relying on the name when importing `googleclient.http`.

`httplib2` also takes the same approach (see `ssl_SSLError` in
`httplib2/__init__.py`), but instead of using `None` when faking
`SSLError`, we provide a 'real' class, so that code can continue to
trigger the exception handler for `SSLError` in `googleclient.http` (eg.
in tests, `SSLError`'s are raised). If the shim `_ssl_SSLError` were
`None`, an exception handler for it would not catch anything, as `None`
is not a class and we could never throw anything that could be
"compatible" with `None` [1]

[1] https://docs.python.org/2/reference/compound_stmts.html#except

rctay added a commit to rctay/google-api-python-client that referenced this issue Apr 26, 2016

googleapiclient.http: guard when importing ssl
Fix googleapis#191 by adding a guard when importing `ssl`, and replace all direct
references to `SSLError` (the sole member of `ssl` being used) with a
shim, `_ssl_SSLError`. It is prefixed with a '_' to caution users
against relying on the name when importing `googleclient.http`.

`httplib2` also takes the same approach (see `ssl_SSLError` in
`httplib2/__init__.py`), but instead of using `None` when faking
`SSLError`, we provide a 'real' class, so that code can continue to
trigger the exception handler for `SSLError` in `googleclient.http` (eg.
in tests, `SSLError`'s are raised). If the shim `_ssl_SSLError` were
`None`, an exception handler for it would not catch anything, as `None`
is not a class and we could never throw anything that could be
"compatible" with `None` [1]

[1] https://docs.python.org/2/reference/compound_stmts.html#except

rctay added a commit to rctay/google-api-python-client that referenced this issue Apr 27, 2016

googleapiclient.http: guard when importing ssl
Fix googleapis#191 by adding a guard when importing `ssl`, and replace all direct
references to `SSLError` (the sole member of `ssl` being used) with a
shim, `_ssl_SSLError`. It is prefixed with a '_' to caution users
against relying on the name when importing `googleclient.http`.

`httplib2` also takes a similar approach (see `ssl_SSLError` in
`httplib2/__init__.py`).

rctay added a commit to rctay/google-api-python-client that referenced this issue Apr 28, 2016

googleapiclient.http: guard when importing ssl
Fix googleapis#191 by adding a guard when importing `ssl`, and replace all direct
references to `SSLError` (the sole member of `ssl` being used) with a
shim, `_ssl_SSLError`. It is prefixed with a '_' to caution users
against relying on the name when importing `googleclient.http`.

`httplib2` also takes a similar approach (see `ssl_SSLError` in
`httplib2/__init__.py`).

scudette added a commit to scudette/rekall-agent-server that referenced this issue Jan 29, 2018

Fix missing libraries. (#14)
AppEngine broke compatibility by introducing an additional ssl
dependency (googleapis/google-api-python-client#191)

akrherz pushed a commit to akrherz/google-api-python-client that referenced this issue Apr 1, 2019

Merge pull request googleapis#191 from craigcitro/openssl_detection
Check for OpenSSL.crypto when detecting OpenSSL.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.