ImportError: No module named _ssl on App Engine since 1.5.0

[nabla-c0d3]

After updating to 1.5.0, my App running on App Engine stopped working:

  File "<...>/base_api_client.py", line 4, in <module>
    from googleapiclient.discovery import build
  File "<...>/lib/googleapiclient/discovery.py", line 63, in <module>
    from googleapiclient.http import BatchHttpRequest
  File "<...>/lib/googleapiclient/http.py", line 39, in <module>
    import ssl
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 97, in <module>
    import _ssl             # if we can't import it, let the error propagate
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 948, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named _ssl

The error goes away if I add the ssl module to my app.yaml:

- name: ssl
  version: latest

However, googleapiclient worked fine previously without using/importing the ssl module, so I am not sure this is intended.

Thanks,

[theacodes]

@dhermes any ideas on this one? Seems like oauth2client now requires openssl instead of it being optional.

[dhermes]

I don't see any noticeable changes in setup.py history

[theacodes]

I think it's more that maybe some ``try... import... except` clauses were removed?

[dhermes]

None of them were "removed" (that I know of), just made more explicit (with actual code definitions moved into their own modules). From the stack trace it looks like googleapiclient/http.py is importing ssl.

[theacodes]

Arg, thanks! Sorry for assuming it was on oauth2client.

[dhermes]

No worries. Cheers.

[nabla-c0d3]

Here's the commit that, I think, introduced the error by importing ssl: 703c92c

[nathanielmanistaatgoogle]

(+@rryk, author of the cited commit)

Let's imagine for a moment that I were a boorish project maintainer incapable of grasping nuance: what's the problem being reported here? The ssl module is part of the Python standard library, right? Why would it be a problem that our library uses the language's standard library?

[nabla-c0d3]

The problem is that the App Engine documentation states that you can use googleapiclient on App Engine ( https://developers.google.com/api-client-library/python/guide/google_app_engine ) and this is no longer true unless you make special, non-intuitive changes to your app.yaml file, in order to work around some code that may or may not work on App engine anyway (as the commit did not take this use case into account).

[theacodes]

@nathanielmanistaatgoogle App Engine used a heavily modified version of Python, which does not have ssl included by default. Users have to make a change to their app.yaml to opt-in to ssl. We can document this, but IMO it should remain optional as it was before.

[nathanielmanistaatgoogle]

@nabla-c0d3: thank you for your strong and clear response.

@jonparrott: elaborate a bit more on your opinion? Aside from gumbling in App Engine's general direction I'm not yet sure what the right thing to do here is. I'm reluctant to take on a special requirement for the library like "must not import ssl" that would be awkward or weird to verify in tests.

[theacodes]

@nathanielmanistaatgoogle grumbling in App Engine's direction is deserved, I think. I think we have two options here:

1. Be okay with the fact that we broke users in an environment that previously worked without additional config, and thus shift the burden onto our users.
2. Add import guards and unbreak users in that environment.

We can grumble at App Engine either way, it's just what we want to do between now and whenever I can convince the runtime team to turn on ssl by default.

[nathanielmanistaatgoogle]

A guarded import would wound my pride but I think I might survive. Is there an issue in a public issue tracker about turning on ssl by default to which we might link our TODO?

@rryk, any opinion on any of this?

[theacodes]

Public issue tracker for App engine is in limbo. I'm part of the runtime team, so I will bring it up in our meeting tomorrow.

[nabla-c0d3]

Sorry for insisting but I just want to re-iterate something: I don't think the newly introduced code that is using ssl would even work on App Engine.
App Engine's HTTP stack (httplib, urllib2 and all use it) is heavily customized and I would not expect it to throw an ssl.SSLError ever. I am not 100% sure and the App Engine team would know better of course, but I don't this issue is about enabling ssl on App Engine. It's about Cython-specific code (ie. code that doesn't work on App Engine) that was introduced in the commit.

[nathanielmanistaatgoogle]

@nabla-c0d3: are you sure you're not conflating CPython and Cython? They are very different things.

I'm not following your point, though - are you saying that the version of the ssl module made available on App Engine does not have the same API as the standard library's ssl module?

[nabla-c0d3]

@nathanielmanistaatgoogle yeah I meant CPython.

The ssl modules are probably the same but when making HTTP requests on App Engine, the URL Fetch service is used (https://cloud.google.com/appengine/docs/python/urlfetch/) behind the scenes even when using standard libraries like httplib or urllib2. It is, I think, a completely different implementation which would never throw an ssl.SSLError (what the commit tries to handle) since it probably doesn't even use Python's ssl module at all.

The ssl module is also a recent addition to App Engine and because it is optional, I would not expect the URL Fetch service to throw an ssl.SSLError when something fails at the SSL level; hence the code that was pushed most likely does not work on App Engine.

[rryk]

Hi, I am on vacation and don't have a good internet access or a computer to
make any changes. Import guard sounds like an acceptable solution to me
though.

On Wed, Feb 24, 2016, 14:51 Alban Diquet notifications@github.com wrote:

@nathanielmanistaatgoogle https://github.com/nathanielmanistaatgoogle
yeah I meant CPython.

The ssl modules are probably the same but when making HTTP requests on
App Engine, the URL Fetch service is used (
https://cloud.google.com/appengine/docs/python/urlfetch/) behind the
scenes (even when using standard libraries like httplib or urllib2). It is,
I think, a completely different implementation which would never throw an
ssl.SSLError (what the commit tries to handle) since it probably doesn't
even use Python's ssl module at all.

—
Reply to this email directly or view it on GitHub
#191 (comment)
.

[rctay]

I ran into this issue while on App Engine and saw this issue. Since there appears to be a consensus on import guards, I have created a pull request #220 that guardedly imports (hurhur) ssl.