Track down pypy test failures

[dhermes]

https://travis-ci.org/google/oauth2client/jobs/75439836
https://travis-ci.org/google/oauth2client/jobs/75439618
https://travis-ci.org/google/oauth2client/jobs/75349545

======================================================================
ERROR: test_succeeds (tests.test_crypt.Test_pkcs12_key_as_pem)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/google/oauth2client/tests/test_crypt.py", line 63, in test_succeeds
    self._succeeds_helper()
  File "/home/travis/build/google/oauth2client/tests/test_crypt.py", line 56, in _succeeds_helper
    pem_contents = crypt.pkcs12_key_as_pem(credentials.private_key, password)
  File "/home/travis/build/google/oauth2client/oauth2client/_openssl_crypt.py", line 140, in pkcs12_key_as_pem
    from OpenSSL import crypto  # Delay import due to 0.5s import time.
  File "/home/travis/build/google/oauth2client/.tox/pypy/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import rand, crypto, SSL
  File "/home/travis/build/google/oauth2client/.tox/pypy/site-packages/OpenSSL/rand.py", line 11, in <module>
    from OpenSSL._util import (
  File "/home/travis/build/google/oauth2client/.tox/pypy/site-packages/OpenSSL/_util.py", line 3, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/home/travis/build/google/oauth2client/.tox/pypy/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 11, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: No module named cryptography.hazmat.bindings._openssl

Relates to release of cryptography==1.0.

/cc @nathanielmanistaatgoogle

[dhermes]

As an aside, maybe @anthmgoogle or @craigcitro or someone else can explain why we have py26openssl13 and py27openssl13 tox environments.

Or why pyopenssl==0.14 is pinned in our test configuration when pyOpenSSL==0.15.1 was released on April 14:

• https://pypi.python.org/pypi/pyOpenSSL/0.13 (Sept. 4, 2011)
• https://pypi.python.org/pypi/pyOpenSSL/0.13.1 (Sept. 4, 2013)
• https://pypi.python.org/pypi/pyOpenSSL/0.14 (Feb. 23, 2014)
• https://pypi.python.org/pypi/pyOpenSSL/0.15 (Apr. 14, 2015)
• https://pypi.python.org/pypi/pyOpenSSL/0.15.1 (Apr. 14, 2015)

[dhermes]

Pretty simple "fix" from pyca/cryptography#2275

Cryptography 1.0 requires pypy 2.6 or newer; you appear to be using 2.5 or so. (Unfortunately that's teh version of pypy that travis ships :-()

[dhermes]

Looking at http://docs.travis-ci.com/user/languages/python/#PyPy-Support it points to https://launchpad.net/~pypy/+archive/ubuntu/ppa which currently has 2.6.0+dfsg-1~ppa0+ubuntu14.04 as the version.

[reaperhulk]

@dhermes the container infra uses pyenv and has pypy-2.5.1. You can see how we worked around it in the cryptography travis config here:

https://github.com/pyca/cryptography/blob/master/.travis/install.sh#L54 and
https://github.com/pyca/cryptography/blob/master/.travis/run.sh#L20

[dhermes]

Thanks @reaperhulk!

For book-keeping purposes, these get run as:

install:
    - ./.travis/install.sh

script:
    - ./.travis/run.sh

[craigcitro]

we have the pyopenssl configs to match the versions available somewhere -- maybe GAE? (also possibly internal.) we dropped the 0.13 one to just have fewer tests to run.

unpinning the version should be ok -- i'd do >=0.14, i think.

[dhermes]

I doubt it's GAE, I tried importing both OpenSSL and cryptography and had no luck. Neither show up in https://cloud.google.com/appengine/docs/python/tools/libraries27 either.

[craigcitro]

oh -- i might be mixing up pycrypto and pyopenssl. in that case, maybe it's just the internal version we had at the time?

[dhermes]

The pyopenssl13 targets were added by @pferate in c16eba3

---

For posterity, here is my sleuthing:

$ git log -1 --pretty=%H  # Current HEAD
3bd3fc5a903ebb73f38e473ec6decde85c270b87
$ git blame tox.ini | grep openssl13
4823cc90 (Craig Citro          2014-12-17 18:23:22 -0800 52) [testenv:py26openssl13]
4823cc90 (Craig Citro          2014-12-17 18:23:22 -0800 58) [testenv:py27openssl13]
$ git log -1 4823cc90 --pretty=%H
4823cc900c4b3f8f498db6731d68275f60ecbd4d
$ git blame 4823cc900c4b3f8f498db6731d68275f60ecbd4d^ -- tox.ini | grep openssl13
55ca16d5 (Pat Ferate   2014-07-18 09:01:46 -0700  2) envlist = py26openssl13, py26openssl14,
55ca16d5 (Pat Ferate   2014-07-18 09:01:46 -0700  3)           py27openssl13, py27openssl14,
7dff2c14 (Craig Citro  2014-12-12 17:13:22 -0800  6)           pypyopenssl13, pypyopenssl14,
c16eba36 (Pat Ferate   2014-07-17 13:19:26 -0700 25) [testenv:py26openssl13]
c16eba36 (Pat Ferate   2014-07-17 13:19:26 -0700 35) [testenv:py27openssl13]
55ca16d5 (Pat Ferate   2014-07-18 09:01:46 -0700 55) [testenv:pypyopenssl13]
$ git log -1 c16eba36 --pretty=%H
c16eba364d5602ab3e07eabd9ed5d95ec8546b7c
$ git log -1 c16eba364d5602ab3e07eabd9ed5d95ec8546b7c
commit c16eba364d5602ab3e07eabd9ed5d95ec8546b7c
Author: Pat Ferate <pferate+github@gmail.com>
Date:   Thu Jul 17 13:19:26 2014 -0700

    Setting up multiple tox test environments
$ git blame c16eba364d5602ab3e07eabd9ed5d95ec8546b7c^ -- tox.ini | grep openssl13
$ # No output means there were no py*openssl13 targets before this point

[pferate]

Looking at my comments in issue #25, it looks like there were some major changes in pyOpenSSL between 0.13 and 0.14 that caused tests to pass on 0.13 (from my dev environment at the time) but fail on 0.14 (released during my development). I wanted to test using both versions of pyOpenSSL in all of the applicable versions of Python, so that's why I added them at the time.

[dhermes]

@pferate I really appreciate the reply! You think it's safe to nuke those versions (#262)?

[pferate]

@dhermes No problem. It's been over a year since the update, so I would assume that it would be safe to drop the old version at this point.

[dhermes]

Great thanks!