FAIL: test_set_verify_callback_exception on FreeBSD (pyopenssl 0.14) #38

Open
koobs opened this Issue Feb 24, 2014 · 16 comments

Comments

Projects
None yet
3 participants

koobs commented Feb 24, 2014

OS: FreeBSD 9.2-STABLE #0 r260870
Python: 2.7.6

======================================================================
FAIL: test_set_verify_callback_exception (OpenSSL.test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/home/user/repos/freebsd/ports/security/py-openssl/work/pyca-pyopenssl-0146d44/OpenSSL/test/test_ssl.py", line 1002, in test_set_verify_callback_exception
    Exception, self._handshake_test, serverContext, clientContext)
  File "/mnt/home/user/repos/freebsd/ports/security/py-openssl/work/pyca-pyopenssl-0146d44/OpenSSL/test/util.py", line 270, in failUnlessRaises
    % (exception.__name__, result))
AssertionError: Exception not raised (None returned)
----------------------------------------------------------------------
Contributor

kouk commented Feb 24, 2014

What about OpenSSL.test.test_ssl.ContextTests.test_set_default_verify_paths? On my FreeBSD system that throws an error because my OpenSSL installation does not have any default trusted certificates. This might not be a problem for you of course, but you can check with $ openssl s_client -connect verisign.com:443. If it says "cannot get local issuer certificate" then probably that's the problem.

Contributor

kouk commented Feb 24, 2014

Having said that, I should add that the test also fails for me on FreeBSD regardless of whether verification is properly set up.

koobs commented Feb 24, 2014

@kouk This was the only test failure I saw (other than unrelated #37)

Verify return code: 20 (unable to get local issuer certificate)

FWIW, security/ca_root_nss is installed with ETCSYMLINK option

Can the test be made robust, to either skip or warn based on this? What is actually being tested here?

Contributor

kouk commented Feb 25, 2014

For me this specific problem was solved with f012467. The sleep value could be even lower for me but I left it at 50ms. By the way I get another 8 failures (9 if you count #37) on this old FreeBSD laptop of mine, but I haven't had time to check them out.

Contributor

kouk commented Feb 25, 2014

@koobs do you have security/openssl installed or did you build against /usr/lib/libssl.so ? What version? 7/8 fails I was getting were due to /usr/bin being before /usr/local/bin in PATH and a different openssl version was being used by the test run. But I'm getting a fail from OpenSSL.test.test_crypto.X509Tests.test_digest because the calculated digests are different.

koobs commented Feb 25, 2014

Hmm, so I found an issue (minor) which led me to something else.

Prior to 0.14, the py-openssl port depended on OpenSSL. Now that pyopenssl uses cryptography for its OpenSSL bits, I neglected to move the openssl depends from py-openssl to py-cryptography.

So I checked links for the .so's created there and found:

./build/lib.freebsd-9.2-STABLE-amd64-2.7/cryptography/_cffi__x9ee276ddxb4fc9075.so:
        libcrypto.so.6 => /lib/libcrypto.so.6 (0x80125f000)
        libssl.so.6 => /usr/lib/libssl.so.6 (0x801607000)
        libc.so.7 => /lib/libc.so.7 (0x80081d000)

I do have security/openssl installed, and the dependency was 'supposed' to have linked against that in /usr/local.

Here's what the py-openssl port does (that is no longer relevant since 0.14) to set the correct include/library paths:

pre-configure:
        @${ECHO_CMD} "[build_ext]" >> ${WRKSRC}/setup.cfg
        @${ECHO_CMD} "include-dirs = ${OPENSSLINC}" >> ${WRKSRC}/setup.cfg
        @${ECHO_CMD} "library-dirs = ${OPENSSLLIB}" >> ${WRKSRC}/setup.cfg

I imagine I now need to do the same for cryptography, but it does lend weight to your theory. Why and how do the tests use different bits when pyopenssl uses cryptography for all its functions?

Contributor

kouk commented Feb 25, 2014

The tests that were failing were just running the openssl command to get "good" output to compare against the test output. There were some formatting differences in the output of /usr/bin/openssl so the tests were failing.

As for cryptography, I built it by hand in a virtualenv but first did:

export CFLAGS="-I/usr/local/include -L/usr/local/lib -DCRYPTO_MDEBUG" 

This is how I managed to get cffi which is used by cryptography to pass the correct flags to the compiler. I don't know of any better way atm.

koobs commented Feb 25, 2014

Adding:

CFLAGS+=        -I${OPENSSLINC}
LDFLAGS+=       -L${OPENSSLLIB}

To cryptography's build, results in:

ldd ./work/cryptography-0.2.1/build/lib.freebsd-9.2-STABLE-amd64-2.7/cryptography

/_cffi__x9ee276ddxb4fc9075.so
./work/cryptography-0.2.1/build/lib.freebsd-9.2-STABLE-amd64-2.7/cryptography/_cffi__x9ee276ddxb4fc9075.so:
        libcrypto.so.8 => /usr/local/lib/libcrypto.so.8 (0x80125f000)
        libssl.so.8 => /usr/local/lib/libssl.so.8 (0x80165a000)
        libc.so.7 => /lib/libc.so.7 (0x80081d000)
        libthr.so.3 => /lib/libthr.so.3 (0x8018c0000)

Tests:

 62912 passed, 12 skipped in 145.50 seconds

koobs commented Feb 25, 2014

With py-cryptography compiled & linked against OpenSSL from ports, the following tests fail in pyopenssl:

Ran 352 tests in 1.774s -FAILED (failures=10, errors=1)

FAIL: test_dump_certificate_request (OpenSSL.test.test_crypto.FunctionTests)
FAIL: test_export_without_args (OpenSSL.test.test_crypto.PKCS12Tests)
FAIL: test_export_without_mac (OpenSSL.test.test_crypto.PKCS12Tests)
FAIL: test_friendly_name (OpenSSL.test.test_crypto.PKCS12Tests)
FAIL: test_various_empty_passphrases (OpenSSL.test.test_crypto.PKCS12Tests)
FAIL: test_digest (OpenSSL.test.test_crypto.X509Tests)
FAIL: test_wantWriteError (OpenSSL.test.test_ssl.ConnectionTests)
FAIL: test_set_verify_callback_exception (OpenSSL.test.test_ssl.ContextTests)

/usr/bin/openssl version

OpenSSL 0.9.8y 5 Feb 2013

/usr/local/bin/openssl version

OpenSSL 1.0.1f 6 Jan 2014
Contributor

kouk commented Feb 25, 2014

ok, we have the same versions. Also I believe if you make sure that /usr/local/bin is before /usr/bin in $PATH all tests will pass except test_digest. I opened #41 for test_digest in particular.

koobs commented Feb 25, 2014

Not sure if you expected this:

PATH="/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin" /usr/bin/make regression-test

ERROR: test_set_default_verify_paths (OpenSSL.test.test_ssl.ContextTests)
Error: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
FAIL: test_digest (OpenSSL.test.test_crypto.X509Tests)
AssertionError: 'C1:B5:90:A4:41:11:C8:30:BF:D4:AA:78:13:46:66:59' != 'A8:EB:07:F8:53:25:0A:F2:56:05:C5:A5:C4:C4:C7:15'
FAIL: test_wantWriteError (OpenSSL.test.test_ssl.ConnectionTests)
AssertionError: <class 'OpenSSL.SSL.WantReadError'> raised instead of WantWriteError
FAIL: test_set_verify_callback_exception (OpenSSL.test.test_ssl.ContextTests)
AssertionError: Exception not raised (None returned)
Ran 352 tests in 1.835s - FAILED (failures=3, errors=1)

Note: regression-test is just the port make target that runs python setup.py test

How might the tests be made to run without requiring the PATH be modified?

koobs commented Feb 25, 2014

FWIW, if you're using FreeBSD, I've committed the latest update to the security/py-cryptography port, which has already propagated to the portsnap mirrors if you want it:

http://svnweb.freebsd.org/ports?view=revision&revision=345962

CFLAGS & LDFLAGS environment variables were used.

Contributor

kouk commented Feb 25, 2014

Hmm for test_set_default_verify_paths try export SSL_CERT_FILE=/usr/local/share/certs/ca-root-nss.crt.
Apart from test_digest the other problems are fixed I think in #39 and #43.
Also, thanks for committing the port. Although my work with pyopenssl isn't on FreeBSD (Windows unfortunately) it's nice to be able to get it for cheap on my workstation.

Contributor

kouk commented Feb 25, 2014

about the modified PATH requirement perhaps there is another to get "good" test output apart from running the openssl binary, but off the top of my head I can't think of one.

koobs commented Feb 25, 2014

If the definition of good is that the underlying system openssl output matches that which comes out of the library you leverage for crypto, then it sounds like a test that only cryptography can do, since it has private knowledge of the environment its been built within.

Perhaps the rest of the @pyca crew could provide some input on making these tests more robust?

0-wiz-0 commented Mar 21, 2014

The test fails for me on NetBSD using pyOpenSSL-0.14 and cryptography-0.2.2.

FAIL: test_set_verify_callback_exception (OpenSSL.test.test_ssl.ContextTests)

Traceback (most recent call last):
File "/scratch/security/py-OpenSSL/work/pyOpenSSL-0.14/OpenSSL/test/test_ssl.py", line 1002, in test_set_verify_callback_exception
Exception, self._handshake_test, serverContext, clientContext)
File "/scratch/security/py-OpenSSL/work/pyOpenSSL-0.14/OpenSSL/test/util.py", line 270, in failUnlessRaises
% (exception.__name__, result))
AssertionError: Exception not raised (None returned)

This is in a sandbox with no certificates installed. If the test needs any, IMHO it should install them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment