ssl socket with certificate verification fails on SHA256 digest algorithm

[beda]

BPO	8484
Nosy	@loewis, @gpshead, @pitrou, @giampaolo
Files	ssl_check.py: Sample script demonstrating this error chain.pem: The necessary keychain - should be placed besides the script algos.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-04-21.20:40:22.294>
created_at = <Date 2010-04-21.11:56:18.042>
labels = ['type-bug', 'library']
title = 'ssl socket with certificate verification fails on SHA256\tdigest algorithm'
updated_at = <Date 2010-04-22.22:13:30.926>
user = 'https://bugs.python.org/beda'

bugs.python.org fields:

activity = <Date 2010-04-22.22:13:30.926>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2010-04-21.20:40:22.294>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2010-04-21.11:56:18.042>
creator = 'beda'
dependencies = []
files = ['17021', '17022', '17025']
hgrepos = []
issue_num = 8484
keywords = ['patch']
message_count = 10.0
messages = ['103823', '103827', '103851', '103862', '103882', '103887', '103892', '103909', '103982', '103984']
nosy_count = 7.0
nosy_names = ['loewis', 'gregory.p.smith', 'janssen', 'pitrou', 'giampaolo.rodola', 'beda', 'rpetrov']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8484'
versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']

[beda]

When trying a secure connection to an HTTPS server with server certificate verification, I get very strange behaviour when the digest used in the signing certificate is SHA-256 (+RSA).
On Windows with Python 2.6.4 or 2.6.5, I consistently get the following error:

ssl.SSLError: [Errno 1] _ssl.c:480: error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm

When I tried to reproduce this on Ubuntu Linux, I found that it either failed with the same error or succeeded in case the hashlib was imported before the actual code. I got the same behaviour on Gentoo Linux with Python 2.6.4 and Fedora 11 and Debian unstable with other versions of Python 2.6.
On Windows, importing hashlib prior to the code does not fix it as is does on Linux.

Using openssl s_client (openssl s_client -connect sha256.tbs-internet.com:443 -CAfile chain.pem) give no error, so the problem is not directly with openssl.
It seems that the Python ssl (_ssl) library does not load properly the corresponding hash modules from openssl or something like this.

I attach a sample script with the hashlib import commented out. I also add a pem file with certificates needed for the code to check the server certificate.

P.S.- I was able to reproduce the same behaviour with another site using SHA-256 base digests.

[pitrou]

Changing OpenSSL initialization to the following seems to fix the issue (with OpenSSL 0.9.8k and 1.0.0):

        /* Init OpenSSL */
        SSL_load_error_strings();
        SSL_library_init();
#ifdef WITH_THREAD
        /* note that this will start threading if not already started */
        if (!_setup_ssl_threads()) {
                return;
        }
#endif
        OpenSSL_add_all_algorithms();

_hashopenssl.c might need that code too, therefore I'm adding gps to the nosy list.

[pitrou]

Here is a patch. I'm not really happy about the test because it relies on an external test site, and the certificate apparently expires in 2011.

[pitrou]

After several attempts, I've failed generating a self-signed certificate that would reproduce the issue without relying on an external server. Perhaps there's something else than simply the hashing algorithm. In any case I think I'm simply gonna keep the current patch.

[loewis]

That patch makes no sense. According to SSL_library_init(3SSL),

"OpenSSL_add_ssl_algorithms() and SSLeay_add_ssl_algorithms() are synonyms for SSL_library_init()"

So it shouldn't really matter which of these you call, and it should be sufficient to call only one of them (or the documentation is incorrect).

[pitrou]

"OpenSSL_add_ssl_algorithms() and SSLeay_add_ssl_algorithms() are
synonyms for SSL_library_init()"

The patch calls OpenSSL_add_all_algorithms(), though.
(yes, it's a different one :-))
http://www.openssl.org/docs/crypto/OpenSSL_add_all_algorithms.html

[loewis]

The patch calls OpenSSL_add_all_algorithms(), though.

Ah, ok. The patch looks fine to me, then.

[pitrou]

Fixed in r80314 (trunk), r80316 (2.6), r80317 (py3k), r80318 (3.1). Thank you!

[rpetrov]

Probably test case will fail is sha256 module is not loaded - openssl versions before 0.9.8

[pitrou]

Probably test case will fail is sha256 module is not loaded - openssl versions before 0.9.8

Yes, this has been fixed in r80375.