Python fails to compile/load _ssl module if OpenSSL is compiled with no-tls1-method

[MitchLindgren]

BPO	40457
Nosy	@tiran, @manueljacob, @miss-islington
PRs	bpo-40457: Support OpenSSL without TLS 1.0/1.1 #19862 [3.7] bpo-40457: Support OpenSSL without TLS 1.0/1.1 (GH-19862) (GH-20126) #20126 [3.8] bpo-40457: Support OpenSSL without TLS 1.0/1.1 (GH-19862) #20127

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = <Date 2020-05-16.08:45:54.658>
created_at = <Date 2020-04-30.22:41:34.245>
labels = ['expert-SSL', '3.8', 'build', '3.7', '3.9']
title = 'Python fails to compile/load _ssl module if OpenSSL is compiled with no-tls1-method'
updated_at = <Date 2020-06-01.01:29:21.428>
user = 'https://bugs.python.org/MitchLindgren'

bugs.python.org fields:

activity = <Date 2020-06-01.01:29:21.428>
actor = 'mjacob'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2020-05-16.08:45:54.658>
closer = 'christian.heimes'
components = ['SSL']
creation = <Date 2020-04-30.22:41:34.245>
creator = 'Mitch Lindgren'
dependencies = []
files = []
hgrepos = []
issue_num = 40457
keywords = ['patch']
message_count = 9.0
messages = ['367793', '367797', '367934', '367952', '368997', '369022', '369023', '369024', '370525']
nosy_count = 4.0
nosy_names = ['christian.heimes', 'mjacob', 'miss-islington', 'Mitch Lindgren']
pr_nums = ['19862', '20126', '20127']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue40457'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[MitchLindgren]

I'm working on a project which uses OpenSSL 1.1.1g. For security and compliance reasons, it is built with SSL and TLS < 1.2 methods compiled out, using the following OpenSSL build options:

no-ssl no-ssl3 no-tls1 no-tls1_1 no-ssl3-method no-tls1-method no-tls1_1-method

When compiling Python v3.8.2 with CFLAGS="-DOPENSSL_NO_SSL2 -DOPENSSL_NO_SSL3 -DOPENSSL_NO_TLS1 -DOPENSSL_NO_TLS1_1" and --with-openssl=/path/to/custom/openssl, _ssl.c fails to compile with the following error:

gcc -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DOPENSSL_NO_SSL2 -DOPENSSL_NO_SSL3 -DOPENSSL_NO_TLS1 -DOPENSSL_NO_TLS1_1 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_SSL3 -DOPENSSL_NO_TLS1 -DOPENSSL_NO_TLS1_1 -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I/home/mitch/openssl/include -I./Include -I. -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/home/mitch/cpython/Include -I/home/mitch/cpython -c /home/mitch/cpython/Modules/_ssl.c -o build/temp.linux-x86_64-3.8/home/mitch/cpython/Modules/_ssl.o
/home/mitch/cpython/Modules/_ssl.c: In function ‘_ssl__SSLContext_impl’:
/home/mitch/cpython/Modules/_ssl.c:3088:27: error: implicit declaration of function ‘TLSv1_method’; did you mean ‘DTLSv1_method’? [-Werror=implicit-function-declaration]
ctx = SSL_CTX_new(TLSv1_method());
^~~~~~~~~~~~
DTLSv1_method
/home/mitch/cpython/Modules/_ssl.c:3088:27: warning: passing argument 1 of ‘SSL_CTX_new’ makes pointer from integer without a cast [-Wint-conversion]
In file included from /home/mitch/cpython/Modules/_ssl.c:62:0:
/home/mitch/openssl/include/openssl/ssl.h:1503:17: note: expected ‘const SSL_METHOD * {aka const struct ssl_method_st *}’ but argument is of type ‘int’
__owur SSL_CTX *SSL_CTX_new(const SSL_METHOD *meth);
^~~~~~~~~~~
/home/mitch/cpython/Modules/_ssl.c:3091:27: error: implicit declaration of function ‘TLSv1_1_method’; did you mean ‘TLSv1_2_method’? [-Werror=implicit-function-declaration]
ctx = SSL_CTX_new(TLSv1_1_method());
^~~~~~~~~~~~~~
TLSv1_2_method
/home/mitch/cpython/Modules/_ssl.c:3091:27: warning: passing argument 1 of ‘SSL_CTX_new’ makes pointer from integer without a cast [-Wint-conversion]
In file included from /home/mitch/cpython/Modules/_ssl.c:62:0:
/home/mitch/openssl/include/openssl/ssl.h:1503:17: note: expected ‘const SSL_METHOD * {aka const struct ssl_method_st *}’ but argument is of type ‘int’
__owur SSL_CTX *SSL_CTX_new(const SSL_METHOD *meth);
^~~~~~~~~~~
cc1: some warnings being treated as errors

This also affects older versions. With v3.5.6, the _ssl module compiles successfully (it may be getting the declaration of TLSv1_method from the system default OpenSSL header since the --with-openssl option doesn't exist in this version), but importing the module at runtime fails:

root@10:/tmp/acmstest# python3
Python 3.5.6 (default, Mar 23 2020, 05:11:33)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/ssl.py", line 99, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: /usr/lib/python3.5/lib-dynload/_ssl.cpython-35m-aarch64-linux-gnu.so: undefined symbol: TLSv1_method

[MitchLindgren]

I'd be happy to work on a patch for this. I think the simplest approach would be to change this block starting on line 3087:

    if (proto_version == PY_SSL_VERSION_TLS1)
        ctx = SSL_CTX_new(TLSv1_method());
#if HAVE_TLSv1_2
    else if (proto_version == PY_SSL_VERSION_TLS1_1)
        ctx = SSL_CTX_new(TLSv1_1_method());
    else if (proto_version == PY_SSL_VERSION_TLS1_2)
        ctx = SSL_CTX_new(TLSv1_2_method());
#endif
#ifndef OPENSSL_NO_SSL3
    else if (proto_version == PY_SSL_VERSION_SSL3)
        ctx = SSL_CTX_new(SSLv3_method());
#endif
#ifndef OPENSSL_NO_SSL2
    else if (proto_version == PY_SSL_VERSION_SSL2)
        ctx = SSL_CTX_new(SSLv2_method());
#endif
    else if (proto_version == PY_SSL_VERSION_TLS) /* SSLv23 */
        ctx = SSL_CTX_new(TLS_method());
    else if (proto_version == PY_SSL_VERSION_TLS_CLIENT)
        ctx = SSL_CTX_new(TLS_client_method());
    else if (proto_version == PY_SSL_VERSION_TLS_SERVER)
        ctx = SSL_CTX_new(TLS_server_method());
    else
        proto_version = -1;

into a switch and add additional #if !defined(OPENSSL_NO_XXX) macros to exclude version-specific methods. Please let me know if this sounds okay.

[tiran]

Thanks for the bug report. I've created a PR to check for the correct flags in _ssl__SSLContext_impl(). I'll backport the fix to 3.8 and 3.7. 3.6 and older are in security-only mode.

[MitchLindgren]

Thanks for the quick turnaround!

[miss-islington]

New changeset 6e8cda9 by Christian Heimes in branch 'master':
bpo-40457: Support OpenSSL without TLS 1.0/1.1 (GH-19862)
6e8cda9

[miss-islington]

New changeset a669443 by Miss Islington (bot) in branch '3.8':
bpo-40457: Support OpenSSL without TLS 1.0/1.1 (GH-19862)
a669443

[tiran]

New changeset 43b355e by Miss Islington (bot) in branch '3.7':
[3.7] bpo-40457: Support OpenSSL without TLS 1.0/1.1 (GH-19862) (GH-20126)
43b355e

[tiran]

Fixes have landed in 3.7 to 3.9.

Thanks for the report! :)

[manueljacob]

For the record, I’ve added a comment to the pull request about that ssl.PROTOCOL_TLSv1_1 / ssl.PROTOCOL_TLSv1_2 are now defined unconditionally.

6e8cda9#r39569316