test_ssl: test_msg_callback_deadlock_bpo43577() failed on macOS of GitHub Action

[vstinner]

BPO	43855
Nosy	@vstinner, @tiran, @erlend-aasland
Superseder	bpo-44229: test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-06-24.12:27:43.459>
created_at = <Date 2021-04-15.12:49:57.023>
labels = ['tests', '3.10']
title = 'test_ssl: test_msg_callback_deadlock_bpo43577() failed on macOS of GitHub Action'
updated_at = <Date 2021-06-24.12:27:43.458>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-06-24.12:27:43.458>
actor = 'erlendaasland'
assignee = 'none'
closed = True
closed_date = <Date 2021-06-24.12:27:43.459>
closer = 'erlendaasland'
components = ['Tests']
creation = <Date 2021-04-15.12:49:57.023>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 43855
keywords = []
message_count = 3.0
messages = ['391131', '394753', '396480']
nosy_count = 3.0
nosy_names = ['vstinner', 'christian.heimes', 'erlendaasland']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '44229'
type = None
url = 'https://bugs.python.org/issue43855'
versions = ['Python 3.10']

[vstinner]

macOS of GitHub Action failed:
https://github.com/python/cpython/pull/25400/checks?check_run_id=2344085246

ERROR: test_msg_callback_deadlock_bpo43577 (test.test_ssl.TestSSLDebug)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Users/runner/work/cpython/cpython/Lib/test/test_ssl.py", line 4799, in test_msg_callback_deadlock_bpo43577
    s.connect((HOST, server.port))
  File "/Users/runner/work/cpython/cpython/Lib/ssl.py", line 1342, in connect
    self._real_connect(addr, False)
  File "/Users/runner/work/cpython/cpython/Lib/ssl.py", line 1329, in _real_connect
    super().connect(addr)
ConnectionRefusedError: [Errno 61] Connection refused

Test code:

    def test_msg_callback_deadlock_bpo43577(self):
        client_context, server_context, hostname = testing_context()
        server_context2 = testing_context()[1]

        def msg_cb(conn, direction, version, content_type, msg_type, data):
            pass

        def sni_cb(sock, servername, ctx):
            sock.context = server_context2

        server_context._msg_callback = msg_cb
        server_context.sni_callback = sni_cb

        server = ThreadedEchoServer(context=server_context, chatty=False)
        with server:
            with client_context.wrap_socket(socket.socket(),
                                            server_hostname=hostname) as s:
                s.connect((HOST, server.port))
            with client_context.wrap_socket(socket.socket(),
                                            server_hostname=hostname) as s:
                s.connect((HOST, server.port))  # <===== FAIL HERE 

====

test.pythoninfo:

ssl.HAS_SNI: True
ssl.OPENSSL_VERSION: OpenSSL 1.1.1k 25 Mar 2021
ssl.OPENSSL_VERSION_INFO: (1, 1, 1, 11, 15)
ssl.OP_ALL: 0x80000054
ssl.OP_NO_TLSv1_1: 0x10000000
ssl.SSLContext.maximum_version: MAXIMUM_SUPPORTED
ssl.SSLContext.minimum_version: MINIMUM_SUPPORTED
ssl.SSLContext.options: OP_NO_COMPRESSION|OP_ENABLE_MIDDLEBOX_COMPAT|OP_CIPHER_SERVER_PREFERENCE|OP_NO_SSLv3|0x80000054
ssl.SSLContext.protocol: PROTOCOL_TLS
ssl.SSLContext.verify_mode: CERT_NONE
ssl.default_https_context.maximum_version: MAXIMUM_SUPPORTED
ssl.default_https_context.minimum_version: MINIMUM_SUPPORTED
ssl.default_https_context.options: OP_NO_COMPRESSION|OP_ENABLE_MIDDLEBOX_COMPAT|OP_CIPHER_SERVER_PREFERENCE|OP_NO_SSLv3|0x80000054
ssl.default_https_context.protocol: PROTOCOL_TLS
ssl.default_https_context.verify_mode: CERT_REQUIRED
ssl.stdlib_context.maximum_version: MAXIMUM_SUPPORTED
ssl.stdlib_context.minimum_version: MINIMUM_SUPPORTED
ssl.stdlib_context.options: OP_NO_COMPRESSION|OP_ENABLE_MIDDLEBOX_COMPAT|OP_CIPHER_SERVER_PREFERENCE|OP_NO_SSLv3|0x80000054
ssl.stdlib_context.protocol: PROTOCOL_TLS
ssl.stdlib_context.verify_mode: CERT_NONE

The test was added recently:

commit 77cde50
Author: Christian Heimes <christian@python.org>
Date: Sun Mar 21 16:13:09 2021 +0100

bpo-43577: Fix deadlock with SSLContext._msg_callback and sni_callback (GH-24957)

OpenSSL copies the internal message callback from SSL_CTX->msg_callback to
SSL->msg_callback. SSL_set_SSL_CTX() does not update SSL->msg_callback
to use the callback value of the new context.

PySSL_set_context() now resets the callback and _PySSL_msg_callback()
resets thread state in error path.

Signed-off-by: Christian Heimes <christian@python.org>

[erlend-aasland]

See also gh-88395 (bpo-44229).

[erlend-aasland]

Duplicate of #88395. Please reopen if you disagree.