test_ssl: skip tests for disabled TLS/SSL versions

[tiran]

BPO	38275
Nosy	@vstinner, @tiran, @miss-islington
PRs	bpo-38275: Skip ssl tests for disabled versions #16386 [3.8] bpo-38275: Skip ssl tests for disabled versions (GH-16386) #16425 [3.7] bpo-38275: Skip ssl tests for disabled versions (GH-16427) #16427 bpo-38275: Fix test_ssl issue caused by GH-16386 #16428

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = None
created_at = <Date 2019-09-25.12:38:51.389>
labels = ['3.8', '3.7', 'tests', '3.9', 'performance']
title = 'test_ssl: skip tests for disabled TLS/SSL versions'
updated_at = <Date 2019-09-26.20:54:37.031>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2019-09-26.20:54:37.031>
actor = 'christian.heimes'
assignee = 'christian.heimes'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2019-09-25.12:38:51.389>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 38275
keywords = ['patch']
message_count = 6.0
messages = ['353189', '353315', '353324', '353326', '353337', '353338']
nosy_count = 3.0
nosy_names = ['vstinner', 'christian.heimes', 'miss-islington']
pr_nums = ['16386', '16425', '16427', '16428']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue38275'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[tiran]

test_ssl has some checks for TLS/SSL versions that are not available or disabled during compile-time of Python and OpenSSL. It's missing run-time checks and checks for TLS 1.0 and 1.1. Several test scenarios assume that TLS 1.0 and 1.1 are always available and properly working. These tests fail when e.g. TLS 1.0 is disabled.

Further more compile-time checks for ssl.HAS_TLSv1 or ssl.PROTOCOL_TLSv1 are not sufficient. OpenSSL 1.1.0+ got more flexible. TLS/SSL versions can be disabled at runtime with a crypto policy. Recent Debian, Fedora, and RHEL 8 systems have TLS 1.0 and 1.1 disabled by default.

[miss-islington]

New changeset df6ac7e by Miss Islington (bot) (Christian Heimes) in branch 'master':
bpo-38275: Skip ssl tests for disabled versions (GH-16386)
df6ac7e

[vstinner]

The change caused many buildbot failures, see examples from IRC logs below. I understand that PR 16428 will fix them.

17:22 < py-bb> Build s390x SLES 3.x bpo-3824 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/16/builds/3824
17:23 < py-bb> Build AMD64 RHEL7 3.x #46 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/254/builds/46
17:24 < py-bb> Build s390x RHEL 3.x bpo-3873 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/21/builds/3873
17:32 < py-bb> Build AMD64 RHEL7 LTO + PGO 3.x #46 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/258/builds/46
17:38 < py-bb> Build AMD64 RHEL7 LTO 3.x #47 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/283/builds/47
17:41 < py-bb> Build PPC64 Fedora 3.x bpo-3789 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/53/builds/3789
17:53 < py-bb> Build x86 Gentoo Installed with X 3.x bpo-3148 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/103/builds/3148
17:58 < py-bb> Build x86 Gentoo Non-Debug with X 3.x bpo-3377 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/99/builds/3377
18:00 < py-bb> Build POWER6 AIX 3.x bpo-1710 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/161/builds/1710
18:04 < py-bb> Build PPC64 AIX 3.x bpo-3477 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/10/builds/3477
18:08 < py-bb> Build PPC64LE Fedora 3.x bpo-3617 is complete: Failure [failed test (failure)] - https://buildbot.python.org/all/#builders/85/builds/3617

[tiran]

New changeset 9f77268 by Christian Heimes in branch 'master':
bpo-38275: Fix test_ssl issue caused by #60590 (bpo-16428)
9f77268

[tiran]

New changeset 1931132 by Christian Heimes in branch '3.8':
[3.8] bpo-38275: Skip ssl tests for disabled versions (GH-16386) (GH-16425)
1931132

[tiran]

New changeset 2c24f2c by Christian Heimes in branch '3.7':
[3.7] bpo-38275: Skip ssl tests for disabled versions (GH-16427)
2c24f2c

[kumaraditya303]

Fixed by #16386