Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support OpenSSL 1.1.1 #77128

Closed
tiran opened this issue Feb 25, 2018 · 21 comments
Closed

Support OpenSSL 1.1.1 #77128

tiran opened this issue Feb 25, 2018 · 21 comments
Assignees
Labels
3.7 3.8 expert-SSL type-feature

Comments

@tiran
Copy link
Member

@tiran tiran commented Feb 25, 2018

BPO 32947
Nosy @vstinner, @tiran, @benjaminp, @ned-deily, @alex, @dstufft, @stratakis, @hroncok
PRs
  • #5663
  • #5923
  • #8760
  • #8761
  • #11612
  • #11612
  • #11612
  • #11876
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tiran'
    closed_at = <Date 2019-05-29.02:06:57.441>
    created_at = <Date 2018-02-25.11:54:40.611>
    labels = ['expert-SSL', '3.8', 'type-feature', '3.7']
    title = 'Support OpenSSL 1.1.1'
    updated_at = <Date 2019-05-29.02:06:57.440>
    user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2019-05-29.02:06:57.440>
    actor = 'ned.deily'
    assignee = 'christian.heimes'
    closed = True
    closed_date = <Date 2019-05-29.02:06:57.441>
    closer = 'ned.deily'
    components = ['SSL']
    creation = <Date 2018-02-25.11:54:40.611>
    creator = 'christian.heimes'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 32947
    keywords = ['patch']
    message_count = 21.0
    messages = ['312804', '312897', '317529', '317575', '322490', '322795', '323526', '323901', '323903', '323909', '325793', '325795', '325926', '326719', '326720', '333990', '335623', '342616', '342617', '343853', '343854']
    nosy_count = 10.0
    nosy_names = ['janssen', 'vstinner', 'christian.heimes', 'benjamin.peterson', 'ned.deily', 'alex', 'dstufft', 'kroeckx', 'cstratak', 'hroncok']
    pr_nums = ['5663', '5923', '8760', '8761', '11612', '11612', '11612', '11876']
    priority = 'high'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue32947'
    versions = ['Python 3.7', 'Python 3.8']

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Feb 25, 2018

    I'm using this ticket as an epos to track commits and required changes for OpenSSL 1.1.1 and TLS 1.3. Fixes need to be backported to 2.7 and 3.6 to 3.8. We might have to consider backports to 3.4 and 3.5, too.

    If all goes to plan, OpenSSL 1.1.1 final is scheduled for 8th May 2018, https://www.openssl.org/policies/releasestrat.html . It will contain support for TLS 1.3. Python should either support TLS 1.3 by then or disable TLS 1.3 by default.

    Fixes:

    Issues:

    • A new option OP_ENABLE_MIDDLEBOX_COMPAT is enabled by default. We need to expose the flag to make test pass.
    • TLS 1.3 has changed session handling. The current session code cannot handle TLS 1.3 session resumption.
    • Threaded echo server and asynchat based tests are failing with TLS 1.3. I haven't analyzed the issue properly. It looks like the server thread dies when a handshake error occurs.

    @tiran tiran self-assigned this Feb 25, 2018
    @tiran tiran added expert-SSL type-feature labels Feb 25, 2018
    @tiran
    Copy link
    Member Author

    @tiran tiran commented Feb 26, 2018

    Ned, Benjamin

    OpenSSL 1.1.1 is scheduled to be released just before 3.7.0rc1 will come out. I'd rather address as many issues now instead of adding last minute patches to the release candidate.

    Once OpenSSL 1.1.1 is out and Python 3.7 supports it, we can discuss backports. 3.6 and 2.7 are going to need fixing, too.

    @ned-deily
    Copy link
    Member

    @ned-deily ned-deily commented May 24, 2018

    Christian, I'm not sure how this issue now differs from bpo-33618 (and whether it can be closed as a duplicate) but, with the delay in OpenSSL 1.1.1 and as discussed over there, full 1.1.1 support will have to wait for 3.7.1 et al so I'm removing the "deferred blocker" status.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented May 24, 2018

    3.7 and 3.8 support OpenSSL 1.1.1-pre7-dev. For 3.6 and 2.7 I have to backport some test fixes and documentation.

    I prefer to wait until both TLS 1.3 and OpenSSL 1.1.1 have been finalized. Once 1.1.1 is out, I'll fix the outstanding issues on master and then backport all fixes to 3.6 / 2.7.

    @hroncok
    Copy link
    Mannequin

    @hroncok hroncok mannequin commented Jul 27, 2018

    Note that Fedora 29 updated openssl from 1.1.0h to 1.1.1-0.pre8 and Python 3.7 tests are failing. Not 100% sure it's related, but full report at:

    https://bugzilla.redhat.com/show_bug.cgi?id=1609291

    @stratakis
    Copy link
    Mannequin

    @stratakis stratakis mannequin commented Jul 31, 2018

    Yes test_poplib and test_ftplib on fedora rawhide when run against openssl 1.1.1 pre8. Haven't tried the pr7, but assuming that the tests were fine before here is the list of changes between pre7 and pre8:

    openssl/openssl@OpenSSL_1_1_1-pre7...OpenSSL_1_1_1-pre8

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Aug 14, 2018

    New changeset 2a4ee8a by Christian Heimes in branch '3.6':
    bpo-32947: Fixes for TLS 1.3 and OpenSSL 1.1.1 (GH-8761)
    2a4ee8a

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Aug 22, 2018

    Kurt, can you try again with a current git checkout from master? I fixed a couple of issues lately. CPython master passes all tests with vanilla OpenSSL 1.1.1-pre9. Does Debian change some default settings?

    @kroeckx
    Copy link
    Mannequin

    @kroeckx kroeckx mannequin commented Aug 22, 2018

    This are automated tests for the packages in Debian. I uploaded the pre9 version to unstable, and as a result of that all reverse dependencies got tested. I don't have any experience with python myself.

    Anyway, the openssl.cnf in Debian contains:
    [system_default_sect]
    MinProtocol = TLSv1.2
    CipherString = DEFAULT@SECLEVEL=2

    So you might want to override that during the test suite by calling SSL_CTX_set_min_proto_version(ctx, 0).

    @kroeckx
    Copy link
    Mannequin

    @kroeckx kroeckx mannequin commented Sep 19, 2018

    Christian,

    Do you have any update on this? Any idea when we can expect relased python versions that work with OpenSSL 1.1.1?

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Sep 19, 2018

    Soonish, I'm still working on post handshake auth.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Sep 20, 2018

    Kurt, see bpo-34670 for PHA for server and client side.

    @kroeckx
    Copy link
    Mannequin

    @kroeckx kroeckx mannequin commented Sep 30, 2018

    Do you have any idea when the next release will be? I think python is currently our biggest blocker for getting OpenSSL 1.1.1 in Debian testing.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Sep 30, 2018

    The release candidates came out a couple of days ago.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Jan 18, 2019

    On Fedora 29 with OpenSSL 1.1.1 FIPS 11 Sep 2018, test_connect_cadata() of test_ssl fails randomly:

    ---

    $ ./python -m test -u all -F -m test_connect_cadata test_ssl 
    Run tests sequentially
    0:00:00 load avg: 0.43 [  1] test_ssl
    test test_ssl failed -- Traceback (most recent call last):
      File "/home/vstinner/prog/python/3.6/Lib/test/test_ssl.py", line 1642, in test_connect_cadata
        s.connect(self.server_addr)
      File "/home/vstinner/prog/python/3.6/Lib/ssl.py", line 1109, in connect
        self._real_connect(addr, False)
      File "/home/vstinner/prog/python/3.6/Lib/ssl.py", line 1100, in _real_connect
        self.do_handshake()
      File "/home/vstinner/prog/python/3.6/Lib/ssl.py", line 1077, in do_handshake
        self._sslobj.do_handshake()
      File "/home/vstinner/prog/python/3.6/Lib/ssl.py", line 689, in do_handshake
        self._sslobj.do_handshake()
    ConnectionResetError: [Errno 104] Connection reset by peer

    test_ssl failed

    == Tests result: FAILURE ==

    1 test failed:
    test_ssl

    Total duration: 131 ms
    Tests result: FAILURE
    ---

    This bug has been fixed in master by commit 529525f. It was partially backported in 3.6 with commit 2a4ee8a, but the backport is incomplete.

    I wrote PR 11612 to backport remaining fixes.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Feb 15, 2019

    New changeset 2149a9a by Victor Stinner (stratakis) in branch '2.7':
    [2.7] bpo-32947: Fixes for TLS 1.3 and OpenSSL 1.1.1 (GH-8761) (GH-11876)
    2149a9a

    @benjaminp
    Copy link
    Contributor

    @benjaminp benjaminp commented May 15, 2019

    Was using OpenSSL to verify hostnames intentionally not backported?

    @tiran
    Copy link
    Member Author

    @tiran tiran commented May 15, 2019

    Yes, the feature requires OpenSSL 1.0.2 and a more recent version of LibreSSL. 2.7 and 3.6 branches still target platforms with ancient versions of OpenSSL (e.g. Ubuntu 14.04 has 1.0.1f + patches). People were complain A LOT, because there were not able to install Python 3.7 on TravisCI. Like really a lot, alot.

    I propose to close this bug as fixed in 3.7+

    @ned-deily
    Copy link
    Member

    @ned-deily ned-deily commented May 29, 2019

    New changeset 3dbc43f by Ned Deily (Victor Stinner) in branch '3.6':
    bpo-32947: test_ssl fixes for TLS 1.3 and OpenSSL 1.1.1 (GH-11612)
    3dbc43f

    @ned-deily
    Copy link
    Member

    @ned-deily ned-deily commented May 29, 2019

    I don't have a strong opinion about backporting to 3.6. With OpenSSL 1.0.2 official support ending at the end of 2019 and 3.6.z retired towards the ned of 2021, there would be a 2-year window where 3.6 is still in security-fix-only status. But, if we don't do the backport now, we could always choose to backport it later if the need arises.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 3.8 expert-SSL type-feature
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants