Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

self-signed.pythontest.net TLS certificate key is too weak #80997

Closed
gpshead opened this issue May 6, 2019 · 15 comments
Closed

self-signed.pythontest.net TLS certificate key is too weak #80997

gpshead opened this issue May 6, 2019 · 15 comments
Assignees
Labels
3.7 (EOL) end of life 3.8 only security fixes tests Tests in the Lib/test dir topic-SSL type-bug An unexpected behavior, bug, or error

Comments

@gpshead
Copy link
Member

gpshead commented May 6, 2019

BPO 36816
Nosy @gpshead, @larryhastings, @ned-deily, @aixtools, @yan12125, @ewdurbin, @miss-islington
PRs
  • bpo-35925: Skip SSL tests that fail due to weak external certs. #13124
  • bpo-36816: Update the self-signed.pythontest.net cert #13192
  • [3.7] bpo-36816: Update the self-signed.pythontest.net cert (GH-13192) #13197
  • [3.6] bpo-36816: Update the self-signed.pythontest.net cert (GH-13192) #13198
  • [2.7] bpo-36816: Update the self-signed.pythontest.net cert (GH-13192) #13199
  • [3.5] bpo-36816: Update the self-signed.pythontest.net cert (GH-13192) #13200
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/gpshead'
    closed_at = <Date 2019-09-09.09:39:04.028>
    created_at = <Date 2019-05-06.18:05:40.595>
    labels = ['3.7', 'expert-SSL', '3.8', 'type-bug', 'tests']
    title = 'self-signed.pythontest.net TLS certificate key is too weak'
    updated_at = <Date 2019-09-09.09:39:04.027>
    user = 'https://github.com/gpshead'

    bugs.python.org fields:

    activity = <Date 2019-09-09.09:39:04.027>
    actor = 'gregory.p.smith'
    assignee = 'gregory.p.smith'
    closed = True
    closed_date = <Date 2019-09-09.09:39:04.028>
    closer = 'gregory.p.smith'
    components = ['Tests', 'SSL']
    creation = <Date 2019-05-06.18:05:40.595>
    creator = 'gregory.p.smith'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 36816
    keywords = ['patch']
    message_count = 15.0
    messages = ['341579', '341590', '341843', '341865', '341866', '341908', '341931', '341946', '341950', '342988', '342997', '343020', '343027', '347841', '351382']
    nosy_count = 7.0
    nosy_names = ['gregory.p.smith', 'larry', 'ned.deily', 'Michael.Felt', 'yan12125', 'EWDurbin', 'miss-islington']
    pr_nums = ['13124', '13192', '13197', '13198', '13199', '13200']
    priority = 'normal'
    resolution = 'remind'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue36816'
    versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8']

    @gpshead
    Copy link
    Member Author

    gpshead commented May 6, 2019

    test_httplib uses self-signed.pythontest.net in it's test_networked_good_cert test.

    On modern Linux distros (current Debian testing sid), the certificate it currently uses is rightfully rejected as being too weak:

    ERROR: test_networked_good_cert (test.test_httplib.HTTPSTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/home/greg/oss/cpython/Lib/test/test_httplib.py", line 1628, in test_networked_good_cert
        h.request('GET', '/')
      File "/home/greg/oss/cpython/Lib/http/client.py", line 1221, in request
        self._send_request(method, url, body, headers, encode_chunked)
      File "/home/greg/oss/cpython/Lib/http/client.py", line 1267, in _send_request
        self.endheaders(body, encode_chunked=encode_chunked)
      File "/home/greg/oss/cpython/Lib/http/client.py", line 1216, in endheaders
        self._send_output(message_body, encode_chunked=encode_chunked)
      File "/home/greg/oss/cpython/Lib/http/client.py", line 1004, in _send_output
        self.send(msg)
      File "/home/greg/oss/cpython/Lib/http/client.py", line 944, in send
        self.connect()
      File "/home/greg/oss/cpython/Lib/http/client.py", line 1383, in connect
        self.sock = self._context.wrap_socket(self.sock,
      File "/home/greg/oss/cpython/Lib/ssl.py", line 405, in wrap_socket
        return self.sslsocket_class._create(
      File "/home/greg/oss/cpython/Lib/ssl.py", line 853, in _create
        self.do_handshake()
      File "/home/greg/oss/cpython/Lib/ssl.py", line 1117, in do_handshake
        self._sslobj.do_handshake()
    ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1055)

    The TLS certificate on the server needs to be updated to something modern. I _believe_ this can be done by someone with infrastructure access via an update to https://github.com/python/pythontestdotnet/tree/master/tls

    Assigning to EWDurbin for triage and redirection to someone else infrastructury if he's not the right person.

    How to know if it has been fixed? Monitor the test_networked_good_cert test on any "Debian buster" builtbot(s) such as https://buildbot.python.org/all/#/workers/23 to make sure it is not skipped. (the test _currently_ fails, I am going to have it be _skipped_ on this specific key too small error for the time being to get that stable buildbot green again)

    @gpshead gpshead added 3.7 (EOL) end of life 3.8 only security fixes tests Tests in the Lib/test dir topic-SSL type-bug An unexpected behavior, bug, or error labels May 6, 2019
    @gpshead
    Copy link
    Member Author

    gpshead commented May 6, 2019

    EWDurbin says I can just open a PR with new certs in the repo and it'll go from there. :)

    @gpshead gpshead assigned gpshead and unassigned ewdurbin May 6, 2019
    @gpshead
    Copy link
    Member Author

    gpshead commented May 8, 2019

    Updated cert+key committed to pythontestdotnet. reassigning to EWDurbin to see that they're deployed.

    python/pythontestdotnet@2d12141

    @gpshead gpshead assigned ewdurbin and unassigned gpshead May 8, 2019
    @ewdurbin
    Copy link
    Member

    ewdurbin commented May 8, 2019

    Cert updated, reassigning back to gregory.p.smith to verify and close this out.

    @ewdurbin ewdurbin assigned gpshead and unassigned ewdurbin May 8, 2019
    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented May 8, 2019

    Lib/test/selfsigned_pythontestdotnet.pem in the cpython repository needs to be updated to match https://github.com/python/pythontestdotnet/blob/master/tls/self-signed-cert.pem, or the test fails :)

    ======================================================================
    ERROR: test_networked_good_cert (test.test_httplib.HTTPSTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/home/yen/tmp/cpython/Lib/test/test_httplib.py", line 1632, in test_networked_good_cert
        h.request('GET', '/')
      File "/home/yen/tmp/cpython/Lib/http/client.py", line 1221, in request
        self._send_request(method, url, body, headers, encode_chunked)
      File "/home/yen/tmp/cpython/Lib/http/client.py", line 1267, in _send_request
        self.endheaders(body, encode_chunked=encode_chunked)
      File "/home/yen/tmp/cpython/Lib/http/client.py", line 1216, in endheaders
        self._send_output(message_body, encode_chunked=encode_chunked)
      File "/home/yen/tmp/cpython/Lib/http/client.py", line 1004, in _send_output
        self.send(msg)
      File "/home/yen/tmp/cpython/Lib/http/client.py", line 944, in send
        self.connect()
      File "/home/yen/tmp/cpython/Lib/http/client.py", line 1383, in connect
        self.sock = self._context.wrap_socket(self.sock,
      File "/home/yen/tmp/cpython/Lib/ssl.py", line 405, in wrap_socket
        return self.sslsocket_class._create(
      File "/home/yen/tmp/cpython/Lib/ssl.py", line 853, in _create
        self.do_handshake()
      File "/home/yen/tmp/cpython/Lib/ssl.py", line 1117, in do_handshake
        self._sslobj.do_handshake()
    ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1055)

    @miss-islington
    Copy link
    Contributor

    New changeset 6bd8173 by Miss Islington (bot) (Gregory P. Smith) in branch 'master':
    bpo-36816: Update the self-signed.pythontest.net cert (GH-13192)
    6bd8173

    @ned-deily
    Copy link
    Member

    New changeset 2b9d7ab by Ned Deily (Gregory P. Smith) in branch '3.6':
    [3.6] bpo-36816: Update the self-signed.pythontest.net cert (GH-13192) (GH-13198)
    2b9d7ab

    @miss-islington
    Copy link
    Contributor

    New changeset 6daaf3f by Miss Islington (bot) (Gregory P. Smith) in branch '3.7':
    [3.7] bpo-36816: Update the self-signed.pythontest.net cert (GH-13192) (GH-13197)
    6daaf3f

    @gpshead
    Copy link
    Member Author

    gpshead commented May 9, 2019

    New changeset 7b5dca8 by Gregory P. Smith in branch '2.7':
    [2.7] bpo-36816: Update the self-signed.pythontest.net cert (GH-13192) (GH-13199)
    7b5dca8

    @aixtools
    Copy link
    Contributor

    I am not an OpenSSL expert - and I am conscious of OpenSSL changes with regard to 'acceptance' of anything self-signed.

    And, what it looks like you are trying to do with an updated 'signing" .pem is to remove the 'self-signed' charasteric.

    On AIX - atm - I get, as did Chih-Hsuan Yen (yan12125),

    ======================================================================
    ERROR: test_networked_good_cert (test.test_httplib.HTTPSTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/home/buildbot/python-master/Lib/test/test_httplib.py", line 1632, in test_networked_good_cert
        h.request('GET', '/')
      File "/home/buildbot/python-master/Lib/http/client.py", line 1221, in request
        self._send_request(method, url, body, headers, encode_chunked)
      File "/home/buildbot/python-master/Lib/http/client.py", line 1267, in _send_request
        self.endheaders(body, encode_chunked=encode_chunked)
      File "/home/buildbot/python-master/Lib/http/client.py", line 1216, in endheaders
        self._send_output(message_body, encode_chunked=encode_chunked)
      File "/home/buildbot/python-master/Lib/http/client.py", line 1004, in _send_output
        self.send(msg)
      File "/home/buildbot/python-master/Lib/http/client.py", line 944, in send
        self.connect()
      File "/home/buildbot/python-master/Lib/http/client.py", line 1383, in connect
        self.sock = self._context.wrap_socket(self.sock,
      File "/home/buildbot/python-master/Lib/ssl.py", line 405, in wrap_socket
        return self.sslsocket_class._create(
      File "/home/buildbot/python-master/Lib/ssl.py", line 853, in _create
        self.do_handshake()
      File "/home/buildbot/python-master/Lib/ssl.py", line 1117, in do_handshake
        self._sslobj.do_handshake()
    ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1055)

    And I see why now:
    test_networked_good_cert (test.test_httplib.HTTPSTest) ... skipped "Use of the 'network' resource not enabled"

    Digging a bit:

    buildbot@x064:[/home/buildbot/python-master]openssl s_client -connect self-signed.pythontest.net:443
    CONNECTED(00000003)
    depth=0 C = XY, ST = Castle Anthrax, L = Argument Clinic, O = Python Software Foundation, CN = self-signed.pythontest.net
    verify error:num=18:self signed certificate
    verify return:1
    depth=0 C = XY, ST = Castle Anthrax, L = Argument Clinic, O = Python Software Foundation, CN = self-signed.pythontest.net
    verify return:1
    ---
    Certificate chain
    0 s:/C=XY/ST=Castle Anthrax/L=Argument Clinic/O=Python Software Foundation/CN=self-signed.pythontest.net

    i:/C=XY/ST=Castle Anthrax/L=Argument Clinic/O=Python Software Foundation/CN=self-signed.pythontest.net

    And while this:
    How to know if it has been fixed? Monitor the test_networked_good_cert test on any "Debian buster" builtbot(s) such as https://buildbot.python.org/all/#/workers/23 to make sure it is not skipped. (the test _currently_ fails, I am going to have it be _skipped_ on this specific key too small error for the time being to get that stable buildbot green again)

    is nice for some, it is not nice for all!

    Perhaps the test should be switched to 'warn' on failure, rather than error on failure, until fixed!

    @aixtools
    Copy link
    Contributor

    p.s. On Centos I could not even get a python3 (at least not easily).

    On debian (on POWER) I get the same error (message) as on AIX - although the line number did change.

    ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1056)

    so, not a message about "key too small error" - pure, this is self-signed, so error.

    @aixtools
    Copy link
    Contributor

    On 21/05/2019 12:08, Michael Felt wrote:

    Michael Felt <aixtools@felt.demon.nl> added the comment:

    p.s. On Centos I could not even get a python3 (at least not easily).

    On debian (on POWER) I get the same error (message) as on AIX - although the line number did change.

    ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1056)

    so, not a message about "key too small error" - pure, this is self-signed, so error.

    ----------
    p.s. blush: seems I was testing against the wrong fork - seems to be
    cleared in 'master'. My apologies for the noise.


    Python tracker <report@bugs.python.org>
    <https://bugs.python.org/issue36816\>


    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented May 21, 2019

    Hi Michael Felt,

    And, what it looks like you are trying to do with an updated 'signing" .pem is to remove the 'self-signed' charasteric.

    If I understand it correctly, the new certificate is indeed still self-signed. It's updated to match the certificate deployed at https://self-signed.pythontest.net/. Under the hood load_verify_locations() at line 1628 is used to make the test accept any valid certificate signed with the given certificate.

    As a record, with CPython e7cb23b and OpenSSL 1.1.1b on Arch Linux x86_64, the test is green:

    test_networked_good_cert (test.test_httplib.HTTPSTest) ... ok

    By the way, I believe the "key too weak" workaround can be removed now and then this issue can be closed.

    @larryhastings
    Copy link
    Contributor

    New changeset 221178a by larryhastings (Gregory P. Smith) in branch '3.5':
    [3.5] bpo-36816: Update the self-signed.pythontest.net cert (GH-13192) (bpo-13200)
    221178a

    @gpshead
    Copy link
    Member Author

    gpshead commented Sep 9, 2019

    I believe this has been addressed.

    @gpshead gpshead closed this as completed Sep 9, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes tests Tests in the Lib/test dir topic-SSL type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants