Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Better Default Ciphers for the SSL Module #65194

Closed
dstufft opened this issue Mar 20, 2014 · 69 comments
Closed

Use Better Default Ciphers for the SSL Module #65194

dstufft opened this issue Mar 20, 2014 · 69 comments

Comments

@dstufft
Copy link
Member

@dstufft dstufft commented Mar 20, 2014

BPO 20995
Nosy @malemburg, @jcea, @ncoghlan, @pitrou, @vstinner, @tiran, @benjaminp, @ezio-melotti, @alex, @bitdancer, @dstufft
Files
  • better-ciphers.diff
  • better-ciphers-better-priority.diff
  • better-ciphers-dss.diff
  • better-ciphers-with-docs.diff
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2014-03-22.17:27:59.923>
    created_at = <Date 2014-03-20.14:11:16.718>
    labels = []
    title = 'Use Better Default Ciphers for the SSL Module'
    updated_at = <Date 2014-04-23.14:28:37.006>
    user = 'https://github.com/dstufft'

    bugs.python.org fields:

    activity = <Date 2014-04-23.14:28:37.006>
    actor = 'alex'
    assignee = 'none'
    closed = True
    closed_date = <Date 2014-03-22.17:27:59.923>
    closer = 'dstufft'
    components = []
    creation = <Date 2014-03-20.14:11:16.718>
    creator = 'dstufft'
    dependencies = []
    files = ['34539', '34546', '34548', '34558']
    hgrepos = []
    issue_num = 20995
    keywords = ['patch']
    message_count = 69.0
    messages = ['214239', '214240', '214243', '214244', '214249', '214251', '214253', '214271', '214277', '214278', '214286', '214290', '214291', '214292', '214293', '214294', '214295', '214296', '214297', '214298', '214300', '214301', '214303', '214304', '214305', '214306', '214307', '214309', '214310', '214315', '214347', '214348', '214351', '214353', '214356', '214357', '214358', '214359', '214360', '214361', '214362', '214363', '214387', '214393', '214394', '214396', '214401', '214402', '214403', '214404', '214414', '214416', '214417', '214418', '214420', '214422', '214423', '214425', '214428', '214429', '214488', '214495', '214496', '217066', '217067', '217068', '217069', '217070', '217071']
    nosy_count = 14.0
    nosy_names = ['lemburg', 'jcea', 'ncoghlan', 'pitrou', 'vstinner', 'christian.heimes', 'benjamin.peterson', 'ezio.melotti', 'Arfrever', 'alex', 'r.david.murray', 'python-dev', 'dstufft', 'markk']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue20995'
    versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4', 'Python 3.5']

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    As of right now the default cipher list for the ssl module is DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2, additionally on Python 3.4 when you use create_default_context() then you also additionally get HIGH:!aNULL:!RC4:!DSS.

    I think we should change this to the cipher string:

    ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS

    This will:

    • Prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE)
    • prefer ECDHE over DHE for better performance
    • prefer any AES-GCM over any AES-CBC for better performance and security
    • use 3DES as fallback which is secure but slow
    • disable NULL authentication, MD5 MACs and DSS for security reasons

    This cipher string is taken from urllib3 where it was compiled through the resources of:

    The compatibility of this is pretty good. The only time this should cause a connection to *fail* is if a server is using an insecure cipher and in that case you can re-enable it by simply passing the original cipher list through the ssl.wrap_socket ciphers function.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 20, 2014

    I really don't think hardcoding specific ciphers is a good idea.

    @malemburg
    Copy link
    Member

    @malemburg malemburg commented Mar 20, 2014

    On 20.03.2014 15:11, Donald Stufft wrote:

    The compatibility of this is pretty good. The only time this should cause a connection to *fail* is if a server is using an insecure cipher and in that case you can re-enable it by simply passing the original cipher list through the ssl.wrap_socket ciphers function.

    Depends on who "you" is :-) Most of the time this will be the user of
    some script or application with no clue as to how to change this or
    what a cipher string is.

    I think we should leave this decision to the OpenSSL lib vendors
    and developers.

    @bitdancer
    Copy link
    Member

    @bitdancer bitdancer commented Mar 20, 2014

    create_default_context is about best practices, though, so it seems to me it wouldn't be crazy to do it there.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 20, 2014

    create_default_context is about best practices, though, so it seems to
    me it wouldn't be crazy to do it there.

    Agreed, but the real problem here is maintenance. Hardcoding a list of
    specific ciphers means someone must follow closely the introduction of
    new ciphers in OpenSSL, and choose whether or not to include them in the
    list.

    I'd prefer an open-ended cipher string. Here is a proposal:
    'ECDH:EDH:AESGCM:HIGH:!eNULL:!aNULL:!DSS'

    It prioritizes Diffie-Hellman key exchange (for perfect forward
    secrecy), and AESGCM for the symmetric cipher; it also lets OpenSSL
    append other possible ciphers.

    BTW, apparently removing RC4 prevents ECDHE in SSv23 mode:

    $ ./python -c 'import ssl, socket; ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23); ctx.set_ciphers("EECDH:EDH:AESGCM:HIGH:!eNULL:!aNULL");  s = ctx.wrap_socket(socket.socket()); s.connect(("linuxfr.org", 443)); print(s.cipher()); s.close()'
    ('ECDHE-RSA-RC4-SHA', 'TLSv1/SSLv3', 128)
    
    $  ./python -c 'import ssl, socket; ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23); ctx.set_ciphers("EECDH:EDH:AESGCM:HIGH:!eNULL:!aNULL:!RC4");  s = ctx.wrap_socket(socket.socket()); s.connect(("linuxfr.org", 443)); print(s.cipher()); s.close()'
    ('DHE-RSA-AES256-SHA', 'TLSv1/SSLv3', 256)

    @alex
    Copy link
    Member

    @alex alex commented Mar 20, 2014

    That's because of the set of ciphersuites offered by the server (see https://www.ssllabs.com/ssltest/analyze.html?d=linuxfr.org), it's not an inevitable property of TLS. For example jenkins.cryptography.io (see https://www.ssllabs.com/ssltest/analyze.html?d=jenkins.cryptography.io) offers ECDHE suites without any RC4 at all.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    Yea I noticed that, so I was doing some more testing, here's what I think we should be using (It Adds back in RC4):

    ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:ECDH+RC4:DH+RC4:RSA+RC4!aNULL:!MD5:!DSS

    This gives us everything that DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2 does except for the ciphers list here https://gist.github.com/dstufft/251dbeb8962e2182e668 on my OpenSSL 1.0.1f install.

    Antoine, your cipher string priortizes ECDHE RC4 over DHE AES or even just plain AES. The string I'm proposing has been carefully crafted in order to get the ciphers in a very particular order. That order is basically - 1) Security of the cipher itself 2) PFS 3) Performance while also maintaining compatibility both forwards and backwards.

    RC4 is in a precarious condition and it's use should be heavily discouraged. It is still required in some cases which is why my revised default cipher suggestion includes it, but at the end as a last fall back. At that point if RC4 gets selected it's the servers fault and the client did everything it could except refuse.

    I still do believe that this should be the default ciphers while my original string should be the "restricted" ciphers that create_default_context() uses.

    @ncoghlan
    Copy link
    Contributor

    @ncoghlan ncoghlan commented Mar 20, 2014

    In terms of following closely, I'd be willing to encourage Red Hat's SRT to
    keep an eye on this.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    Another bit of maintenance here:

    If a new cipher suite is added to OpenSSL it won' be generally available for a long while so very few if any services are going to be willing to depend on *only* it. For the very rare and unlikely case that somebody does setup a service that requires some brand new cipher they can override this list easily.

    Using the default or the "wide" open strings are inherently more dangerous because of the wide range of OpenSSL's that are in production use. It's hard without auditing every version of OpenSSL to figure out what ciphers will be available in what circumstances. It also means that if OpenSSL adds a new cipher that ends up being insecure that it will be picked up automatically. Therefore the strings I've posted take the opinion that a whitelist is more secure than a blacklist and whitelist the cipher suites to a very specific set that happen to be best practices at this current time.

    The only *required* maintenance would be if one of the selected ciphers are found to be insecure. However that was already a required maintenance because (again) of the wide range of OpenSSL versions available and the fact that these strings don't *add* any new ciphers, only remove some and create an explicit priority.

    @alex
    Copy link
    Member

    @alex alex commented Mar 20, 2014

    It's also worth noting that users appear to be FAR more likely to have an up to date Python than they are an up to date OpenSSL, meaning that if a change needs to be made, we're much better situated to get that disseminated to actual users than OpenSSL is

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 20, 2014

    The string I'm proposing has been carefully crafted in order to get
    the ciphers in a very particular order. That order is basically - 1)
    Security of the cipher itself 2) PFS 3) Performance while also
    maintaining compatibility both forwards and backwards.

    I still think the ciphers list should be open-ended, i.e. have "HIGH" somewhere at the end.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    Why? At best users will get yet another secure algorithm and at worst they'll get an insecure algorithm.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 20, 2014

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 20, 2014

    By the way:

    Using the default or the "wide" open strings are inherently more
    dangerous because of the wide range of OpenSSL's that are in
    production use. It's hard without auditing every version of OpenSSL to
    figure out what ciphers will be available in what circumstances

    This doesn't parse. If the system OpenSSL isn't maintained properly, it's not Python's job to workaround that. And we certainly don't have the required knowledge and dedication anyway.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    This doesn't parse. If the system OpenSSL isn't maintained properly, it's not Python's job to workaround that. And we certainly don't have the required knowledge and dedication anyway.

    Please let's not have a repeat of https://bugs.ruby-lang.org/issues/9424, Python is in a better place to workaround that than anyone else.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 20, 2014

    Please let's not have a repeat of
    https://bugs.ruby-lang.org/issues/9424, Python is in a better place to
    workaround that than anyone else.

    Please stop the FUD. I proposed an alternative, how is it insecure
    according according to you?

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    Oh and don't confuse me that I think Python's current situation is as bad as Ruby's was, but that attitude is dangerous and wrong :/

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    I'm still looking into what "HIGH" entails across all the various OpenSSLs that are in production that I can access. That "FUD" was responding to the attitude that it's not Python's job to do this. Python is exposing a security sensitive API, it is it's job.

    @malemburg
    Copy link
    Member

    @malemburg malemburg commented Mar 20, 2014

    On 20.03.2014 23:36, Donald Stufft wrote:

    Donald Stufft added the comment:

    I'm still looking into what "HIGH" entails across all the various OpenSSLs that are in production that I can access. That "FUD" was responding to the attitude that it's not Python's job to do this. Python is exposing a security sensitive API, it is it's job.

    I disagree. Python only provides an interface to OpenSSL, so the OpenSSL
    system defaults should be used.

    Maintaining system security is an easier and more scalable approach than
    trying to properly configure half a dozen sub-systems which happen to use
    OpenSSL as basis for their SSL configuration. By forcing a specific
    set of ciphers, we're breaking this approach.

    By restricting the set of allowed ciphers you can also create the
    situation that Python in its default configuration cannot talk to
    certain web servers which use a different set of ciphers than the
    one you are proposing.

    We shouldn't do this in Python for the same reason we're not including
    a predefined set of CA root certificates with the distribution.

    @malemburg
    Copy link
    Member

    @malemburg malemburg commented Mar 20, 2014

    On 20.03.2014 21:52, Alex Gaynor wrote:

    It's also worth noting that users appear to be FAR more likely to have an up to date Python than they are an up to date OpenSSL, meaning that if a change needs to be made, we're much better situated to get that disseminated to actual users than OpenSSL is

    This depends a lot on the type of users you're looking at. Corporate
    users won't upgrade their Python version easily. They will happily
    install patched OpenSSL versions.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    I disagree. Python only provides an interface to OpenSSL, so the OpenSSL
    system defaults should be used.

    Python is already changing the OpenSSL defaults, also you're advocating that
    Python should support 40bit encryption that can be brute forced in a matter of
    days.

    Maintaining system security is an easier and more scalable approach than
    trying to properly configure half a dozen sub-systems which happen to use
    OpenSSL as basis for their SSL configuration. By forcing a specific
    set of ciphers, we're breaking this approach.

    Again, Python is already forcing a set of ciphers. I don't know what sort of
    Systems you use, but even RHEL 6.5 has *horrible* ciphers by in the OpenSSL
    default set. Things like DES (not 3DES, DES) and 40bit RC4.

    By restricting the set of allowed ciphers you can also create the
    situation that Python in its default configuration cannot talk to
    certain web servers which use a different set of ciphers than the
    one you are proposing.

    Of course, any restriction does that, that's not reason to also allow aNULL
    or eNULL by default just because somewhere someone out there might be running
    a server that only speaks them. Secure, Sane Defaults and the Ability to
    override.

    We shouldn't do this in Python for the same reason we're not including
    a predefined set of CA root certificates with the distribution.

    The difference here is that there are properly maintained alternatives to
    Python including a predefined set of CA root certificates. This isn't the
    case with OpenSSL. OpenSSL doesn't provide good defaults and I'm not aware of
    a single OS which ships with OpenSSL that patches it to provide good defaults.

    Python exposes this API, it's Python's job to properly secure it.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 20, 2014

    Again, Python is already forcing a set of ciphers. I don't know what sort of
    Systems you use, but even RHEL 6.5 has *horrible* ciphers by in the OpenSSL
    default set. Things like DES (not 3DES, DES) and 40bit RC4.

    I wonder why RedHat doesn't bother changing the defaults.
    Did nobody ever report the issue to them, or are they more conservative
    than we are?

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    > Again, Python is already forcing a set of ciphers. I don't know what sort of
    > Systems you use, but even RHEL 6.5 has *horrible* ciphers by in the OpenSSL
    > default set. Things like DES (not 3DES, DES) and 40bit RC4.

    I wonder why RedHat doesn't bother changing the defaults.
    Did nobody ever report the issue to them, or are they more conservative
    than we are?

    I don't know why. Probably because the OpenSSL defaults are not intended to
    be secure so OpenSSL is working as intended. The users of OpenSSL are intended
    to use the cipher selection string to secure themselves.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    Ok Antoine I've looked around.

    Using a string like this:

    ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:ECDH+RC4:DH+RC4:RSA+RC4:ECDH+HIGH:DH+HIGH:RSA+HIGH:!aNULL:!eNULL:!MD5:!DSS

    The only *additional* ciphers that get added from the use of HIGH are various Camellia ciphers. These ciphers are not known to be insecure at this point in time so as of right now this is not an insecure cipher string.

    However I still content that using HIGH in the cipherstring actually adds additional maintenance burden. In order to know if that cipherstring is still safe you must run it against every target OpenSSL you want to make secure to ensure that it doesn't allow a new cipher that doesn't meet the security strength that was attempted to be had with that cipherstring. If you use an explicit cipher string then you know exactly which cipher suites Python will use no matter what the OpenSSL claims is HIGH or not. This means that instead of having to monitor all the various OpenSSL versions for new ciphers you only have to periodically check that the suites that Python selected are still secure.

    Remember the "failure" mode for not having a cipher in the list is that a different cipher is selected unless there are no other ciphers. A New cipher being added to OpenSSL is not going to be the only cipher available in any meaningful timeframe. The "failure" mode for having a bad cipher in the list is possibly making the users of Python insecure. That's why an explicit approach is preferred over an open ended approach. Because you don't have to audit a moving target.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    Oh, additionally OpenSSL makes no promises what the meaning of "HIGH" will be in the future. So you can only look at what it means now and what it means in the past.

    OpenSSL is not a good library and it's unfortunate that they don't attempt to make people secure by default.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    Oh, Additionally Marc:

    Even if some system administrator or some system out there does patch their OpenSSL to actually be safe by default Python changing it's cipher string only adds to the potential security (or at worst does nothing). If even one system (of which there are legion) does not do that patch then Python changing it's ciphers will protect that user.

    The failure mode for a bad cipher is silent insecurity, the failure mode for not having a needed cipher is an obvious error.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 20, 2014

    However I still content that using HIGH in the cipherstring actually
    adds additional maintenance burden. In order to know if that
    cipherstring is still safe you must run it against every target
    OpenSSL you want to make secure to ensure that it doesn't allow a new
    cipher that doesn't meet the security strength that was attempted to
    be had with that cipherstring.

    I think that is a bit reverse. The main configuration point for ciphers
    should be the server, not the client. We set a cipher string to guide
    cipher selection in case the server lets us choose amongst its supported
    ciphers, but that's all.

    Besides, the ssl module doesn't promise a specific "security strength".
    The defaults are a best effort thing, and paranoid people should
    probably override the cipher string (and deal with the consequences).

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 20, 2014

    > However I still content that using HIGH in the cipherstring actually
    > adds additional maintenance burden. In order to know if that
    > cipherstring is still safe you must run it against every target
    > OpenSSL you want to make secure to ensure that it doesn't allow a new
    > cipher that doesn't meet the security strength that was attempted to
    > be had with that cipherstring.

    I think that is a bit reverse. The main configuration point for ciphers
    should be the server, not the client. We set a cipher string to guide
    cipher selection in case the server lets us choose amongst its supported
    ciphers, but that's all.

    The Python ssl module is used for servers and clients. Ideally servers will
    have prefer server ciphers on, but that doesn't always happen and providing
    a modern level of security for end users is preferable.

    Besides, the ssl module doesn't promise a specific "security strength".
    The defaults are a best effort thing, and paranoid people should
    probably override the cipher string (and deal with the consequences).

    These are not things that affect only paranoid people and expecting someone
    to even know what OpenSSL is much less how to configure it and what they want
    to configure it to in order to get modern levels of security is backwards. The
    danger for breakage here is *tiny*, *miniscule*, almost non existent and the
    failure case is obvious and easy to fix.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 20, 2014

    The Python ssl module is used for servers and clients. Ideally servers will
    have prefer server ciphers on, but that doesn't always happen and providing
    a modern level of security for end users is preferable.

    We should have specific defaults for servers in
    create_default_context().

    The
    danger for breakage here is *tiny*, *miniscule*, almost non existent and the
    failure case is obvious and easy to fix.

    Again: the point is maintenance later, not breakage now.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 21, 2014

    Again: the point is maintenance later, not breakage now.

    Ok, well I don't agree that it's more maintenance later to be explicit and not include HIGH, but whatever it's not insecure at the moment so.

    Attached is a patch against 3.5 for folks to review.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 21, 2014

    Yup :) Just being explicit in that!

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 21, 2014

    Ok, so I think the latest patch is mostly good but I don't understand why the "restricted ciphers" (again, misnomer) would ban RC4 (and DSS?). These are the ciphers used by higher-level client libs, and connection failures will confuse the hell out of people.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 21, 2014

    Note: The RC4 and DSS exclusion existed previously on the restricted ciphers so we'd have to ask Christian why he did that. For me personally the restricted ciphers are intended to be best practice ciphers and that means no RC4. DSS here I'm kind of meh about the same way I was for the default ciphers. DSA has historically had problems with weak RNGs and as far as I'm aware no CA's actually issue DSS certificates. But I mostly left !DSS in the restricted set because Christian had it in originally.

    This might be a case where to really do "best practices" we need to diverge between client and server. For a server I definitely think putting RC4 in the cipher string is a bad thing. For clients it is not the greatest thing but it more closely matches what browsers do because there are a few services here and there which only expose RC4.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 21, 2014

    Forgot to add! If you think splitting between "restricted" server and client ciphers I can split them like that and upload a new patch.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 21, 2014

    Forgot to add! If you think splitting between "restricted" server and
    client ciphers I can split them like that and upload a new patch.

    I was about to open a separate issue for the server side. How about
    restricting this issue to client usage?

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 21, 2014

    Not sure what you mean by client issue. Do you mean to keep RC4?

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 21, 2014

    Which "client issue"? Sorry, I've lost track :-)

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 21, 2014

    Er, I typed issue and meant usage. Right now the only difference between restricted ciphers and the default ciphers is restricted ciphers have no RC4 and no DSS. You wanted this issue limited to client changes and I'm not sure how to do that without enabling RC4/DSS for servers (which is a regression in the security of the restricted ciphers).

    I think if we want to make restricted ciphers apply only for servers that's OK but as this ticket doesn't change the restrictions (other than omitting SRP/PSK and SEED/IDEA) that there's no changes to be made here, it should be accepted and then another ticket for restricting the restricted ciphers to servers only? Or what did you have in mind?

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 21, 2014

    Er, I typed issue and meant usage. Right now the only difference
    between restricted ciphers and the default ciphers is restricted
    ciphers have no RC4 and no DSS. You wanted this issue limited to
    client changes and I'm not sure how to do that without enabling
    RC4/DSS for servers (which is a regression in the security of the
    restricted ciphers).

    Hmm, fair enough, let's change them all at once here. Also, since
    "restricted ciphers" aren't actually used by stdlib modules, I changed
    my mind and think it's ok to disable RC4 and DSS :-)

    I'll still open another issue for server-specific configuration: not the
    ciphers, but other stuff.

    @ncoghlan
    Copy link
    Contributor

    @ncoghlan ncoghlan commented Mar 21, 2014

    Shall we commit the new string for 3.5 for the time being? I'm currently working on a PEP to help define a policy for dealing with network security related issues/enhancements in maintenance branches, so I don't think we should touch those until we have that discussion on python-dev and get an explicit direction from Guido.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 22, 2014

    (for the record and for the sake of comparison, Postfix's "high" security setting is "ALL:!EXPORT:!LOW:!MEDIUM:+RC4:@strength")

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 22, 2014

    The patch will also need updating the "Cipher selection" paragraph in ssl.rst, I think.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 22, 2014

    I can add that.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 22, 2014

    Hmm, I'm not sure what needs updated. The docs only say that ssl module disabled certain weak ciphers by default which is still the case. Was there some specific place or wording you were looking for?

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 22, 2014

    Well, the doc currently says:

    """Starting from Python 3.2.3, the
    ssl module disables certain weak ciphers by default, but you may want
    to further restrict the cipher choice. For example::

       context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
       context.set_ciphers('HIGH:!aNULL:!eNULL')"""

    But after your changes, calling set_ciphers('HIGH:!aNULL:!eNULL') will actually weaken security, so this example should simply be removed (IMHO).

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 22, 2014

    Ah yes, I skipped over that looking for a place where we were detailing what ciphers were picked. Ok Thanks!

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 22, 2014

    Added the docs changes

    @python-dev
    Copy link
    Mannequin

    @python-dev python-dev mannequin commented Mar 22, 2014

    New changeset e9749a7aa958 by Donald Stufft in branch '3.4':
    Issue bpo-20995: Enhance default ciphers used by the ssl module
    http://hg.python.org/cpython/rev/e9749a7aa958

    @python-dev
    Copy link
    Mannequin

    @python-dev python-dev mannequin commented Mar 22, 2014

    New changeset 60f696488c4f by Donald Stufft in branch 'default':
    Merge changes from 3.4 to bring in fixes for Issue bpo-20995
    http://hg.python.org/cpython/rev/60f696488c4f

    @python-dev
    Copy link
    Mannequin

    @python-dev python-dev mannequin commented Mar 22, 2014

    New changeset 869277faf3dc by Antoine Pitrou in branch '3.4':
    Issue bpo-21015: SSL contexts will now automatically select an elliptic curve for ECDH key exchange on OpenSSL 1.0.2 and later, and otherwise default to "prime256v1".
    http://hg.python.org/cpython/rev/869277faf3dc

    New changeset 3b81d1b3f9d1 by Antoine Pitrou in branch 'default':
    Issue bpo-21015: SSL contexts will now automatically select an elliptic curve for ECDH key exchange on OpenSSL 1.0.2 and later, and otherwise default to "prime256v1".
    http://hg.python.org/cpython/rev/3b81d1b3f9d1

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Mar 22, 2014

    The buildbot failures should have been fixed by bpo-21015, should we close this one?

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Mar 22, 2014

    Yes

    @dstufft dstufft closed this as completed Mar 22, 2014
    @markk
    Copy link
    Mannequin

    @markk markk mannequin commented Apr 23, 2014

    The cipher strings rely too much on AES for my taste. Imagine that ChaCha20Poly1305 or any other strong cipher suite is introduced to OpenSSL in the future.

    Enabling using general, and demoting using narrow terms, seems IMHO a better approach. For example:

    ECDH+HIGH:DH+HIGH:!aNULL:!MD5:!RC4:-3DES:HIGH
    

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Apr 23, 2014

    The cipher string includes HIGH, so if ChaCha20Poly1305 or another cipher suite is added to OpenSSL it'll get included in the cipher string by default.

    So the major difference of what you suggest would be no longer prioritizing ciphers. However I would argue that would be bad. The priority exists so that we get the best possible cipher is as many situations as we possibly can. It doesn't mean that we'll get the best possible cipher in *every* single situation, but generally we will.

    To this ends it prioritizes:
    * PFS with a secure cipher over everything else (Your string would do this as well)
    * After that prefer ECDHE over DHE
    * After that, prefer AES-GCM
    * After that, prefer AES-CBC
    * After that, any other HIGH cipher
    * After that, 3DES
    * After that, any use of RC4 including those with PFS

    So if OpenSSL added ChaCha20Poly1305 it would fit into the priority after AES-GCM and AES-CBC.

    For any device that has hardware support for AES (AES-NI) AES-GCM is hands down a better choice of cipher. It is secure, has no issues in the spec itself, and it is *fast*, like 900MB/s for AES-128-GCM on a Sandy Bridge Xeon w/ AES-NI (ChaCha20Poly1305 got 500MB/s on the same hardware, however it is a 256bit cipher will AES-128-GCM is a 128 bit cipher). Using ChaCha20 on those devices would be a worse choice than AES-GCM.

    However on lower powered devices, such as smart phones, especially those without hardware support for AES, ChaCha20 really shines. A Galaxy Nexus can do AES-256-GCM at 20MB/s whereas it can do ChaCha20Poly1305 at 92MB/s (same phone).

    So in an ideal world, assuming ChaCha20 was implemented in OpenSSL, we'd adjust the default cipher string based on the hardware they are running on. However since we don't have the ability to do that then preferring AES (which we know on some systems will be much faster) over an unknown future cipher (which we have no knowledge of if it will be faster or not) is a much more reasonable choice. If at some point in the future OpenSSL gains ChaCha20Poly1305 support then these strings should probably change to put ChaCha20Poly1305 in between AES-GCM and AES-CBC because on any given the system the likelyhood that you want AES-GCM is still higher than ChaCha20, but the likelyhood you want ChaCha20 over AES-CBC is greater.

    It's also important to note that the server in any TLS communication is the end that picks exactly which cipher we select. Ideally all servers will be configured to have the strongest cipher first, and to prefer their own cipher order. In that case for the *client* side of a TLS connection the order of the ciphers doesn't matter and thus your string vs the implemented string has no difference in behavior. However if the server doesn't enforce their own preference for ciphers, then the difference will be that an undefined cipher will be selected (could be AESGCM, AESCBC, ChaCha20, or Camellia). On the server side of this, if you're using Python to terminate your TLS on the server side, the likelyhood that a server is running on a low powered device where the benefits of ChaCha20Poly1305 are the highest are pretty low and preferring AES-GCM is an even safer idea.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Apr 23, 2014

    For any device that has hardware support for AES (AES-NI) AES-GCM is
    hands down a better choice of cipher. It is secure, has no issues in
    the spec itself, and it is *fast*, like 900MB/s for AES-128-GCM on a
    Sandy Bridge Xeon w/ AES-NI (ChaCha20Poly1305 got 500MB/s on the same
    hardware, however it is a 256bit cipher will AES-128-GCM is a 128 bit
    cipher). Using ChaCha20 on those devices would be a worse choice than
    AES-GCM.

    I think performance isn't really relevant, except perhaps on very busy
    servers. A smartphone acting as a *client* certainly shouldn't need to
    download 20 MB/s of encrypted data.

    @dstufft
    Copy link
    Member Author

    @dstufft dstufft commented Apr 23, 2014

    I think performance isn't really relevant, except perhaps on very busy
    servers. A smartphone acting as a *client* certainly shouldn't need to
    download 20 MB/s of encrypted data.

    Well, if you factor out performance then ChaCha20Poly1305 and AES-GCM are more
    or less equivalent in preference with AES-CBC still less than either of them
    because of problematic construction choices in the TLS spec. If you factor
    out performance completely there is maybe a slight preference for
    ChaCha20Poly1305 over AES-GCM simply because AES-GCM is hard to implement in
    a timing safe way in software. However that discussion is mostly academic as
    right now ChaCha20Poly1305 is not available in OpenSSL.

    In general I agree that the performance of all of these are "good enough" that
    the average user of this API won't be able to tell the difference, however
    there is no cost to selecting the generally more performant of the two so I
    think it still makes sense to consider it.

    Hopefully what I was trying to achieve was provide some more context for markk so he'd hopefully be able to better understand why the string cipher calls out AES specifically before falling back to HIGH.

    @markk
    Copy link
    Mannequin

    @markk markk mannequin commented Apr 23, 2014

    Thanks for the detailed insight, Donald! And I certainly love the progress these changes here bring. :-)

    Perhaps limiting the scope to ChaCha20Poly1305 (»CCP«) has been a wrong approach of mine to explain my concerns:

    We should not refer to any particular cipher in those lists, and by that avoid to revisit the defaults at any point in the future.

    1. Properties of any cipher to come are known to the makers of OpenSSL first.
    2. Python shouldn't duplicate the work of ordering ciphers, which is already done by OpenSSL.
    3. … especially because it is unknown which ciphers a user's OpenSSL does actually implement (Is EC present? CCP? HC-256 or HC-128? WIERZA? Rabbit? NTRU…) or will implement in the future.
    4. Whether a cipher is regarded as more secure than another depends on its implementation, too. The implementors are better judges of that, and hence ordering should done by them and could vary between versions [e.g., of OpenSSL].
    5. Given our experiences with Python 2.7 I'd like to argue that there is reluctance to upgrading existing installations and its cipher suite strings. ;-)

    But we know from experience with already established ciphers if and when to demote them.

    That said I don't insist on any changes.

    @alex
    Copy link
    Member

    @alex alex commented Apr 23, 2014

    It would be great if we could rely on OpenSSL's ordering. It would be seriously fantastic. OpenSSL is best positioned to be able to do the right things, it's updated at the right times. It should be where we do this.

    Unfortunately the OpenSSL maintainers have utterly abdicated any responsibility for helping secure users, and has gone with poor defaults, obligating us to fill the hole.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    None yet
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants