Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LibreSSL is lacking host and IP verify parameters #381

Closed
tiran opened this issue Jan 18, 2018 · 5 comments

Comments

@tiran
Copy link

commented Jan 18, 2018

LibreSSL is not OpenSSL 1.0.2 compatible. It's lacking X509 verify parameters to verify hostname or IP address during TLS handshake. Python 3.7 will require a fully OpenSSL 1.0.2 compatible API and the X509 verify parameter functions to work.

  • X509_VERIFY_PARAM_set1_host
  • X509_VERIFY_PARAM_set1_ip
  • X509_VERIFY_PARAM_set1_ip_asc
  • X509_VERIFY_PARAM_set_hostflags

https://www.openssl.org/docs/man1.0.2/crypto/X509_VERIFY_PARAM_set1_host.html

PS: You may want to start introducing OpenSSL 1.1.0 compatibility, too. Either Python 3.8 or 3.9 will require a fully OpenSSL 1.1 compatible API.

@4a6f656c

This comment has been minimized.

Copy link
Member

commented Jan 18, 2018

Thanks for the heads up - LibreSSL was forked from OpenSSL 1.0.1 and we do not guarantee that we will fully implement later OpenSSL APIs. That said, the OpenSSL 1.0.2 functions that you mention have come up before and we'll take another look at those.

Re OpenSSL 1.1.x and Python 3.8/3.9, are you suggesting that Python is planning on dropping support for OpenSSL 1.0.x even though there is still no LTS release of OpenSSL 1.1 (the current release is only supported until 2018-08-31)? Presumably this is also likely to heading towards December 2019 (3.8) or June 2021 (3.9)?

@tiran

This comment has been minimized.

Copy link
Author

commented Jan 18, 2018

CPython development has very little resources to work on ssl. Our current approach to verify hostnames and IP addresses has multiple issues. X509_VERIFY_PARAM_set1_host and related functions solve these issues with very little effort. I'm currently discussion whether it's more important to keep compatibility with LibreSSL or to make Python's TLS support more secure and standard conform.

OpenSSL 1.0.2 reaches EOL about the same time 3.9.0 is scheduled to release. We haven't decided a release date yet, the final will probably be released between December 2019 and February 2020. OpenSSL 1.0.2 supports ends 2019-12-31. Unless there are very compelling reasons, it doesn't make sense to hold back features and increase our maintenance burden.

I'm willing to keep the basic features of Python's ssl module compatible with LibreSSL -- if and only if LibreSSL is kept fully compatible with required OpenSSL APIs. Hostname verification is such a feature. Some advanced features like env var are already documented as not supported when Python is compiled with LibreSSL.

Could you please clarify your backwards compatibility statement on https://www.libressl.org/ ? It says

libssl: a TLS library, backwards-compatible with OpenSSL

but that's no longer true. LibreSSL's libssl is no longer compatible with any supported OpenSSL version.

@tiran

This comment has been minimized.

Copy link
Author

commented Jan 20, 2018

After some discussion on the Python core developer list, I have decided to require OpenSSL 1.0.2 APIs. Every core developer in the discussion was in favor of using the new APIs. Python 3.7.0 beta1 will not by compatible with OpenSSL <= 1.0.1 or LibreSSL <= 2.6.4. Incompatibility with LibreSSL is only temporarily. I have implemented additional autoconf rules to test for required features and I'll keep all LibreSSL quirks. Python 3.7 will regain compatibility with LibreSSL as soon as you release a version which implements all required features.

OpenSSL recommends the API

The X509_VERIFY_PARAM_set1_host is OpenSSL's preferred way to validate host names. OpenSSL strongly recommends the API. From its man page

X509_VERIFY_PARAM_set1_host() sets the expected DNS hostname to name clearing any previously specified host name or names. If name is NULL, or empty the list of hostnames is cleared, and name checks are not performed on the peer certificate. If name is NUL-terminated, namelen may be zero, otherwise namelen must be set to the length of name. When a hostname is specified, certificate verification automatically invokes X509_check_host with flags equal to the flags argument given to X509_VERIFY_PARAM_set_hostflags() (default zero). Applications are strongly advised to use this interface in preference to explicitly calling X509_check_host, hostname checks are out of scope with the DANE-EE(3) certificate usage, and the internal check will be suppressed as appropriate when DANE support is added to OpenSSL.

X509_VERIFY_PARAM_set1_host resolves several Python issues

Python had multiple security issues just caused by incorrect or
buggy hostname matching. X509_VERIFY_PARAM_set1_host simplifies our code and resolves all outstanding issues.

  • Until Python 3.2 and 2.7.9, the ssl module was not capable of
    performing host name matching. ssl.match_hostname() was
    introduced in 3.2.0 and later back-ported to 2.7.9.
  • Issue #12000: Subject CN was ignored when a subject alternative
    name extension (SAN) was present without dNSName entries, thus
    violating RFC 2818.
  • CVE-2013-2099: Multiple wildcard characters could be abused
    for Denial-of-Service attack in the re module.
  • Issue #17997: RFC 2818 was superseded by RFC 6125, which no longer
    allows multiple wildcard characters. Wildcards are only supported
    in the left-most label.
  • Issue #17997: ssl.match_hostname() did not implement partial
    wildcards of international domain names correctly.
  • Issue #18709: The ssl module used an inappropriate OpenSSL function
    to convert host names from ASN.1 to strings. A host name with an
    embedded NULL byte could be abused to trick validation.
  • Issue #17305: The ssl module does not handle IDNA 2008-encoded
    host names correctly. It converts from IDN A-label (ASCII
    compatible encoding) to IDN U-label (unicode) with Python's idna
    encoding, which is IDNA 2003-only.
  • Issue #30141: The host name is not verified when a SSLSocket is
    created with do_handshake_on_connect=False and the application
    causes an implicit handshake w/o calling do_handshake() explicitly.
  • A SSLSocket performs host name matching after the handshake and
    during the handshake. In case of an invalid host name, a client
    is suppose to abort the connection with appropriate TLS alert.
    This causes two problem. For one the server is not informed about
    a problem with the certificate. Also an invalid host name does not
    prevent the client from sending a TLS client authentication
    cert to a malicious server. The cert typically contains personal
    information like username and department.
  • The server is not informed that the client has rejected the certificate.
    Standard-conform clients send a TLS ALERT with alert type BAD CERTIFICATE.
@yan12125

This comment has been minimized.

Copy link

commented Jan 20, 2018

@4a6f656c:

Looks like server name verification in LibreSSL is handled by libtls and servername is set via tls_connect_servername? It might be better to move SAN and CN handling (tls_check_name) from libtls to libssl so that users (e.g. CPython) can utilize it via existing SSL contexts instead of another tls context.

@4a6f656c

This comment has been minimized.

Copy link
Member

commented Feb 10, 2018

@yan12125 libtls is a completely different beast - it has hostname verification enabled and required by default. You have to go out of your way to disable it, if you really want to do that. In comparison, as far as I know even with OpenSSL 1.1 you still have to explicitly request and enable hostname verification.

@tiran for a long time we've provided the X509_check_* functionality - it is just the X509_VERIFY_PARAM_* functions that were not currently available. While OpenSSL "strongly advise" to use this interface, there is no reason that code cannot implement the same functionality via the existing API (and I fully understand Python wanting to remove its own verification code) (and there appears to be about twice an many pieces of software using the X509_check_* interface than the X509_VERIFY_PARAM_* one).

I have just exposed the requested X509_VERIFY_PARAM_* functions, along with some others that are in the OpenSSL 1.0.2 API. If there is further functionality that you need, please let us know.

@4a6f656c 4a6f656c closed this Feb 10, 2018

BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jun 18, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
BoboTiG added a commit to nuxeo/nuxeo-drive that referenced this issue Jul 1, 2019
NXDRIVE-1280: Add GNU/Linux support 🍾!
For now, we are stuck with Python 3.6.8 because of an incomptibility with SSL:

    Could not build the ssl module!
     Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl
     with X509_VERIFY_PARAM_set1_host().
     LibreSSL 2.6.4 and earlier do not provide the necessary APIs,
     libressl-portable/portable#381

* Compile Python with --enabled-shared on GNU/Linux too;
* Display the SKIP envar only if set.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.