Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TLS: Provide ESNI support framework for curl and libcurl #4011

Closed
wants to merge 12 commits into from
Closed
139 changes: 139 additions & 0 deletions ESNI.md
@@ -0,0 +1,139 @@
# TLS: ESNI support in curl and libcurl

## Summary

**ESNI** means **Encrypted Server Name Indication**, a TLS 1.3
extension which is currently the subject of an
[IETF Draft][tlsesni].

This file is intended to show the latest current state of ESNI support
in **curl** and **libcurl**.

At end of August 2019, an [experimental fork of curl][niallorcurl],
built using an [experimental fork of OpenSSL][sftcdopenssl], which in
turn provided an implementation of ESNI, was demonstrated
interoperating with a server belonging to the [DEfO
Project][defoproj].

Further sections here describe

- resources needed for building and demonstrating **curl** support
for ESNI,

- progress to date,

- TODO items, and

- additional details of specific stages of the progress.

## Resources needed

To build and demonstrate ESNI support in **curl** and/or **libcurl**,
you will need

- a TLS library, supported by **libcurl**, which implements ESNI;

- an edition of **curl** and/or **libcurl** which supports the ESNI
implementation of the chosen TLS library;

- an environment for building and running **curl**, and at least
building **OpenSSL**;

- a server, supporting ESNI, against which to run a demonstration
and perhaps a specific target URL;

- some instructions.

The following set of resources is currently known to be available.

| Set | Component | Location | Remarks |
|:-----|:-------------|:------------------------------|:-------------------------------------------|
| DEfO | TLS library | [sftcd/openssl][sftcdopenssl] | Tag *esni-2019-08-30* avoids bleeding edge |
| | curl fork | [niallor/curl][niallorcurl] | Tag *esni-2019-08-30* likewise |
| | instructions | [ESNI-README][niallorreadme] | |

## Progress

### PR 4011 (Jun 2019) expected in curl release 7.67.0 (Oct 2019)

- Details [below](#pr4011);

- New **curl** feature: `CURL_VERSION_ESNI`;

- New configuration option: `--enable-esni`;

- Build-time check for availability of resources needed for ESNI
support;

- Pre-processor symbol `USE_ESNI` for conditional compilation of
ESNI support code, subject to configuration option and
availability of needed resources.

## TODO

- (next PR) Add libcurl options to set ESNI parameters.

- (next PR) Add curl tool command line options to set ESNI parameters.

- (WIP) Extend DoH functions so that published ESNI parameters can be
retrieved from DNS instead of being required as options.

- (WIP) Work with OpenSSL community to finalize ESNI API.

- Track OpenSSL ESNI API in libcurl

- Identify and implement any changes needed for CMake.

- Optimize build-time checking of available resources.

- Encourage ESNI support work on other TLS/SSL backends.

## Additional detail

### PR 4011

**TLS: Provide ESNI support framework for curl and libcurl**

The proposed change provides a framework to facilitate work to
implement ESNI support in curl and libcurl. It is not intended
either to provide ESNI functionality or to favour any particular
TLS-providing backend. Specifically, the change reserves a
feature bit for ESNI support (symbol `CURL_VERSION_ESNI`),
implements setting and reporting of this bit, includes dummy
book-keeping for the symbol, adds a build-time configuration
option (`--enable-esni`), provides an extensible check for
resources available to provide ESNI support, and defines a
compiler pre-processor symbol (`USE_ESNI`) accordingly.

Proposed-by: @niallor (Niall O'Reilly)\
Encouraged-by: @sftcd (Stephen Farrell)\
See-also: [this message](https://curl.haxx.se/mail/lib-2019-05/0108.html)

Limitations:
- Book-keeping (symbols-in-versions) needs real release number, not 'DUMMY'.

- Framework is incomplete, as it covers autoconf, but not CMake.

- Check for available resources, although extensible, refers only to
specific work in progress ([described
here](https://github.com/sftcd/openssl/tree/master/esnistuff)) to
implement ESNI for OpenSSL, as this is the immediate motivation
for the proposed change.

## References

CloudFlare blog: [Encrypting SNI: Fixing One of the Core Internet Bugs][corebug]

Cloudflare blog: [Encrypt it or lose it: how encrypted SNI works][esniworks]

IETF Draft: [Encrypted Server Name Indication for TLS 1.3][tlsesni]

---

[tlsesni]: https://datatracker.ietf.org/doc/draft-ietf-tls-esni/
[esniworks]: https://blog.cloudflare.com/encrypted-sni/
[corebug]: https://blog.cloudflare.com/esni/
[defoproj]: https://defo.ie/
[sftcdopenssl]: https://github.com/sftcd/openssl/
[niallorcurl]: https://github.com/niallor/curl/
[niallorreadme]: https://github.com/niallor/curl/blob/master/ESNI-README.md
35 changes: 35 additions & 0 deletions configure.ac
Expand Up @@ -49,6 +49,7 @@ CURL_CHECK_OPTION_CURLDEBUG
CURL_CHECK_OPTION_SYMBOL_HIDING
CURL_CHECK_OPTION_ARES
CURL_CHECK_OPTION_RT
CURL_CHECK_OPTION_ESNI

XC_CHECK_PATH_SEPARATOR

Expand Down Expand Up @@ -4497,6 +4498,35 @@ if test "$enable_altsvc" = "yes"; then
experimental="$experimental alt-svc"
fi

dnl *************************************************************
dnl check whether ESNI support, if desired, is actually available
dnl
if test "x$want_esni" != "xno"; then
AC_MSG_CHECKING([whether ESNI support is available])

dnl assume NOT and look for sufficient condition
ESNI_ENABLED=0
ESNI_SUPPORT=''

dnl OpenSSL with a chosen ESNI function should be enough
dnl so more exhaustive checking seems unnecessary for now
if test "x$OPENSSL_ENABLED" == "x1"; then
AC_CHECK_FUNCS(SSL_get_esni_status,
ESNI_SUPPORT="ESNI support available (OpenSSL with SSL_get_esni_status)"
ESNI_ENABLED=1)
niallor marked this conversation as resolved.
Show resolved Hide resolved

dnl add 'elif' chain here for additional implementations
fi

dnl now deal with whatever we found
if test "x$ESNI_ENABLED" == "x1"; then
AC_DEFINE(USE_ESNI, 1, [if ESNI support is available])
AC_MSG_RESULT($ESNI_SUPPORT)
else
AC_MSG_ERROR([--enable-esni ignored: No ESNI support found])
fi
fi

dnl ************************************************************
dnl hiding of library internal symbols
dnl
Expand Down Expand Up @@ -4618,6 +4648,10 @@ if test "x$OPENSSL_ENABLED" = "x1" -o "x$GNUTLS_ENABLED" = "x1" \
SUPPORT_FEATURES="$SUPPORT_FEATURES HTTPS-proxy"
fi

if test "x$ESNI_ENABLED" = "x1"; then
SUPPORT_FEATURES="$SUPPORT_FEATURES ESNI"
fi

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could squash all your configure commits into a single one to reduce the number of commits and make the PR easier to review...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, indeed. Sorry.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need me to consolidate this, or is it tolerable as it is just this once?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I began to feel that the jumble was hurting even my head. I felt it was useful to use two commits (rather than just one) in order to keep handling of the command-line token --enable-esni separate from applying the corresponding configuration.

AC_SUBST(SUPPORT_FEATURES)

dnl For supported protocols in pkg-config file
Expand Down Expand Up @@ -4801,6 +4835,7 @@ AC_MSG_NOTICE([Configured to build curl/libcurl:
Alt-svc: ${curl_altsvc_msg}
HTTP2: ${curl_h2_msg}
HTTP3: ${curl_h3_msg}
ESNI: ${curl_esni_msg}
Protocols: ${SUPPORT_PROTOCOLS}
Features: ${SUPPORT_FEATURES}
])
Expand Down
1 change: 1 addition & 0 deletions docs/libcurl/symbols-in-versions
Expand Up @@ -926,6 +926,7 @@ CURL_VERSION_BROTLI 7.57.0
CURL_VERSION_CONV 7.15.4
CURL_VERSION_CURLDEBUG 7.19.6
CURL_VERSION_DEBUG 7.10.6
CURL_VERSION_ESNI 7.67.0
CURL_VERSION_GSSAPI 7.38.0
CURL_VERSION_GSSNEGOTIATE 7.10.6 7.38.0
CURL_VERSION_HTTP2 7.33.0
Expand Down
2 changes: 2 additions & 0 deletions include/curl/curl.h
Expand Up @@ -2800,6 +2800,8 @@ typedef struct {
#define CURL_VERSION_ALTSVC (1<<24) /* Alt-Svc handling built-in */
#define CURL_VERSION_HTTP3 (1<<25) /* HTTP3 support built-in */

#define CURL_VERSION_ESNI (1<<26) /* ESNI support */

/*
* NAME curl_version_info()
*
Expand Down
3 changes: 3 additions & 0 deletions lib/version.c
Expand Up @@ -366,6 +366,9 @@ static curl_version_info_data version_info = {
#endif
#if defined(USE_ALTSVC)
| CURL_VERSION_ALTSVC
#endif
#ifdef USE_ESNI
| CURL_VERSION_ESNI
#endif
,
NULL, /* ssl_version */
Expand Down
36 changes: 36 additions & 0 deletions m4/curl-confopts.m4
Expand Up @@ -648,3 +648,39 @@ AC_DEFUN([CURL_CHECK_NTLM_WB], [
NTLM_WB_ENABLED=1
fi
])

dnl CURL_CHECK_OPTION_ESNI
dnl -----------------------------------------------------
dnl Verify whether configure has been invoked with option
dnl --enable-esni or --disable-esni, and set
dnl shell variable want_esni as appropriate.

AC_DEFUN([CURL_CHECK_OPTION_ESNI], [
AC_MSG_CHECKING([whether to enable ESNI support])
OPT_ESNI="default"
AC_ARG_ENABLE(esni,
AC_HELP_STRING([--enable-esni],[Enable ESNI support])
AC_HELP_STRING([--disable-esni],[Disable ESNI support]),
OPT_ESNI=$enableval)
case "$OPT_ESNI" in
no)
dnl --disable-esni option used
want_esni="no"
curl_esni_msg="no (--enable-esni)"
AC_MSG_RESULT([no])
;;
default)
dnl configure option not specified
want_esni="no"
curl_esni_msg="default (--enable-esni)"
AC_MSG_RESULT([(assumed) no])
;;
*)
dnl --enable-esni option used
want_esni="yes"
curl_esni_msg="enabled (--disable-esni)"
experimental="esni"
AC_MSG_RESULT([yes])
;;
esac
])
1 change: 1 addition & 0 deletions src/tool_help.c
Expand Up @@ -540,6 +540,7 @@ static const struct feat feats[] = {
{"MultiSSL", CURL_VERSION_MULTI_SSL},
{"PSL", CURL_VERSION_PSL},
{"alt-svc", CURL_VERSION_ALTSVC},
{"ESNI", CURL_VERSION_ESNI},
};

void tool_help(void)
Expand Down