Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ech] rewrite ESNI to ECH draft 15 #437

Merged
merged 110 commits into from
Dec 7, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
e638cc7
Merge branch 'kazuho/hpke' into kazuho/ech
kazuho Nov 8, 2022
9a31fde
remove ESNI stuff
kazuho Nov 7, 2022
22013e4
be certain `tls` is immutable when decoding CH
kazuho Nov 8, 2022
1e3b900
Merge branch 'kazuho/hpke' into kazuho/ech
kazuho Nov 15, 2022
af8deb6
Merge branch 'kazuho/hpke' into kazuho/ech
kazuho Nov 15, 2022
8514d77
encode CH without touching `ptls_t` directly, decode ECHConfigList
kazuho Nov 16, 2022
d91e865
send ech extension / send fake psk in outer
kazuho Nov 16, 2022
b53ad77
emit ECH
kazuho Nov 16, 2022
d872a7d
Merge branch 'kazuho/hpke' into kazuho/ech
kazuho Nov 17, 2022
b1eaf7c
it works (for the most straight-forward case)
kazuho Nov 16, 2022
15dba60
Merge branch 'master' into kazuho/ech
kazuho Nov 17, 2022
16afb15
signal if ech is used
kazuho Nov 17, 2022
5fdebbf
remove esni command line tool
kazuho Nov 17, 2022
5c412b9
[cli] -E and -K options to handle ECH (it works)
kazuho Nov 17, 2022
c06805b
check existence of the extension (and the error code)
kazuho Nov 17, 2022
400ed9e
Update include/picotls.h
kazuho Nov 17, 2022
1cefe7b
`ech` will always be non-NULL with modes other than INNER
kazuho Nov 18, 2022
36f855e
less magic numbers
kazuho Nov 18, 2022
9468610
add and recognize padding
kazuho Nov 18, 2022
b851be2
restore msghash_off, it points mid-message when resuming
kazuho Nov 18, 2022
d410403
emit and check ECH accept confirmation hash
kazuho Nov 18, 2022
3fc88d0
Merge branch 'master' into kazuho/ech
kazuho Nov 18, 2022
c7fc502
run two hashes for CHInner and CHOuter, choose the right one
kazuho Nov 18, 2022
636a724
refactor as a preparation
kazuho Nov 18, 2022
3f942d8
generate HRR.ECH (and we can roll the key schedule when sending state…
kazuho Nov 18, 2022
f77c13f
"confirm" implies acceptance
kazuho Nov 18, 2022
2c6f84c
[ECH] handle HRR correctly
kazuho Nov 19, 2022
c417915
check ECH.type always (as well as concentrating the logic)
kazuho Nov 21, 2022
c9b6b9b
ServerHello.ECH can exist unless when the server responds to inner CH
kazuho Nov 21, 2022
459e998
add I/F to obtain the type of the handshake
kazuho Nov 22, 2022
6705508
fix the encoded order
kazuho Nov 22, 2022
d6cdb2c
HKDF-Expand-Label being used is that of RFC 8446, hence uses the "tls…
kazuho Nov 22, 2022
8310c09
use const-time op
kazuho Nov 22, 2022
828eefc
key-schedule uses the transcript with confirmation hash
kazuho Nov 22, 2022
a9ac007
Merge branch 'master' into kazuho/ech
kazuho Nov 22, 2022
df0891d
CHinner MUST NOT offer tls 1.2 or below
kazuho Nov 22, 2022
d7d4c46
[ECH] test variations, e.g., retry
kazuho Nov 24, 2022
71479e4
use wrapper function so as to not miss setting fields
kazuho Nov 24, 2022
205e194
we can say that ECH is used whenever ECH AEAD context is available
kazuho Nov 24, 2022
c0f58ca
ciphers given significance, as it is the only attribute used on both …
kazuho Nov 24, 2022
f63838e
test configuration mismatch
kazuho Nov 24, 2022
c385e1f
send / receive retry_configs
kazuho Nov 24, 2022
ba1baf3
add FIXME
kazuho Nov 24, 2022
bddb83a
oops
kazuho Nov 24, 2022
6068d6f
[ECH] do not touch key_schedule when determining acceptance
kazuho Nov 24, 2022
36a6c79
remove ESNI stuff
kazuho Nov 24, 2022
4cfcc64
replay entire ECH extension when ECH is rejected via HRR
kazuho Nov 24, 2022
6d193a0
upon ech config mismatch, report retry_config to the application iff …
kazuho Nov 24, 2022
5875465
split ECH config applicability testing (ignore upon failure) vs. ECH …
kazuho Nov 24, 2022
c461703
send ECH_REQUIRED alert if rejected, saving retry_configs correctly
kazuho Nov 25, 2022
9edab68
it's a MISmatch
kazuho Nov 25, 2022
e2e6dc2
p256 might be the only algorithm that we support
kazuho Nov 25, 2022
9669e49
dispose state when AEAD decryption fails, otherwise `ptls_is_ech_hand…
kazuho Nov 25, 2022
2ea2080
clarify the contract
kazuho Nov 25, 2022
50c428c
make it simple
kazuho Nov 25, 2022
54d10fa
consistent naming convention
kazuho Nov 25, 2022
7f59712
add comment
kazuho Nov 25, 2022
5038530
better to rename "select_one" now that we have `select_outer` that se…
kazuho Nov 25, 2022
3f07e64
move the condition out, add comment
kazuho Nov 25, 2022
697e7b4
unless the client offered ECH, reject EE.ECH
kazuho Nov 25, 2022
6ff7ee6
outer- and inner-random have to be identical unless ECH is used
kazuho Nov 25, 2022
909d974
retain innerCH.random separately
kazuho Nov 25, 2022
1672380
[ECH] add I/F to obtain kem/cipher being used
kazuho Nov 25, 2022
a91ae5f
send retry_config only when we are capable of accepting ECH
kazuho Nov 25, 2022
37d4c33
in PSK mode, CertificateRequest is rejected by the state machine (and…
kazuho Nov 28, 2022
1c7b115
clang-format
kazuho Nov 28, 2022
761cc03
add note that we are not following the spec
kazuho Nov 28, 2022
c58adc7
do not use ECH even when config is provided, unless server name is a …
kazuho Nov 28, 2022
92479ba
merge the struct
kazuho Nov 28, 2022
054db85
pass server-name as argument as it can be ECH.public_name
kazuho Nov 28, 2022
a6672c0
public_name is at least one byte
kazuho Nov 28, 2022
04b67ad
report error code
kazuho Nov 28, 2022
89779c4
create helper
kazuho Nov 28, 2022
9a95d7f
enc is at least one byte
kazuho Nov 28, 2022
4e01405
Merge branch 'master' into kazuho/ech
kazuho Nov 28, 2022
486e6f6
use `ptls_decode8`
kazuho Nov 28, 2022
3ad9194
ignore ECHConfig that have IP address as public name
kazuho Nov 28, 2022
65f4c7a
oops
kazuho Nov 28, 2022
901be76
payload is at least one byte
kazuho Nov 28, 2022
449bbec
reorder and clarify the logic
kazuho Nov 28, 2022
a374e42
rely on the decode function
kazuho Nov 28, 2022
cd4aaa4
use constant, state check in `decode_server_hello`
kazuho Nov 28, 2022
7208a7e
Merge branch 'master' into kazuho/ech
kazuho Nov 29, 2022
cab1a37
add new extensions to the table, rely on that
kazuho Nov 29, 2022
6648158
dispose of ECH AEAD context during handshake, decryption failure of i…
kazuho Nov 29, 2022
5aa73f3
use the existing function to discard ECH state after Hello exchange
kazuho Nov 29, 2022
e3666d4
track known extensions rather than the smallest 64 (otherwise we cann…
kazuho Nov 29, 2022
89cfbe2
Merge branch 'master' into kazuho/ech
kazuho Nov 29, 2022
80e1c4f
clear remaining ECH state even when HRR is used
kazuho Nov 29, 2022
21cf7c2
when ECH exchange is complete reduce the number of hashes too
kazuho Nov 29, 2022
48c7a92
no need to write after duplicate
kazuho Nov 29, 2022
f0360b4
add test for rebuilding inner CH
kazuho Nov 29, 2022
a3cfa2f
rebuild error is ILLEGAL_PARAMETER
kazuho Nov 29, 2022
df35659
encrypted_client_hello extension cannot be referred to by ech_outer_e…
kazuho Nov 29, 2022
c6d52f3
Merge branch 'master' into kazuho/ech
kazuho Dec 2, 2022
e8fe79e
Merge branch 'master' into kazuho/ech
kazuho Dec 4, 2022
7470a50
[msvc] remove picotls-esni
huitema Dec 4, 2022
398d39c
explicit cast to suppress warning
huitema Dec 4, 2022
5bbd77b
fix stuff that MSVC does not like
huitema Dec 4, 2022
ae30a2c
use sprintf instead
kazuho Dec 4, 2022
bf81477
use bogus blob rather than adding one byte to only one of the variabl…
kazuho Dec 4, 2022
8758ee3
Merge branch 'kazuho/ech-msvc' into kazuho/ech (reorganize & merge #449)
kazuho Dec 4, 2022
62c4bca
Merge branch 'master' into kazuho/ech
kazuho Dec 5, 2022
04dfe46
reflect the fact that the supported set of HPKE cipher-suites can be …
kazuho Dec 5, 2022
347a32e
add API for encoding ECHConfig
kazuho Dec 5, 2022
e6d9bd0
turn `-E` into a read-write file so that it can be used for storing r…
kazuho Dec 7, 2022
113fb5d
length of public_name field is 1-byte
kazuho Dec 7, 2022
8641129
immedaetly send alert and exit when ECH_REQUIRED is generated
kazuho Dec 7, 2022
8755654
send ECH_REQUIRED alert after Finished, as the draft suggests
kazuho Dec 7, 2022
444b745
update expected behavior following the change in the previous commit
kazuho Dec 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 0 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,6 @@ IF (OPENSSL_FOUND AND NOT (OPENSSL_VERSION VERSION_LESS "1.0.1"))
TARGET_LINK_LIBRARIES(picotls-openssl ${OPENSSL_CRYPTO_LIBRARIES} picotls-core ${CMAKE_DL_LIBS})
ADD_EXECUTABLE(cli t/cli.c lib/pembase64.c)
TARGET_LINK_LIBRARIES(cli picotls-openssl picotls-core)
ADD_EXECUTABLE(picotls-esni src/esni.c)
TARGET_LINK_LIBRARIES(picotls-esni picotls-openssl picotls-core ${OPENSSL_CRYPTO_LIBRARIES} ${CMAKE_DL_LIBS})

ADD_EXECUTABLE(test-openssl.t
${MINICRYPTO_LIBRARY_FILES}
Expand Down
128 changes: 55 additions & 73 deletions include/picotls.h
Original file line number Diff line number Diff line change
Expand Up @@ -171,12 +171,6 @@ extern "C" {
#define PTLS_HPKE_AEAD_AES_256_GCM 2
#define PTLS_HPKE_AEAD_CHACHA20POLY1305 3

/* ESNI */
#define PTLS_ESNI_VERSION_DRAFT03 0xff02

#define PTLS_ESNI_RESPONSE_TYPE_ACCEPT 0
#define PTLS_ESNI_RESPONSE_TYPE_RETRY_REQUEST 1

/* error classes and macros */
#define PTLS_ERROR_CLASS_SELF_ALERT 0
#define PTLS_ERROR_CLASS_PEER_ALERT 0x100
Expand Down Expand Up @@ -212,9 +206,11 @@ extern "C" {
#define PTLS_ALERT_INTERNAL_ERROR 80
#define PTLS_ALERT_USER_CANCELED 90
#define PTLS_ALERT_MISSING_EXTENSION 109
#define PTLS_ALERT_UNSUPPORTED_EXTENSION 110
#define PTLS_ALERT_UNRECOGNIZED_NAME 112
#define PTLS_ALERT_CERTIFICATE_REQUIRED 116
#define PTLS_ALERT_NO_APPLICATION_PROTOCOL 120
#define PTLS_ALERT_ECH_REQUIRED 121

/* TLS 1.2 */
#define PTLS_TLS12_MASTER_SECRET_SIZE 48
Expand All @@ -233,7 +229,6 @@ extern "C" {
#define PTLS_ERROR_STATELESS_RETRY (PTLS_ERROR_CLASS_INTERNAL + 6)
#define PTLS_ERROR_NOT_AVAILABLE (PTLS_ERROR_CLASS_INTERNAL + 7)
#define PTLS_ERROR_COMPRESSION_FAILURE (PTLS_ERROR_CLASS_INTERNAL + 8)
#define PTLS_ERROR_ESNI_RETRY (PTLS_ERROR_CLASS_INTERNAL + 8)
#define PTLS_ERROR_REJECT_EARLY_DATA (PTLS_ERROR_CLASS_INTERNAL + 9)
#define PTLS_ERROR_DELEGATE (PTLS_ERROR_CLASS_INTERNAL + 10)
#define PTLS_ERROR_ASYNC_OPERATION (PTLS_ERROR_CLASS_INTERNAL + 11)
Expand Down Expand Up @@ -588,41 +583,6 @@ typedef const struct st_ptls_hpke_cipher_suite_t {
ptls_aead_algorithm_t *aead;
} ptls_hpke_cipher_suite_t;

/**
* holds ESNIKeys and the private key (instantiated by ptls_esni_parse, freed using ptls_esni_dispose)
*/
typedef struct st_ptls_esni_context_t {
ptls_key_exchange_context_t **key_exchanges;
struct {
ptls_cipher_suite_t *cipher_suite;
uint8_t record_digest[PTLS_MAX_DIGEST_SIZE];
} * cipher_suites;
uint16_t padded_length;
uint64_t not_before;
uint64_t not_after;
uint16_t version;
} ptls_esni_context_t;

/**
* holds the ESNI secret, as exchanged during the handshake
*/

#define PTLS_ESNI_NONCE_SIZE 16

typedef struct st_ptls_esni_secret_t {
ptls_iovec_t secret;
uint8_t nonce[PTLS_ESNI_NONCE_SIZE];
uint8_t esni_contents_hash[PTLS_MAX_DIGEST_SIZE];
struct {
ptls_key_exchange_algorithm_t *key_share;
ptls_cipher_suite_t *cipher;
ptls_iovec_t pubkey;
uint8_t record_digest[PTLS_MAX_DIGEST_SIZE];
uint16_t padded_length;
} client;
uint16_t version;
} ptls_esni_secret_t;

#define PTLS_CALLBACK_TYPE0(ret, name) \
typedef struct st_ptls_##name##_t { \
ret (*cb)(struct st_ptls_##name##_t * self); \
Expand Down Expand Up @@ -668,10 +628,6 @@ typedef struct st_ptls_on_client_hello_parameters_t {
const uint8_t *list;
size_t count;
} server_certificate_types;
/**
* if ESNI was used
*/
unsigned esni : 1;
/**
* set to 1 if ClientHello is too old (or too new) to be handled by picotls
*/
Expand Down Expand Up @@ -710,9 +666,11 @@ PTLS_CALLBACK_TYPE(int, sign_certificate, ptls_t *tls, ptls_async_job_t **async,
* callback to the invocation of the verify_sign callback, verify_sign is called with both data and sign set to an empty buffer.
* The implementor of the callback should use that as the opportunity to free any temporary data allocated for the verify_sign
* callback.
* The name of the server to be verified, if any, is provided explicitly as `server_name`. When ECH is offered by the client but
* the was rejected by the server, this value can be different from that being sent via `ptls_get_server_name`.
*/
typedef struct st_ptls_verify_certificate_t {
int (*cb)(struct st_ptls_verify_certificate_t *self, ptls_t *tls,
int (*cb)(struct st_ptls_verify_certificate_t *self, ptls_t *tls, const char *server_name,
int (**verify_sign)(void *verify_ctx, uint16_t algo, ptls_iovec_t data, ptls_iovec_t sign), void **verify_data,
ptls_iovec_t *certs, size_t num_certs);
/**
Expand Down Expand Up @@ -766,10 +724,12 @@ typedef struct st_ptls_decompress_certificate_t {
ptls_iovec_t input);
} ptls_decompress_certificate_t;
/**
* provides access to the ESNI shared secret (Zx). API is subject to change.
* ECH: creates the AEAD context to be used for "Open"-ing inner CH. Given `config_id`, the callback looks up the ECH config and the
* corresponding private key, invokes `ptls_hpke_setup_base_r` with provided `cipher`, `enc`, and `info_prefix` (which will be
* "tls ech" || 00).
*/
PTLS_CALLBACK_TYPE(int, update_esni_key, ptls_t *tls, ptls_iovec_t secret, ptls_hash_algorithm_t *hash,
const void *hashed_esni_contents);
PTLS_CALLBACK_TYPE(ptls_aead_context_t *, ech_create_opener, ptls_hpke_kem_t **kem, ptls_hpke_cipher_suite_t **cipher, ptls_t *tls,
uint8_t config_id, ptls_hpke_cipher_suite_id_t cipher_id, ptls_iovec_t enc, ptls_iovec_t info_prefix);

/**
* the configuration
Expand Down Expand Up @@ -799,9 +759,30 @@ struct st_ptls_context_t {
size_t count;
} certificates;
/**
* list of ESNI data terminated by NULL
* ECH
*/
ptls_esni_context_t **esni;
struct {
struct {
/**
* list of HPKE symmetric cipher-suites (set to NULL to disable ECH altogether)
*/
ptls_hpke_cipher_suite_t **ciphers;
/**
* KEMs being supported
*/
ptls_hpke_kem_t **kems;
} client;
struct {
/**
* callback that does ECDH key exchange and returns the AEAD context
*/
ptls_ech_create_opener_t *create_opener;
/**
* ECHConfigList to be sent to the client when there is mismatch (or when the client sends a grease)
*/
ptls_iovec_t retry_configs;
} server;
} ech;
/**
*
*/
Expand Down Expand Up @@ -897,10 +878,6 @@ struct st_ptls_context_t {
*
*/
ptls_decompress_certificate_t *decompress_certificate;
/**
*
*/
ptls_update_esni_key_t *update_esni_key;
/**
*
*/
Expand Down Expand Up @@ -961,9 +938,18 @@ typedef struct st_ptls_handshake_properties_t {
*/
unsigned negotiate_before_key_exchange : 1;
/**
* ESNIKeys (the value of the TXT record, after being base64-"decoded")
* ECH
*/
ptls_iovec_t esni_keys;
struct {
/**
* config offered by server e.g., by HTTPS RR
*/
ptls_iovec_t configs;
/**
* slot to save the config obtained from server on mismatch; user must free the returned blob by calling `free`
*/
ptls_iovec_t *retry_configs;
} ech;
} client;
struct {
/**
Expand Down Expand Up @@ -1159,7 +1145,7 @@ static uint8_t *ptls_encode_quicint(uint8_t *p, uint64_t v);
ptls_buffer_push(_buf, (type)); \
ptls_buffer_push_block(_buf, 3, block); \
if (_key_sched != NULL) \
ptls__key_schedule_update_hash(_key_sched, _buf->base + mess_start, _buf->off - mess_start); \
ptls__key_schedule_update_hash(_key_sched, _buf->base + mess_start, _buf->off - mess_start, 0); \
} while (0)

#define ptls_push_message(emitter, key_sched, type, block) \
Expand Down Expand Up @@ -1481,6 +1467,10 @@ int ptls_handshake_is_complete(ptls_t *tls);
* returns if a PSK (or PSK-DHE) handshake was performed
*/
int ptls_is_psk_handshake(ptls_t *tls);
/**
* return if a ECH handshake was performed, as well as optionally the kem and cipher-suite being used
*/
int ptls_is_ech_handshake(ptls_t *tls, ptls_hpke_kem_t **kem, ptls_hpke_cipher_suite_t **cipher);
/**
* returns a pointer to user data pointer (client is reponsible for freeing the associated data prior to calling ptls_free)
*/
Expand Down Expand Up @@ -1683,7 +1673,7 @@ static void ptls_aead__do_encrypt_v(ptls_aead_context_t *ctx, void *_output, ptl
/**
* internal
*/
void ptls__key_schedule_update_hash(ptls_key_schedule_t *sched, const uint8_t *msg, size_t msglen);
void ptls__key_schedule_update_hash(ptls_key_schedule_t *sched, const uint8_t *msg, size_t msglen, int use_outer);
/**
* clears memory
*/
Expand All @@ -1696,6 +1686,11 @@ extern int (*volatile ptls_mem_equal)(const void *x, const void *y, size_t len);
* checks if a server name is an IP address.
*/
int ptls_server_name_is_ipaddr(const char *name);
/**
* encodes one ECH Config
*/
int ptls_ech_encode_config(ptls_buffer_t *buf, uint8_t config_id, ptls_hpke_kem_t *kem, ptls_iovec_t public_key,
ptls_hpke_cipher_suite_t **ciphers, uint8_t max_name_length, const char *public_name);
/**
* loads a certificate chain to ptls_context_t::certificates. `certificate.list` and each element of the list is allocated by
* malloc. It is the responsibility of the user to free them when discarding the TLS context.
Expand All @@ -1713,19 +1708,6 @@ int ptls_hpke_setup_base_s(ptls_hpke_kem_t *kem, ptls_hpke_cipher_suite_t *ciphe
*/
int ptls_hpke_setup_base_r(ptls_hpke_kem_t *kem, ptls_hpke_cipher_suite_t *cipher, ptls_key_exchange_context_t *keyex,
ptls_aead_context_t **ctx, ptls_iovec_t pk_s, ptls_iovec_t info);
/**
*
*/
int ptls_esni_init_context(ptls_context_t *ctx, ptls_esni_context_t *esni, ptls_iovec_t esni_keys,
ptls_key_exchange_context_t **key_exchanges);
/**
*
*/
void ptls_esni_dispose_context(ptls_esni_context_t *esni);
/**
* Obtain the ESNI secrets negotiated during the handshake.
*/
ptls_esni_secret_t *ptls_get_esni_secret(ptls_t *ctx);
/**
*
*/
Expand Down
6 changes: 3 additions & 3 deletions lib/openssl.c
Original file line number Diff line number Diff line change
Expand Up @@ -1420,7 +1420,7 @@ static int verify_cert_chain(X509_STORE *store, X509 *cert, STACK_OF(X509) * cha
return ret;
}

static int verify_cert(ptls_verify_certificate_t *_self, ptls_t *tls,
static int verify_cert(ptls_verify_certificate_t *_self, ptls_t *tls, const char *server_name,
int (**verifier)(void *, uint16_t, ptls_iovec_t, ptls_iovec_t), void **verify_data, ptls_iovec_t *certs,
size_t num_certs)
{
Expand All @@ -1445,7 +1445,7 @@ static int verify_cert(ptls_verify_certificate_t *_self, ptls_t *tls,
}
sk_X509_push(chain, interm);
}
ret = verify_cert_chain(self->cert_store, cert, chain, ptls_is_server(tls), ptls_get_server_name(tls), &ossl_x509_err);
ret = verify_cert_chain(self->cert_store, cert, chain, ptls_is_server(tls), server_name, &ossl_x509_err);
} else {
ret = PTLS_ALERT_CERTIFICATE_REQUIRED;
ossl_x509_err = 0;
Expand Down Expand Up @@ -1515,7 +1515,7 @@ X509_STORE *ptls_openssl_create_default_certificate_store(void)
return NULL;
}

static int verify_raw_cert(ptls_verify_certificate_t *_self, ptls_t *tls,
static int verify_raw_cert(ptls_verify_certificate_t *_self, ptls_t *tls, const char *server_name,
int (**verifier)(void *, uint16_t algo, ptls_iovec_t, ptls_iovec_t), void **verify_data,
ptls_iovec_t *certs, size_t num_certs)
{
Expand Down