Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ECH-PR #22938

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

ECH-PR #22938

wants to merge 7 commits into from

Conversation

sftcd
Copy link
Contributor

@sftcd sftcd commented Dec 4, 2023

This PR contains an implementation of Encrypted ClientHello (ECH) as specified in the IETF's TLS working group
Internet-draft. As noted in the minutes of a recent IETF meeting, the spec is on the cusp of working group last call, and should become an RFC in the new year. This PR responds to this issue.

The OpenSSL project's policy of course is to not merge such PRs until the RFC issues, which is entirely reasonable. In this case, unfortunately, the PR is (ahem;-) pretty BIG, so as review will likely take some time, I'm opening the PR now in the hope that the stars roughly align and the PR will be sufficiently reviewed and in a suitable state to be merged at about the time the RFC issues.

Development of this PR has been funded by the OTF in the DEfO project which is currently funded for another ~18 months, so we should be able to respond to reviews and continue development of this code until it's part of an OpenSSL release assuming all goes well. (Even if all doesn't go well and it takes longer, we'll still be about:-)

ECH is currently shipping in browsers and is no longer behind a flag. This code successfully interoperates with current browser releases. As part of the DEfO project, we have developed proof-of-concept integrations for ECH with curl, haproxy, apache2, nginx and lighttpd all of which seem to interoperate. This PR also supports the so-called "split-mode" for ECH and the nginx and haproxy integrations can make use of that. More details can be found on the DEfO web site. As the TLS WG have decided to register the ECH extension code-points that are currently in use in browsers and this code, we don't expect to see protocol changes that would affect interop between now and when the RFC issues.

Given that this is a pretty huge PR, I've tried to make it a little easier to review by grouping the changes into a smallish number of related commits, those are:

As posted, this PR passes almost all the CI build tests (there's one cross-compile error I don't understand, and some of the newer ABI diff stuff I'm not sure how to fix). I've not so far done code-coverage testing, as I expect that'll be better done when we're closer to being in a position to merge.

Apologies again that this is so big - we tried and failed to find ways to break it up into meaningful smaller PRs. It's probably just in the nature of the kind of change ECH represents that that's not really doable. I look forward to
responding to any and all review comments as this PR is reviewed.

Checklist
  • documentation is added or updated
  • tests are added or updated

@github-actions github-actions bot added the severity: fips change The pull request changes FIPS provider sources label Dec 4, 2023
@t8m t8m added branch: master Merge to master branch approval: review pending This pull request needs review by a committer approval: otc review pending This pull request needs review by an OTC member triaged: feature The issue/pr requests/adds a feature tests: present The PR has suitable tests present labels Dec 4, 2023
@t8m t8m closed this Dec 5, 2023
@t8m t8m reopened this Dec 5, 2023
@github-actions github-actions bot added the severity: ABI change This pull request contains ABI changes label Dec 5, 2023
Copy link
Member

@t8m t8m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The libcrypto.num and libssl.num changes looks just wrong. Please reset the libcrypto.num and libssl.num files to pristine from the master branch and run make update.

@sftcd
Copy link
Contributor Author

sftcd commented Dec 6, 2023

The libcrypto.num and libssl.num changes looks just wrong. Please reset the libcrypto.num and libssl.num files to pristine from the master branch and run make update.

Ah! So make update does that? Great, and thanks. (The number of times I've manually edited those while rebasing;-). Will do that. Is it better to do that now or wait and batch up a bunch of changes once I've gotten more feedback? (I'm fine with either.)

@t8m
Copy link
Member

t8m commented Dec 6, 2023

Please do that. I want to verify that the ABIDIFF check will detect the changes.

@sftcd
Copy link
Contributor Author

sftcd commented Dec 6, 2023

Please do that. I want to verify that the ABIDIFF check will detect the changes.

Did that, looks like it passed the ABIDIFF one?

@t8m
Copy link
Member

t8m commented Dec 6, 2023

It shows that there is an ABI change and that is reflected by the label.
https://github.com/openssl/openssl/actions/runs/7115342092/job/19371275333?pr=22938

@sftcd
Copy link
Contributor Author

sftcd commented Dec 6, 2023

It shows that there is an ABI change and that is reflected by the label. https://github.com/openssl/openssl/actions/runs/7115342092/job/19371275333?pr=22938

Just checking: that's the expected outcome given new APIs are being added isn't it? (If not, I'm confused:-)

@t8m
Copy link
Member

t8m commented Dec 7, 2023

Just checking: that's the expected outcome given new APIs are being added isn't it? (If not, I'm confused:-)

Yes. I am just saying that the label is just indication of an ABI change. If that ABI change is OK or not has to be carefully reviewed. Also that the ABIDIFF output in the CI job might give some hints.

@hlandau
Copy link
Member

hlandau commented Dec 18, 2023

Thanks for your work on this. I'd like to see this.

However, realistically, this is too big to land as one PR.

I think the way forward here is for you to create a subset of this PR implementing only the minimum viable functionality for ECH on the client side only.

If my reading of your design document is correct, this means providing only the following APIs (at most, preferably less):

int SSL_ech_set1_echconfig(SSL *s, const unsigned char *val, size_t len);
int SSL_CTX_ech_set1_echconfig(SSL_CTX *ctx, const unsigned char *val,
                               size_t len);
int SSL_CTX_ech_set_outer_alpn_protos(SSL *s, const unsigned char *protos,
                                      unsigned int protos_len);
int SSL_ech_set_outer_server_name(SSL *s, const char *outer_name, int no_outer);
int SSL_ech_get_retry_config(SSL *s, unsigned char **ec, size_t *eclen);
int SSL_ech_get_status(SSL *s, char **inner_sni, char **outer_sni);

void SSL_ech_set_callback(SSL *s, SSL_ech_cb_func f);
void SSL_CTX_ech_set_callback(SSL_CTX *ctx, SSL_ech_cb_func f);

#define SSL_OP_ECH_IGNORE_CID                           SSL_OP_BIT(36)

Command line tool updates, and convenience functionality (e.g. to allow clients to calculate ECH configs) can also be omitted in this initial PR. The idea is a MVP PR which makes client-side usage feasible, not necessarily easy.

@sftcd
Copy link
Contributor Author

sftcd commented Dec 18, 2023 via email

@hlandau
Copy link
Member

hlandau commented Dec 18, 2023

I think by the time you had a working client with test code
for the server you'd be back at a nearly-as-big PR. Or were you
thinking of something else? I'm assuming that merging client
code that can only be tested via non-merged application layer
code against the CF test server or one of my test servers wouldn't
be ok to merge. But maybe that's not a good assumption?

I don't see the issue here. IMO an existing ECH server is an acceptable test target.

See 95-test_external_cf_quiche.t for an example of how we did this with QUIC.

@sftcd
Copy link
Contributor Author

sftcd commented Dec 18, 2023

I don't see the issue here. IMO an existing ECH server is an acceptable test target.

Hmm. Interesting. Happy to take a look again at what it'd be like splitting into multiple PRs. That said, and while I don't doubt your advice, I'd be a bit happier putting in that chunk of effort if other maintainers also figured this was a good path forward...

@hlandau
Copy link
Member

hlandau commented Dec 18, 2023

That said, and while I don't doubt your advice, I'd be a bit happier putting in that chunk of effort if other maintainers also figured this was a good path forward...

Absolutely — it would be good to have a second opinion here.

@t8m
Copy link
Member

t8m commented Dec 18, 2023

@sftcd given you've already a working implementation and that an interop test would be if not mandatory then very much welcome, I'd say doing what Hugo suggests would definitely make sense to me.

@sftcd
Copy link
Contributor Author

sftcd commented Dec 18, 2023

OK, I'll take a look at splitting into separate PRs with external tooling for tests and get back here. Given the time of year that'll likely take a while (I'm also looking at fuzzing as discussed earlier). Meanwhile, I'd still appreciate any comments if someone wants to delve into the code changes - those won't be much affected by a split into >1 PR.

@sftcd
Copy link
Contributor Author

sftcd commented Dec 20, 2023

So I've taken a look at splitting the PR into 3: server, client and extras (in that order, each building on the earlier) and while I fear it'll feel a bit like treading a treacle torus (in that I'll end up where I started after loadsa trudging;-) I can see that'll make review/landing more tractable, and refactoring generally does improve things, so I'm up for giving it a shot.

Two asks first though:

  1. I'd like to get some review of the internal data structures now(ish) if possible, as those are used by both client and server so if maintainers prefer changes to those (I suspect some will be, just on the basis that that's always true:-) then it'd be way better to know that now. Those are basically the typedefs in include/internal/ech_local.h so if it were possible to get eyeballs on those before splitting the PR that'd really help. I figure it should be possible to get good review of those, as they are, from folk who know OpenSSL internals, and have a reasonably good idea of the ECH protocol. (Could be a fine over-the-holidays way to spend an hour:-) My main reason for asking this is that if we stick with using the same typedefs for both server and client, the 1st PR (server), will likely involve at least some housekeeping code that'll only be used by clients in the end.

  2. The project policy of only merging after the RFC issues might be more of an issue with multiple, staged PRs - given everything takes a while, that staging could easily add a year, so is there any way the project would be ok with e.g. merging the 1st PR (server, once that's gotten normal approvals) even if the RFC hasn't quite issued (but is clearly on the way)? I guess that'd need an OTC discussion/decision so could we tee that up for an early new year OTC meeting? (Or whatever's the right process.) A hint at a positive answer there would make it easier to keep the wellies on and persist in treacle-trudging:-)

Sound reasonable?

Copy link

@Kichura Kichura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makefile:3686: build_sw got an error code 2 in the m68k-linux-gnu workflow, you might wanna check upon this little detail.

EDIT: Makefile:30897: test/ssl_old_test has an error code of 1.

@sftcd
Copy link
Contributor Author

sftcd commented Jan 5, 2024

Makefile:3686: build_sw got an error code 2 in the m68k-linux-gnu workflow, you might wanna check upon this little detail.

EDIT: Makefile:30897: test/ssl_old_test has an error code of 1.

Yeah, I've no idea about that one tbh, so hints welcome!

this links to a build log for that, the meaty part of which says:

libcrypto.a(libcrypto-lib-hpke_util.o): in function `ossl_hpke_labeled_extract':
hpke_util.c:(.text+0x702): relocation truncated to fit: R_68K_GOT16O against `__func__.3'
libcrypto.a(libcrypto-lib-params.o): in function `OSSL_PARAM_set_octet_string':
params.c:(.text+0x2cea): relocation truncated to fit: R_68K_GOT16O against `__func__.[5](https://github.com/sftcd/openssl/actions/runs/7276480816/job/19826437375#step:9:6)'
params.c:(.text+0x2d5c): relocation truncated to fit: R_[6](https://github.com/sftcd/openssl/actions/runs/7276480816/job/19826437375#step:9:7)8K_GOT16O against `__func__.5'
libcrypto.a(libcrypto-lib-passphrase.o): in function `ossl_pw_get_passphrase':
passphrase.c:(.text+0x4e4): relocation truncated to fit: R_68K_GOT16O against `.LC4'
libcrypto.a(libcrypto-lib-t_x509.o): in function `ossl_x509_print_ex_brief':
t_x509.c:(.text+0xe3a): relocation truncated to fit: R_68K_GOT16O against `.LC42'
libcrypto.a(libcrypto-lib-x_pubkey.o): in function `ossl_i2d_DH_PUBKEY':
x_pubkey.c:(.text+0x14fe): relocation truncated to fit: R_68K_GOT16O against `__func__.[7](https://github.com/sftcd/openssl/actions/runs/7276480816/job/19826437375#step:9:8)'
libcrypto.a(libdefault-lib-cipher_aes.o): in function `aes_cbc_cts_set_ctx_params':
cipher_aes.c:(.text+0xd42): relocation truncated to fit: R_6[8](https://github.com/sftcd/openssl/actions/runs/7276480816/job/19826437375#step:9:9)K_GOT16O against `.LC2'
libcrypto.a(libdefault-lib-cipher_aes.o): in function `aes_cbc_cts_get_ctx_params':
cipher_aes.c:(.text+0xe06): relocation truncated to fit: R_68K_GOT16O against `.LC2'
libcrypto.a(libdefault-lib-cipher_aes.o): in function `aes_cbc_cts_dinit':
cipher_aes.c:(.text+0xf6a): relocation truncated to fit: R_68K_GOT16O against `.LC2'
libcrypto.a(libdefault-lib-cipher_aes.o): in function `aes_cbc_cts_einit':
cipher_aes.c:(.text+0x102e): relocation truncated to fit: R_68K_GOT16O against `.LC2'
libcrypto.a(libdefault-lib-encode_key2any.o): in function `ec_to_EncryptedPrivateKeyInfo_pem_encode':
encode_key2any.c:(.text+0x[9](https://github.com/sftcd/openssl/actions/runs/7276480816/job/19826437375#step:9:10)0fc): additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:30989: test/ssl_old_test] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:3679: build_sw] Error 2
Error: Process completed with exit code 2.

@sftcd
Copy link
Contributor Author

sftcd commented Jan 21, 2024

@sftcd given you've already a working implementation and that an interop test would be if not mandatory then very much welcome, I'd say doing what Hugo suggests would definitely make sense to me.

So (after holidays:-), I've started on this. So far I've got a basic external test using boringssl that runs it's client vs the openssl server and vice versa. That's here and seems to work - comments are welcome. Next step is to do some kind of split into ECH server, client and extras that could each be a separate PR. That'll take a bit of time maybe as it'll call for some possibly extensive refactoring of code I'm sure. (I've not updated this PR yet with that, will think about whether it'll be better to close this and open new ones or to keep this PR number for the first of those.)

Copy link
Contributor

@FdaSilvaYY FdaSilvaYY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very short review

That padding will apply to the Certificate, CertificateVerify and
EncryptedExtensions handshake messages emitted by the server and
will ensure those are at least a minimum length, (currently 1808,
480 and 32 octet minima respectively). The value will also be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

32 Octets...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think the "32 octet" is correct there, as the "minima" is the plural. But maybe my grammar's wrong too;-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember seeing written "octets" few lines below, no ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, but I think the grammar in both cases is correct tbh

doc/man1/openssl-s_client.pod.in Show resolved Hide resolved
doc/man1/openssl-s_client.pod.in Outdated Show resolved Hide resolved
include/openssl/ech.h Outdated Show resolved Hide resolved
@sftcd
Copy link
Contributor Author

sftcd commented May 7, 2024

I made the above changes to a rebased dev branch that I've pushed to github - will push to this branch in a bit. (Late here now.)

@sftcd
Copy link
Contributor Author

sftcd commented May 7, 2024

Huh, I seem to have accidentally closed the PR when pushing to it.... will fix and reopen when I figure out how:-)

@sftcd
Copy link
Contributor Author

sftcd commented May 7, 2024

I think I undid that bad push so hopefully the PR is now up to date with those changes from yesterday and rebased..

@sftcd sftcd reopened this May 7, 2024
Copy link
Contributor

@FdaSilvaYY FdaSilvaYY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs more review, will continue later .

return 0;
}
OSSL_TRACE_BEGIN(TLS) {
BIO_printf(trc_out,"set 8 zeros for ECH acccpt confirm in HRR\n");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo : acccpt

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

/*
* if we called this for inner and did send then
* the following two things were set just before
* returning (i.e. at the bottom of this fucntion)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: fucntion

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

* (because pkt may be a subpacket so we don't here have a pointer to
* the allocated buffer)
*/
__owur static ossl_inline int PACKET_replace(PACKET *pkt, PACKET *newpkt)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const PACKET *newpkt ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, seems ok

ssl/ech.c Outdated
*
* If an extension constructor has side-effects then it is (in general)
* unsafe to call twice. For others, we need to be able to call twice,
* if we do want possibly different values in inner and outer, of if
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or if ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

ssl/ech.c Outdated
* @brief decode the DNS name in a binary RRData
* @param buf points to the buffer (in/out)
* @param remaining points to the remaining buffer length (in/out)
* @param dnsname returns the string form name on success
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

form or from ?

Copy link
Contributor Author

@sftcd sftcd May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

form is right, but string-form is better so did that (same in a few other places)

@@ -428,6 +431,28 @@ typedef int (*SSL_async_callback_fn)(SSL *s, void *arg);

#define SSL_OP_PREFER_NO_DHE_KEX SSL_OP_BIT(35)

#ifndef OPENSSL_NO_ECH
/* Set this to tell client to emit greased ECH values */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: indentation ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure (though indentation in that file seems all over the place)

util/libssl.num Outdated
SSL_CTX_ech_server_flush_keys ? 3_4_0 EXIST::FUNCTION:ECH
SSL_CTX_ech_raw_decrypt ? 3_4_0 EXIST::FUNCTION:ECH
SSL_CTX_ech_set_callback ? 3_4_0 EXIST::FUNCTION:ECH
ossl_ech_make_echconfig ? 3_4_0 EXIST::FUNCTION:ECH
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ossl_... is the prefix for internals methods, not for public ones.

Copy link
Contributor Author

@sftcd sftcd May 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, I made that change in response to Viktor's review (here)

if (ecl_string == NULL)
goto out;
memcpy(ecl_string, lnbuf, readbytes);
/* zap the '\n' if present */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ending ```\n ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure


#if 0
#ifndef OPENSSL_NO_USABLE_ECH
if (!is_fips && hpke_setlibctx(libctx)!=1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: space around !=

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's all under a #if 0 so I just took it out. (Probably a remnant from the HPKE PR which originated in this code too.)

apps/lib/s_cb.c Outdated
{"encrypted ClientHello (draft-13)", TLSEXT_TYPE_ech},
{"outer exts", TLSEXT_TYPE_outer_extensions},
/* not an ECH change, but seems to be missing */
{"early_data", TLSEXT_TYPE_early_data},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is early_data dependent of ECH ?
Else the #endif should move up, no ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not dependent, no. I just added that as it made reading logs easier when I was testing and kept it within the ifdef , but sure, moving it out is better so did that.

Copy link
Contributor

@FdaSilvaYY FdaSilvaYY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another few remarks.

apps/s_client.c Outdated
@@ -2060,11 +2376,37 @@ int s_client_main(int argc, char **argv)
*/
SSL_CTX_set_session_cache_mode(ctx, SSL_SESS_CACHE_CLIENT
| SSL_SESS_CACHE_NO_INTERNAL_STORE);
BIO_printf(bio_err,"Setting new_session_cb\n");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: space after comma.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


When not really doing Encrypted Client Hello (ECH), one can emit a so-called
GREASE value, which is essentially a random value in order to try ensure that
server code doesn't ossify.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember some documentation writing guidelines stating that :

  • do not write using short forms like `don't' .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure - reworded that. (The curl build actually has a linter that objects to that and other usage they don't like;-)

=item B<-ech_dir> I<dir>

Load all new/modified Encrypted Client Hello (ECH) PEM files from the relevant directory.
A file is considered modified if it's modification time is one second or more younger
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit : its modification time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

outer CH messages.

```c
int ossl_ech_make_echconfig(unsigned char *echconfig, size_t *echconfiglen,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General remark: you are almost using only POD data type as method arguments types.

If I was you, i will replace all the pairs of (buffer, length) arguments
with a dedicated simple structure like this :

typedef struct binbuf_t {
   size_t len;
   unsigned char* data;
} BINBUF;

This will factorize and reduce the amount of "basic shores" code, imho.
Unless the project already have this kind of datatype, i will not use it publicly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm - defining a BINBUF struct like the above seems like either a big change that ought be used in loads of places, or else creating an oddity for ECH. Given many other OpenSSL APIs use pairs of unsigned char * and a size_t to represent buffers, I'd say leaving this as-is is better. (Unless of course I'm misinterpreting your comment:-)

various forms and their components displayed.

The "ECHConfig PEM file" format mentioned below is specified in
L<https://datatracker.ietf.org/doc/draft-farrell-tls-pemesni/> and conists of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo : conists

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


/*
* @brief answer a draft-13 ECH, as needed
* @param s is the SSL session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc : better check each session mention as it seems that a rename/refactoring has occured with QUIC support

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, hopefully I got all those now in the various files

@@ -653,6 +680,13 @@ int tls_collect_extensions(SSL_CONNECTION *s, PACKET *packet,
SSLfatal(s, SSL_AD_ILLEGAL_PARAMETER, SSL_R_BAD_EXTENSION);
goto err;
}
#ifndef OPENSSL_NO_ECH
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit : useless #ifndef ... as there is no code inside.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I convinced myself at some stage there may be a crash there and added the comment to remind me to check with some unsupported extension. For the moment, I've added a TODO to make such a test - could be I was wrong in the first place, or something's been fixed since.

ssl/ssl_lib.c Outdated
@@ -4098,6 +4185,13 @@ SSL_CTX *SSL_CTX_new_ex(OSSL_LIB_CTX *libctx, const char *propq,

ssl_ctx_system_config(ret);

#ifndef OPENSSL_NO_ECH
ret->ext.nechs = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: one indent far !

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, some tab chars, fixed

ssl/ech_local.h Outdated
# define OSSL_ECH_SAME_EXT_CONTINUE 2 /* generate a new value for outer CH */

/*
* During extension construction (in extensions_clnt.c
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you may format this block on 80 cols instead of 54 ?
Or there is a reason to keep it narrow ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no reason, re-formatted

unsigned char *pub;
unsigned int nsuites;
ech_ciphersuite_t *ciphersuites;
unsigned int maximum_name_length;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'uint8' ( or uint8_t ?) is already used in current codebase.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I had that at one stage but changed back for some reason (maybe one of the ports run in the workflows) but I don't recall. I've added a TODO to look again at the types used in ECHConfig and ECHConfigList ('cause as you say other structs do use uint16_t etc now).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I checked this out and I think those types are best left as-is. As a test I changed the version field to be a uint16_t but then that generates compile warnings such as the one below due to the PACKET APis dealing with unsigned ints. I'd argue that making such changes would therefore just add lots of casts that could be more error-prone and no real benefit, other than slightly smaller storage for structs which are not expected to be big in real use-cases anyway.

ssl/ech.c: In function 'ECHConfigList_from_binary':
ssl/ech.c:801:40: warning: passing argument 2 of 'PACKET_get_net_2' from incompatible pointer type [-Wincompatible-pointer-types]
  801 |             || !PACKET_get_net_2(&pkt, &ec->version)
      |                                        ^~~~~~~~~~~~
      |                                        |
      |                                        uint16_t * {aka short unsigned int *}
In file included from ssl/ssl_local.h:32,
                 from ssl/ech.c:11:
include/internal/packet.h:150:75: note: expected 'unsigned int *' but argument is of type 'uint16_t *' {aka 'short unsigned int *'}
  150 | __owur static ossl_inline int PACKET_get_net_2(PACKET *pkt, unsigned int *data)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replaced the TODO I'd put in for this with a comment reflecting the above. Will push that next time.

Copy link
Contributor

@FdaSilvaYY FdaSilvaYY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few more comments.
I suggest you extract as separated commits :

  • autogenerated files
  • fuzzer tests

SVCB and HTTPS resource records are defined in
L<https://datatracker.ietf.org/doc/draft-ietf-dnsop-svcb-https/>.

If an ECH public value is associated with an I<SSL> connecction, and if TLS version
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: connecction

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

ssl/ech.c Outdated
* Strings used in ECH crypto derivations (odd format for EBCDIC goodness)
*/
/* "ech accept confirmation" */
static char OSSL_ECH_ACCEPT_CONFIRM_STRING[] = "\x65\x63\x68\x20\x61\x63\x63\x65\x70\x74\x20\x63\x6f\x6e\x66\x69\x72\x6d\x61\x74\x69\x6f\x6e";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: static const ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep probably should be so did that

WPACKET_get_total_written(&hpkt, toklen);
*hrrtok = OPENSSL_malloc(*toklen);
if (*hrrtok == NULL)
return 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This return will create a leak of hpkt and hpkt_mem .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, fixed

ssl/ech.c Outdated
OPENSSL_free(tbf->exts[i]);
OPENSSL_free(tbf->exts);
OPENSSL_free(tbf->encoding_start);
memset(tbf, 0, sizeof(ECHConfig));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really need to clear this content ?
And why this free() function don't free the passed object ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The memset() isn't really needed no, but did help debugging earlier.
As to why not free the passed object, it's because there can be an array of 'em, so the higher level code does that.

ssl/ech.c Outdated
for (i = 0; i != tbf->nrecs; i++)
ECHConfig_free(&tbf->recs[i]);
OPENSSL_free(tbf->recs);
memset(tbf, 0, sizeof(ECHConfigList));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free() functions should really free his argument.
Else you are on way to a lot of confusion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same reason as above - there can be an array, so higher level code frees that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the current codebase, arrays are managed using the STACK_OF() api.
The STACK_OF(...)_pop_free() method will do the array memory release.


# define OSSL_ECH_MAX_LINELEN 1000 /* for a sanity check */

static char *echconfiglist_from_PEM(const char *echkeyfile)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may add a second parameter to return the readbytes value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could do, but as it returns a string, using strlen() on that seems ok, esp. for test code.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to avoid calling strlen() when you already know/compute the result.

@sftcd sftcd mentioned this pull request May 16, 2024
2 tasks
@sftcd
Copy link
Contributor Author

sftcd commented May 17, 2024

Few more comments.

Many thanks for all those - given this code's been under development for quite a while, it's great to get more eyeballs on it to spot things like you've done above.

I suggest you extract as separated commits :

  • autogenerated files
  • fuzzer tests

I'll look at that later if that's ok and push up the above changes first (shortly), then go over the various TODOs added in response to comments above.

@sftcd
Copy link
Contributor Author

sftcd commented May 17, 2024

Commits above include all the obvious fixes from above and the TODOs mentioned. That does seem to have some kind of build problem with curl though so looking at that before the TODOs.

Copy link
Contributor

@FdaSilvaYY FdaSilvaYY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you push fixup commits ?

*/
int ech_2bcompressed(int ind)
{
int nexts = OSSL_NELEM(ech_ext_handling);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nits: const int ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

return OSSL_ECH_SAME_EXT_ERR;
type = ech_ext_handling[tind].type;
/* If this index'd extension won't be compressed, we're done */
if (tind >= nexts)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this condition is true, you already have an OOB read on previous line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well spotted! actually though that check (on 3259) is redundant as it was checked already on 3255 (just before the snippet quoted) so there's not OOB issue and I've removed the redundant check now.


/*
* @brief Try figure out ECHConfig encodng by looking for telltales
* @param eklen is the length of rrval
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is a JavaDoc (or Doxygen ?) syntax to highlight any parameter or variable name.


# define OSSL_ECH_MAX_LINELEN 1000 /* for a sanity check */

static char *echconfiglist_from_PEM(const char *echkeyfile)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to avoid calling strlen() when you already know/compute the result.

# include <internal/ech_helpers.h>
#endif

/* unused, to avoid warning. */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a static variable who is unused should be removed, no ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a pattern in many of the fuzz/*.c files I just copied, e.g. here

ssl/ech.c Outdated
for (i = 0; i != tbf->nrecs; i++)
ECHConfig_free(&tbf->recs[i]);
OPENSSL_free(tbf->recs);
memset(tbf, 0, sizeof(ECHConfigList));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the current codebase, arrays are managed using the STACK_OF() api.
The STACK_OF(...)_pop_free() method will do the array memory release.


/**
* @brief Free an OSSL_ECH_ENCCH
* @param tbf is a ptr to an SSL_ECH structure
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment is off-topic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did s/ev/tbf/ in the function prototype to fix that

@sftcd
Copy link
Contributor Author

sftcd commented May 20, 2024

Can you push fixup commits ?

Sorta, but maybe not;-) What I'm current doing is making changes in my "main" local branch (ECH-draft-13c) and then I have a script to update this PR from that. We also have a CI thing (here) that does a nightly merge with upstream and build for openssl and each of the web servers for which we've done PoC ECH integrations that is also fed from the ECH-draft-13c branch (manually, when interesting changes have happened or we need to fix something to handle merge issues with upstream). So it's easier for me, for now, to follow that workflow. When/if we get this PR closer to something that may be merged, then I guess it'll be better to switch to treating this branch as the "main" one, but I guess we're not there yet.

@sftcd
Copy link
Contributor Author

sftcd commented May 20, 2024

@FdaSilvaYY said (in connection with how >1 ECHConfig is handled in structs...)

In the current codebase, arrays are managed using the STACK_OF() api.
The STACK_OF(...)_pop_free() method will do the array memory release.

That's a fair point - I've not looked at using STACK_OF() but will do. That'd probably be a medium sized change (in terms of lines of code) so I've added a TODO to ssl/ech_local.h for that for now.

@FdaSilvaYY
Copy link
Contributor

Question about GREASE : Is there any common parts with PR #19646 : Add RFC8701 GREASE Support

@sftcd
Copy link
Contributor Author

sftcd commented May 27, 2024

Question about GREASE : Is there any common parts with PR #19646 : Add RFC8701 GREASE Support

Ah - didn't know about that one. GREASEing ECH is a little different (*) but there likely should be some commonality in APIs, especially as GREASEing, being a good idea to counter ossification, is being adopted as a mechanism by other protocols, (e.g. QUIC, MLS) so one could envisage general controls that'd work for more than just TLS or ECH. I'd be happy to collaborate on figuring that out if/when either PR is being handled. One possibility might be to move the ECH GREASEing API code to #19646 perhaps. (Search for GREASE in https://github.com/sftcd/openssl/blob/ECH-PR/doc/designs/ech-api.md for details.)

(*) GREASEing ECH is a little different, as GREASE'd ECH, as well as countering ossification, could be considered to be a partial/potential mitigation against the risk that e.g. nation-state censors would block all ECH - once browsers commonly emit GREASE'd ECH extension values (which they do now), then that should somewhat dis-incent some censors from just blocking all ECH. That difference does justify a more complex API so that non-browser applications have the ability to mimic then-current ECH GREASEing behaviour of e.g. a browser in order to be less likely to have their use of ECH blocked. Note though that in all this we're dealing with risks that are hard-to-impossible to quantify at coding time.

@openssl-machine
Copy link
Collaborator

This PR is in a state where it requires action by @openssl/otc but the last update was 30 days ago

@openssl-machine
Copy link
Collaborator

This PR is in a state where it requires action by @openssl/otc but the last update was 61 days ago

@openssl-machine
Copy link
Collaborator

This PR is in a state where it requires action by @openssl/otc but the last update was 92 days ago

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approval: otc review pending This pull request needs review by an OTC member approval: review pending This pull request needs review by a committer branch: master Merge to master branch severity: ABI change This pull request contains ABI changes severity: fips change The pull request changes FIPS provider sources tests: present The PR has suitable tests present triaged: feature The issue/pr requests/adds a feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants