Update dtls max version for dtls1.3

[fwh-dc]

Another dtls 1.3 specific update. It is dependent on #22259

[openssl-machine]

This PR is in a state where it requires action by @openssl/committers but the last update was 30 days ago

[openssl-machine]

This PR is in a state where it requires action by @openssl/committers but the last update was 61 days ago

[openssl-machine]

This PR is in a state where it requires action by @openssl/committers but the last update was 92 days ago

[fwh-dc]

Test failures are relevant. We need to merge a couple of other PR's before this is working.

[mattcaswell]

Test failures are relevant. We need to merge a couple of other PR's before this is working.

Will those failure go away with a rebase now?

[fwh-dc]

Test failures are relevant. We need to merge a couple of other PR's before this is working.

Will those failure go away with a rebase now?

No. We need #22364 to be merged first. Which is dependent on #22366.

And then I expect there are some test cases that needs to be limited to DTLS 1.2 for it to work. I'll add that work to this PR to fix the pipeline.

[fwh-dc]

@t8m @mattcaswell

A quick update on this.

Please prioritise reviewing the following:
#22936
#22401
#22380
#22360

When they are merged I'll do a rebase of this branch and make some adjustments to make the tests pass (some has to be forced to run on DTLS 1.2 for now). That will enable the dtls 1.3 feature branch to negotiate dtls 1.3. And then we can make the required adjustments to make it correct according to the RFC.

[mattcaswell]

Please prioritise reviewing the following:

All of these are now merged

[fwh-dc]

Yes thanks. I'll update this PR today or tomorrow.

[mattcaswell]

Note that #24161 is going to cause problems for this one. My current plan is:

Rebase the feature/dtls-1.3 branch later today on master (there will be no conflicts from this)
Merge #24161 to master
Once #24226 is approved I can rebase the feature/dtls-1.3 branch again which will bring #24161 onto this feature branch

At that point you will need to update this PR again to fix any resulting issues.

[mattcaswell]

feature/dtls-1.3 has been rebased to just before #24161 merge
#24161 has been merged to master
#24226 has been approved - awaiting 24h timer before merge

[mattcaswell]

Once #24226 is approved I can rebase the feature/dtls-1.3 branch again which will bring #24161 onto this feature branch

This has now also been done.

So this PR needs to be rebased against the latest feature branch. It will also need some amendment to handle the changes from #24161

[fwh-dc]

@mattcaswell Please review again. I've marked a couple of tests for reenablement and some has been forced to DTLS 1.2. Please let me know what you think.

[t8m]

At least some of the CI failures look relevant.

[fwh-dc]

@mattcaswell @t8m

I think I found a fix for the memory leaks, but the patch in 6adbb30 seems to break compilation. TBH I don't understand why. Maybe you know what's causing it?

[mattcaswell]

It is because the test file test/tls13secretstest.c mocks outs some internal functions and links in ssl/tls13_enc.c directly:

openssl/test/build.info

Lines 1028 to 1038 in deaa83a

	# We disable this test completely in a shared build because it deliberately
	# redefines some internal libssl symbols. This doesn't work in a non-shared
	# build
	IF[{- !$disabled{shared} -}]
	PROGRAMS{noinst}=tls13secretstest
	SOURCE[tls13secretstest]=tls13secretstest.c
	DEFINE[tls13secretstest]=OPENSSL_NO_KTLS
	SOURCE[tls13secretstest]= ../ssl/tls13_enc.c ../crypto/packet.c ../crypto/quic_vlint.c
	INCLUDE[tls13secretstest]=.. ../include ../apps/include
	DEPEND[tls13secretstest]=../libcrypto ../libssl libtestutil.a
	ENDIF

To fix it you will need to supply dummy implementations of dtls1_clear_received_buffer and dtls1_clear_sent_buffer in tls13secretstest.c:

void dtls1_clear_received_buffer(SSL_CONNECTION *s)
{
}

void dtls1_clear_sent_buffer(SSL_CONNECTION *s)
{
}

[fwh-dc]

@mattcaswell @t8m

This is ready for review once again. I have added some handling of middlebox compatibility and fixed the memory leak. Please let me know what you think.

[openssl-machine]

This pull request is ready to merge

[mattcaswell]

Pushed. Thanks.