ClientConfig and RootCertStore improvements

[daxpedda]

Goals

1. Ability to retry reloading native certificates after failure or when required.
2. Different default SNI settings between protocols.
3. Let users supply quinn::Endpoint (Pass quinn::Endpoint for DoQ and DoH3 #2002).
4. Decide at runtime which certificate store to use instead of selecting it with crate features.

Problems

Storage

ResolverOpts

Storing rustls::ClientConfig in ResolverOpts is currently not possible for several reasons:

• ResolverOpts requires Copy (Remove Copy from ResolverOpts #2029).
• ResolverOpts requires Default ([DON'T MERGE] Remove Default from ResolverOpts #2035).
• ResolverOpts requires De/Serialize.

Copy and Default can be removed, but De/Serialize would require one of the following solutions:

1. Remove De/Serialize from `ResolverOpts. Though it's required in other crates: Separate default rustls::ClientConfig for each protocol #2001 (comment).
2. Custom Deserializer for RustlsConfig passing the native certificates error through ... somehow.
3. Use Option<ClientConfig>s and Option<Arc<RootCertStore>> to create ClientConfigs when needed and store RootCertStore for re-usage. Will require a custom PartialEq implementation with Arc::ptr_eq() for RootCertStore.

GenericConnector

I have originally done this in #2001. Not sure what exactly the downside would be here, so this is my preferred solution right now.

If we decide on this course we could also potentially revert #2029.

A new type

This could be stored in a field inside AsyncResolver, which would be quite similar to GenericConnector.

Runtime root certificate store selection

I believe this would require to store ClientConfig somewhere else then ResolverOpts. If we add fields to ResolverOpts that specify which store to use, already initialized ClientConfigs inside ResolverOpts would require to be re-initialized when changing those fields.

We could mitigate this issue by making sure that ResolverOpts can never be accessed through the API again after ClientConfigs are never initialized, which I believe is currently the case anyway. In this case I'm not sure why we would decide to store them in ResolverOpts anyway.

Related issues and PRs

• DNS over HTTP/3 (DoH3) #1939
• Add webpki-roots and native-certs crate features #1943
• Replace lazy_static with once_cell #1944
• DoH3 support #1987
• DoQ default configuration is invalid #1990
• Separate default rustls::ClientConfig for each protocol #2001
• Add QuicClientStreamBuilder::endpoint() #2003
• Pass quinn::Endpoint for DoQ and DoH3 #2002
• Add webpki-roots and native-certs crate features, take 2 #2005
• Remove Copy from ResolverOpts #2029
• Separate default rustls::ClientConfig for each protocol (take 2) #2031
• [DON'T MERGE] Remove Default from ResolverOpts #2035
• Fix DoQ default configuration #2036

[djc]

Note that rustls-platform-verifier devolves to rustls-native-certs (optionally with webpki-roots as fallback) on Linux, so it probably doesn't solve many problems if you consider Linux an important deployment target.

I think cloning ClientConfig once per protocol is probably okay, but I'm wondering how that makes your "retry reloading certificates" scenario look?

[daxpedda]

Note that rustls-platform-verifier devolves to rustls-native-certs (optionally with webpki-roots as fallback) on Linux, so it probably doesn't solve many problems if you consider Linux an important deployment target.

Ah, will remove it from OP then!

I think cloning ClientConfig once per protocol is probably okay, but I'm wondering how that makes your "retry reloading certificates" scenario look?

In that regard it would solve nothing, unfortunately, I would say both these problems are unrelated.
But if you are saying it's not a problem, then I can as well remove it from the OP.

[djc]

So it seems your currently favored solution is to store client configs in the GenericConnector? How does that work out? How does it enable you to replace/update them at a later point in time? The solution in #2001 still seems to rely on a bunch of Lazy values which I don't think enables updates?

[daxpedda]

How does it enable you to replace/update them at a later point in time?

In the case of GenericConnector it means to reload the native certs, users have to create a new Resolver. Depending on how exactly we implement it, it should also be easy to detect load failure when initializing the Resolver.

We have discussed this previously here: #1943 (comment).

The solution in #2001 still seems to rely on a bunch of Lazy values which I don't think enables updates?

These Lazy values were just option.unwrap_or_else(), I was doing that to lazily intialize ClientConfigs, and therefor load certificates, on demand and not on initialization of the Resolver. But now that we have Arc<RootCertStore>, we might want to load those on initialization of Resolver to get a nice error message if loading native certificates fails.

But initializing a new Resolver would still reload native certificates in #2001.