DoT dns resolution stopped working - Related to Let's Encrypt Root CA Expiry #5257
Comments
|
hi |
|
Yes, I use the os-acme-client plugin to request a Let's Encrypt certificate for use by the OPNsense administration web interface. (i.e. |
|
i think there is no official solution yet. please keep in mind that this is not an official solution you need to go to the System: Trust: Authorities page, only the top should remain: This leaves only a short chain. |
|
Thank you so much. That worked. The bottom cert which I removed was: Am happy to close this issue unless it needs to be left open until an official solution is put in place. |
|
Eventually trust store updates will fix this so there is nothing else to to from our point but wait for their updates and release it along with the next 21.7.x. Cheers, |
|
@fichtner hi, sorry for repeating, but I would not rely on the trusts update (opnsense/plugins#2554 (comment)). there is a feeling that it still have to do something with the |
|
We could exclude expired certs from system: trust: authorities in system_trust_configure() ? Personally I don't see the point of OpenSSL wanting to trust an expired chain but ok :) |
|
As far as I understand, the problem is not in the presence of an expired root certificate in the store, but in the presence of a valid intermediate certificate signed by this root CA in the store (acme plugin puts the ISRG cross-signed intermediate cert in the config, and the system_trust_configure() puts it in the store). the absence of the root certificate will only change the error (instead of complaining about the certificate's dates, there will be a complaint about its absence), but will not change the choice of the chain (because ISRG cross-signed intermediate cert is valid). |
|
Oh right I misread:
Honestly it's difficult ship anything that is self-correcting in this case. Non-self-signed DST Root CA X3 is defunct and needs to be removed manually and ignored from acme-client during registering certificates in config.xml. How this was signed knowing that the root expires 3 years beforehand is... err... reckless. |
acme plugin not registering DST Root cert in config. but it registers |
… to disk to avoid non valid paths being trusted. (ref #5257) ca-root-nss should be valid at all times, we shouldn't (ever) try to cleanse whats being shipped as part of the system, but user input can be unsafe leading to dangerous situations. Eventually we could also consider preventing bundles being imported in the authorities section, but that wouldn't fix issues with already deployed certificates and user input can still lead to broken chains easily.
|
@kulikov-a can you try d8ddef4 ? |
|
@AdSchellevis Hi! thanks for joining! so imho the key is that system_trust_configure puts the intermediate CA's cert (ISRG "Root" X1) from long chain in trusts thus forcing the openssl to select a long chain (ending in an expired DST root) it seems to me that the solution would be to add (along with your changes) a certificate issuer check. and only import self-signed certificates (root). (since I don't know why it was previously decided to import all certificates, I limited to the option of selectively refusing to import in my proposal at opnsense/plugins#2554 (comment)) I apologize if it is difficult to understand me sometimes ) |
|
hi @kulikov-a,
That is annoying indeed, but not something we can fix from our end. Ideally openssl only considers the valid path, in which case it will omit the expired
It shouldn't as far as I know as
yes, it does, hence the expired
I simply can't agree at the moment, I wasn't able to replicate the issue either when only the proper non expired certificates are included. can you please first check my commit and reload the certificates using the standard
No problem, I know you're only trying to help here. |
of course )
and so it happens. if there are no intermediate certificates from the chain in the trusted store. in this case (since
there is two I'm glad that we seem to understand each other, we just don't agree ) |
|
@kulikov-a it could be the case that we don't agree, my question is can you break EDIT never mind, I managed to replicate your behaviour on my end, importing X1 with serial 85078200265644417569109389142156118711. Other than removing the expired cert from nss I don't see a simple and valid option to be honest. |
|
@AdSchellevis yes, so
Oh ( |
|
@kulikov-a the code reference looks reasonable to explain the difference between
I wouldn't go that far, it's just complicated and it looks like not all of the reported cases relate to the same issue, it's easy to get confused.
Right, my assumption on this topic was wrong, but I'm finally able to reproduce the exact same issue as we're talking about here, so that's progress.
I can't for the same reason :) Since their can be side affects (like the webserver not feeding the chain and installs trusting the local intermediates), it would probably be safest to be able to disable the new "default" behaviour. Let me think about this a bit tomorrow and see what we can come up with |
Sure! Thanks a lot! |
… by default into the local trust store. as discussed in #5257 When someone adds an intermediate certificate into the trust store leading either into a missing or expired root, other paths aren't being evaluated anymore, leading into verification errors. In case someone would like to enforce saving the intermediates, System->Settings->General introduces a new trust section to revert back to the old behaviour.
|
@kulikov-a this 5b9d7ba should be it then. Exclude intermediates by default, optionally enable via System->Settings->General |
|
@AdSchellevis |
|
@kulikov-a yes you may :) here you go 56e66ec, the general page reload actions isn't too great flushing every item unconditionally, but this will do. |
|
Thanks again!) |
…om being flushed to disk by default to avoid non valid paths being trusted. PR: #5257
…om being flushed to disk by default to avoid non valid paths being trusted. PR: #5257
|
Thanks for the solution! |
|
@mayerthomas not a stupid question, I do expect we will ship these as part of 21.7.4 as well. (their already queued https://github.com/opnsense/core/commits/stable/21.7/src) |
…om being flushed to disk by default to avoid non valid paths being trusted. PR: opnsense/core#5257
Important notices
Before you add a new report, we ask you kindly to acknowledge the following:
Describe the bug
The root CA DST Root CA X3 expired on September 30, 2021.
See: https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/
Ever since that date, the DNS over TLS feature of Unbound in OPNsense has stopped working for me.
My DoT servers all use a Let's Encrypt certificate and have valid certificate chains.
I believe something in OPNsense is not correctly detecting a valid certificate chain for these servers.
Certificate chain was verified using multiple methods:
To Reproduce
Steps to reproduce the behaviour:
Expected behaviour
I expect the DNS query in step 4 to work and return an IP address for the fqdn www.google.com.
Instead, I get:
Describe alternatives you considered
DNS resolution works fine by using a DoT server which doesn't use a Let's Encrypt certificate, e.g. Quad9's
9.9.9.9:853or Cloudflare's1.1.1.1:853.Relevant log files
Proof that certificate chain is valid for 145.100.185.15:853
Unbound log msgs:
Additional context
I've tried several DoT servers and all of them which use a Let's Encrypt certificate show the same behaviour.
Environment
Software version used and hardware type if relevant, e.g.:
OPNsense 21.7.3_3-amd64
FreeBSD 12.1-RELEASE-p20-HBSD
OpenSSL 1.1.1l 24 Aug 2021
The text was updated successfully, but these errors were encountered: