New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bypass mTLS by mixing SNI and Host headers #3340
Comments
|
FWIW i've recreated this behavior on the latest version of the Helm chart, running AES 1.13.2 as well |
|
Looking at the generated Envoy config, i'm guessing the cause is the first entry in the filter_chain, which matches any TLS traffic: It looks like Ambassador applies every mapping to this "catch-all" TLS filter ( Later in the chain, there is a correct filter for my mTLS backend which accurately matches based on server name: I think this explains the behavior I'm seeing but why does Ambassador do this? It appears to add every route to the default wildcard vhost and filter, which means you can bypass any client TLS requirement by either specifying a bogus SNI (or not including one) and manually supplying a Host header since the upstream route will still be there and match the vhost routes? |
|
Thanks for the report on mTLS, and thanks as well for the excellent reproducer! I have confirmed your research: it is indeed possible to bypass mTLS with your configuration, in 1.7 and subsequent 1.X releases. The catchall behavior you describe is definitely part of it -- I won't bore you to death with the whole story, but the short version is that we sometimes have to infer things in 1.X that require the catchall. Your situation appears to be one where we take that too far. Although there's no real workaround in 1.X, there are some mitigations. A malicious client doesn't actually get to bypass all of TLS: what they really get to do is switch from the TLS configuration dictated by SNI to the TLS configuration dictated by the Obviously, though, confusing mTLS in this way shouldn't be possible at all. We're investigating a fix for the 1.X family, and we have confirmed that our upcoming 2.0.0 release is not vulnerable to this issue at all. We'll be sure to keep you in the loop as we have more news. Thanks again for the report! |
|
Hi @kflynn Thank you for the response. I have one clarifying technical question about the proposed mitigation: even if we ensure that none of our Hosts behind Ambassador disable mTLS, it would still be possible to confuse the client ca_secret for each Host’s TLSContext right? So if Service A and Service B are both using mTLS but with different CAs, a client cert for Service A could be used to establish the connection to Ambassador via sending Service A’s SNI but then talk to Service B via Host header mappings? |
|
@ropnop So first up, the developer preview of 2.0.0 did indeed ship, and it does fix this issue. Investigating a fix for 1.13 mostly turned up roadblocks, so we're not planning to backport to 1.13 at this point – the way it's addressed in 2.0 is really much better than how 1.13 handles things. In 1.13, I think that you can confuse things as you describe only if you don't explicitly define a wildcard
|
|
I'm going to go ahead and close this, but feel free to reopen if needed! |
Describe the bug
If Ambassador is deployed with multiple hosts/tlscontexts with some using mTLS (
cert_required: true) and some not, it is possible to bypass the mTLS requirements for the backends that have it configured by sending an SNI for a TLSContext that does not.I believe this stems from the fact that Ambassador matches TLSContexts based on SNI, but Mappings based on Host header. If
service1.upstreamdoes not require mTLS butservice2.upstreamdoes, it looks like its possible to talk directly toservice1.upstreamwithout a valid client certificate by doing something like:In some configurations it is also possible to do the same by just talking directly to the Ambassador LoadBalancer IP address (not sending SNI at all). This works if the "default" TLSContext does not require mTLS (I'm not exactly clear what TLSContext Ambassador will fall back on if no SNI is given)
To Reproduce
I've put together a working minimal example with 2 services and the YAML configurations:
https://gist.github.com/ropnop/f306a8b46957dbf3a5199899b72be0b5
Expected behavior
Ambassador should not apply a Host mapping if the SNI does not match. If this is expected behavior and you can't securely mix mTLS and non-mTLS upstreams this should definitely be called out in the docs for those who need to rely on mTLS authentication.
Versions (please complete the following information):
Additional context
It is possible my configuration is bad/wrong, so if I have missed something in the docs please let me know
The text was updated successfully, but these errors were encountered: