DNS improvements

[alyssawilk]

HTTPS resource record (will improve QUIC usage)
Serve stale DNS results (improves performance and reliability)
DoT/DoH

[alyssawilk]

Also right now DNS has a hard coded refresh rate, and resolved entries are valid until that expires (ignoring TTL)

Once we have support for stale DNS, I think we can have DNS results be respected for min(ttl, refresh_rate)* and use the stale DNS support to make sure we don't end up stalling requests when we have a slightly-expired entry

*possibly actully max(30s, min(ttl, refresh_rate)) to make sure we don't DNS thrash

[andrewjjenkins]

resolved entries are valid until that expires (ignoring TTL)

This current behavior is a problem for some of my users. They were expecting that once the DNS result TTL expired, stale endpoints would not receive any new connections.

My team saw some other tickets like this one: #2691 and we're not sure what the desired behavior is.

Does the below describe the goal here?

If DNS resolution fails:

1. The stale endpoints would be used until TTL expired
2. Once TTL expires, the stale endpoints are removed (and Envoy would start delivering "no upstream" errors)
3. DNS resolution continues periodically (but as you say, not DNS-thrashing), as soon as it succeeds, the newly-discovered endpoints are used.

[alyssawilk]

Oh hey I totally failed to update this issue.

As of #18408 Envoy is closer to respecting TTL. DNS reresolve will be more aggressive based on the DNS TTL, The main DNS cache will remove stale entries but I don't think it propagates the non-null to null address resolution to worker threads to really ensure no additional traffic. I think it'd be a fairly simple change, which I think should be config guarded as I think it's pretty common to use stale entries if re-resolve fails.

[andrewjjenkins]

Thank you! Didn't notice that issue, sorry.

I think should be config guarded as I think it's pretty common to use stale entries if re-resolve fails.

Understood. That's the opposite of what this user wants in this particular case but I understand as a general principle the desire to allow the use of stale entries.