Support Retry Budget

[quraynai]

Open new MR to solve conflict. Previous MR #2585

Issue: istio/istio#27419

Istio: istio/istio#42232

[istio-policy-bot]

😊 Welcome @quraynai! This is either your first contribution to the Istio api repo, or it's been
awhile since you've been here.

You can learn more about the Istio working groups, code of conduct, and contributing guidelines
by referring to Contributing to Istio.

Thanks for contributing!

Courtesy of your friendly welcome wagon.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: quraynai (7c97616, 35f5435)

[istio-testing]

Hi @quraynai. Thanks for your PR.

I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[linsun]

/ok-to-test

[linsun]

@quraynai re your message on the initial PR, i think having istioctl analyze to report the mismatch (between VS and DR) is reasonable. I can't think of a better way to represent this either - I don't want to hold up your PR, unless someone else has a better way.

[rafatbiin]

hi @linsun & @quraynai, this is a bit unrelated, but how can you configure it for your service where the retry budget depends on something like active requests and active pending requests which is very low level and not easy to measure compared to the number of requests we're sending to upstream from a client. The same goes for min retry concurrency which we've less control over than retry per second(used by linkerd: https://linkerd.io/2.12/tasks/configuring-retries/#retry-budgets they probably implemented the idea of finagle version's of retry budget: https://finagle.github.io/blog/2016/02/08/retry-budgets/ which makes more sense to me). I've opened a discussion on the envoy repository but got no response from the maintainer unfortunately to discuss how we can measure and configure it for a production service.

[quraynai]

@rafatbiin I think number of request is harder to configure and measure compare to number of active connections.
Retries base on number of request we might need to know RPS and interval that we want to measure and it's hard to make retry budget work if we configure incorrectly.

[quraynai]

@howardjohn @linsun sorry for no activities with this MR for a while.
We made our forked version and POC for a while. Could you please help review this MR again?

[quraynai]

@howardjohn @linsun hi could you please help review on this PR?

[rafatbiin]

@rafatbiin I think number of request is harder to configure and measure compare to number of active connections. Retries base on number of request we might need to know RPS and interval that we want to measure and it's hard to make retry budget work if we configure incorrectly.

@quraynai sorry for the late reply. when you said number of connections, you meant active requests, right? how can we measure this in a reliable way to configure the retry budget percentage? because, from my experience, it's very low level and hard to measure. it can literally changes from 100 to 0 in a second and depending on when envoy is taking the measurement, you'll get a totally different retry budget percentage. it's specially true with service which doesn't have a lot of outgoing requests. because if you consider the average case, the retry budget percentage will be very high and when we'll have real retry storm, it'll actually bite us. do you have any suggestion on how to go about measuring this metrics?

[hzxuzhonghu]

This is a very advanced setting provided by envoy, we haven't had very critical requirements for it

[quraynai]

@hzxuzhonghu I understand about it's not critical to have this functionality at the moment from istio.
but it's kinda problem to configure it from EnvoyFilter. You can check more detail from this link
istio/istio#27419

It's a simple config that we tried and we don't issue about retry storm so far

[quraynai]

@rafatbiin maybe we can try with static number and readjust as global.
Understand tuning this value dynamically per service is really hard and might not worth it.

[quraynai]

@hzxuzhonghu @linsun @howardjohn hi could you please help on this?

[howardjohn]

I am concerned about this as retries are in VirtualService, so putting retry budget in DR is odd. It is to align with Envoy semantics only, not really designed from users perspective. Gateway-API is also looking into adding this in HTTPRoute; if we put it in DR it will be incompatible cc @mikemorris.

That being said, I don't know a way to put it in route without envoy changes.

[quraynai]

@howardjohn I understand your concern. we will look into another option

[ericdbishop]

I am concerned about this as retries are in VirtualService, so putting retry budget in DR is odd. It is to align with Envoy semantics only, not really designed from users perspective. Gateway-API is also looking into adding this in HTTPRoute; if we put it in DR it will be incompatible cc @mikemorris.

That being said, I don't know a way to put it in route without envoy changes.

@howardjohn We are interested in having access to the retry_budget circuit breaker threshold via the DR api, as max_retries is a static value that requires tuning for every service within the Mesh, making it difficult for us to update when services begin receiving more traffic. In comparison, retry_budget configurations require no major tuning since the retry value is relative, and it also cuts new requests earlier than using max_retries, performing better in our tests. But using retry_budget via EnvoyFilter's is extremely difficult to maintain across our services. Do you think there is any path forward to exposing this within DestinationRule we could work on? Thanks.

[mikemorris]

@ericdbishop We're starting to look at implementing retries (including retry budgets) upstream in Gateway API in kubernetes-sigs/gateway-api#1731

One thing I'd like to get more clarity on is where/how this should be implemented - my initial thought was as a field on HTTPRoute (similar to Timeouts), but I've seen a few mentions of DestinationRule or client-side config which I'd like to better understand the implications for (or any Envoy implementation constraints) as it would relate to both Ambient mesh without client sidecars and reconciling this functionality with other Gateway API implementations such as Linkerd's retry budgets.