question: How to implement retries for Dynamic Backend?

[yury-kozlov]

We intend use dynamic backends in our routes (defined in Route Groups CRD), but couldn't figure out the proper way to handle retries: for example if dynamic backend returns invalid response (e.g. timeout) how can we tell skipper to give it another try?

So far, we found only two alternatives:

1. use Load Balancer Backend instead of dynamic backend which automatically provides retry infra. The downside, if I understand it right, is that it will be impossible to dynamically resolve backend addresses from code.
2. use CustomHttpRoundTripperWrap + implement our own retry mechanism.

[aryszka]

thanks for the question. If I understand correctly what you described, this is not supported out-of-the-box. Skipper only does a single retry for failed TCP connects, but doesn't do retries for application layer failures, e.g. for 500s received from the backend. This is something that is considered to be a decision of the client. This applies to the LB backend.

your proposed alternative approach (2) of using a custom roundtripper may work. However, my personal opinion is that delegating this kind of functionality to a roundtripper in Skipper can be hard to address in a generic way, because I expect too many edge cases, like when and when not to retry, that depend on application layer considerations. It may not be in the best place in a roundtripper. If you decide to experiment with this options, please let us know how it works out.

[AlexanderYastrebov]

Related feature request #1473