-
Notifications
You must be signed in to change notification settings - Fork 715
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Skip Rollback #1186
Comments
Flagger by default does not roll back the canary release, it just routes all traffic to the primary if an analysis fails. Ref: https://docs.flagger.app/tutorials/istio-progressive-delivery#automated-rollback |
For Flagger, rollback means scaling to zero the canary deployment and routing all traffic back to primary. Currently there is no way to tell Flagger to freeze on failures. I guess you would want the traffic to go to the canary even if it's in a failed state, which contradicts with what Flagger was design to do: route all traffic to the previous version as soon as errors are detected. |
Thanks for you responses @aryan9600 + @stefanprodan ! I was suspecting that flagger wouldn't support this use case, but good to have it confirmed.
That way it would be guaranteed that once a client hit a new version it will never end up at an older version again. I think this behavior could be configured by a flag in the analysis (like I had a peak into the code already (props btw, it's really clean and easy to dive in) and I think it should be doable. But of course I'd like to check this feature with the community first, before I would work on a PR. Any thoughts? |
As far as I know, sticky sessions do not work with weighted traffic split no matter which service mesh or ingress controller you're using. How are you doing this type of routing today without Flagger? |
Sorry, I phrased it incorrectly. We're not using weighted traffic, we're using A/B-testing with istio leveraging header bases routing. Our services and databases are multitenant, every customer is identified by a header. We intend to use a group of "progressive" customers and expose them to the canaries. For the forementioned reasons we don't want them to be fall back on previous versions in case an analysis fails. |
I see no need for a |
That was my first idea as well @stefanprodan . But when I hacked that in yesterday I noticed: When there is a frozen |
I created a protoype for the feature, you can look at the draft PR here: |
ah yeah and you were right, a |
Hey!
We're currently looking into flagger to drive our staged rollout and the following question popped up:
Can flagger be configured to not do a rollback even when the analysis failed?
Our use case:
We have applications which might perform non-backwards-compatible database migrations when they're updated. Then it might be more desirable to leave a app in a failed state instead of rolling it back. The consequences of a rollback might be even worse.
So we were wondering if flagger could be configured in a way so that it would just kind of "freeze" as soon as the analysis fails. Of course it should not promote further either. Alerts and notifications should be sent out so that the responsible team could look at the problem and then deploy a new version with a fix which would then replace the "frozen" canary and the whole cycle starts again.
I hope that was clear.
Thanks in advance for any hints!
The text was updated successfully, but these errors were encountered: