High Availability for Istio Services

[njfix6]

Describe the feature request
It is still not clear to me whether it is safe to run citadel, galley, and the sidecar injector with multiple pods? Also wondering why they don't have an HPA attached to them?

I am thinking of adding the following flags to the helm install, but not sure if it Is it safe:

--set sidecarInjectorWebhook.replicaCount=5 \
--set galley.replicaCount=5 \
--set security.replicaCount=5 \

I would hope this would raise sidecarInjector, galley, and citadel to each have 5 pods and still run with the same functionality as before.

Describe alternatives you've considered

Additional context

[howardjohn]

I believe there was a time when these did not handle multiple replicas, but they now do.

@myidpt and @ayj can confirm?

[ayj]

Galley and sidecar injector should run fine with multiple replicas. HPA is less critical given the amount of load these typically get.

[rolandkool]

What about citadel?

[howardjohn]

In previous versions citadel had issues with multiple replicas, I am fairly certain this is handled now

[padzikm]

With new 1.5 release I'd like to ask this question once again - is it safe to deploy replicated control plane istio in one cluster? my goal is to have istio as high availability service in one shared cluster deployed to multiple zones. As I understand currently there is one monolithic's control plane service as opposed to multiple services - was this new combined control plane tested in multizone replicated in single cluster scenario as I described? Are there any istio's components that cannot be deployed with multiple replicas?

[howardjohn]

Yes, with 1.5 you can deploy multiple replicas. I routinely tested 100+ replicas of the new Istiod in a single cluster

…

On Tue, Mar 10, 2020 at 5:58 AM padzikm ***@***.***> wrote: With new 1.5 release I'd like to ask this question once again - is it safe to deploy replicated control plane istio in one cluster? my goal is to have istio as high availability service in one shared cluster deployed to multiple zones. As I understand currently there is one monolithic's control plane service as opposed to multiple services - was this new combined control plane tested in multizone replicated in single cluster scenario as I described? Are there any components that cannot be deployed with multiple replicas? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#18565?email_source=notifications&email_token=AAEYGXIX3TL3XYMCIECPUKDRGY2OPA5CNFSM4JH7GYAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOLJNEY#issuecomment-597071507>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEYGXL7Z3QXWV5SFO3KFH3RGY2OPANCNFSM4JH7GYAA> .

[padzikm]

Great! Thank you!

[padzikm]

Ok, but how can I configure istiod's deployment to multiple replicas? In docs there are old configuration guidelines telling about configuring each component separately - https://istio.io/docs/setup/install/istioctl/#customizing-the-configuration - but in generated manifest there is only one control plane's component - istiod, that can be scaled (not counting prometheus, etc). However in profile dump there are still different replicaCounts settings for different components (galley, sidecarInjector, etc). I don't see any configuration for istiod in docs, so should I manually adjust istiod's deployment in generated manifest and then apply it, or do it in old way setting each component via command line parameters or yaml as in previous versions? Moreover should (and safely can be) ingressgateway be also scaled (it is not included in istiod's architecture overview and is deployed separately)?

[padzikm]

Any info about configuring istiod's deployment?

[jpapejr]

I've not tested this but I'd think this (scaling up istiod) would be as simple as adjusting the replicas of the istiod deployment, no? Not sure I understand the part about the ingress gateways as those scale on demand don't they?

[padzikm]

The problem is there are no installation options regarding istiod - only that it is enabled. No replicaCount, no nothing. In docs there is no mention how to configure that. Control plane is now monolitic app and as such one deployment unit as I understand, but there are options to configure replicaCounts of istiod's components like pilot or mixer, which is strange - how can you configure scaling of part of monolithic app? So the only option I'm aware of now is to generate yaml manifest, which outputs istiod's deployment resource and manually configure it, but it is error prone, and my question is how to set that (and other options like hpa) without manually tampering with deployment files. I also would like to set replicaCount to ingressgateway, but also haven't found any option for doing that apart from manually correcting deplyment in yaml.

[jpapejr]

Right now, it seems that modifying the deployment manifest(s) is the only way to accomplish what you're asking. It's certainly a gap in both documentation and in [clear] configuration. There's already a related issue open for docs.

[howardjohn]

The `pilot` component will impact istiod. It needs to be better documented/changed though

…

On Tue, Mar 17, 2020 at 8:23 AM John Pape ***@***.***> wrote: Right now, it seems that modifying the deployment manifest(s) is the only way to accomplish what you're asking. It's certainly a gap in both documentation and in [clear] configuration. There's already a related issue <istio/istio.io#6303> open for docs. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#18565 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEYGXLBLPROSPPWV3IISDDRH6IVXANCNFSM4JH7GYAA> .

[neumanndaniel]

Here is an example for Istio 1.5. As @howardjohn mentioned the pilot component impacts the istiod configuration.

-> https://github.com/neumanndaniel/kubernetes/blob/master/istio/istio-1.5.yaml

[howardjohn]

All components now have a PBD and HPA and are fully configurable, so I think this is fixed