Istio management - Design discussion

[lucasponce]

Hi everyone,

A goal of Kiali is to simplify the management of Istio Service Mesh. Kiali helps users to configure, update and validate Istio configuration.

Users can create and edit individual Istio configuration resources [1] or generate full end-to-end scenarios based on wizards [2] (scenarios that may be implemented using several traffic and security resources - VirtualServices, DestinationRules, Gateways, PeerRequests -).

This area needs to improve to cover more scenarios requested by community like:

• Use wizards in a more generic way, not only attached with the Service detail (i.e. be able to add an scenario from the graph).
• Open any existing Istio configuration from the Wizards (today, wizards are used for scenarios started from Kiali, existing Istio resources are edited only via yaml editor).
• Enable actions on list pages.
• Add schema support on yaml editor.
• Introduce Istio registry on Wizards, to expand the destination that can be used in an scenario, this will support more traffic scenarios involving external and federated services.

Please, join us in these discussions :)

[1] https://kiali.io/documentation/latest/features/#_configuration_and_validation_features
[2] https://kiali.io/documentation/latest/tutorial/

[abonas]

can you elaborate on what is Istio registry? I can't find anywhere

[lucasponce]

The Istio control plane maintains an internal service registry [1] to resolve destinations available from the data plane.

So, today, Wizards have some limited scoped to the workloads linked with a Kubernetes service, but using the Istio registry will allow Kiali to create rich configurations (i.e. involving ServiceEntries or selecting destinations from cross-namespace scenarios).

[1] https://istio.io/latest/docs/reference/glossary/#service-registry

[xeviknal]

Better support of canary upgrades

Canary upgrades are done in three steps:

• Installing the upgraded control plane
• Moving the data plane under the new control plane
• removing the old control plane

Kiali can definitely ease some of the jobs of managing an upgrade.

For example in the step 2:

1. Adding a new action at namespace level at overview page to make the transition (change the label accordingly and/or trigger/warn a full roll-out)
2. Let the user filter namespaces by revision.
3. Let the user filter workloads by revision (it can be done now via labels filters though).
4. Let the user change which Control Plane kiali is targeting (changing config map, istiod deployment name and sidecar injector cm name).

At some point kiali would visualize three different items: non-mesh items, items belonging to a Control Plane 1, items belonging to a Control Plane 2.

We should be able to help users to navigate the mesh through the whole versions.

If there is only one Control Plane, probably don't show any sign of help. Not to confuse people.

I believe that in step 1 and 3, kiali can't do much because is more istio installation/removing.

[jmazzitelli]

I don't know if it helps, but multiple Kiali servers (with independent configuration) can now be installed in the same namespace. So its possible to have one Kiali point to the old Istio and the other Kiali point to the new Istio - even if they are all installed in the same namespace.

This doesn't help if you want a single Kiali UI point to different Istios.

[lucasponce]

Perhaps canary deployment falls more into the Deployment model #4081
Where it may affect how having mutiple control planes may affect to the mesh topology in general (and Kiali in particular).

But +1 good topic

[leandroberetta]

+1 to the idea of wizards being invoked from everywhere, especially in the graph where you are getting a lot of insights of your mesh, we can offer things to do depending on what workload/edge you're looking at.

Another interesting concept is to introduce "quick actions" in the graph, for example: If I'm seeing a connection from service A to service B and that it's not supposed to happen, a quick action could be "block this connection" generating all the configuration automatically.

[jmazzitelli]

Has anyone taking a poll of our users to see who uses the wizards? I don't know, but I would think they are only used by Developers and people going through demos and tutorials. I highly doubt these wizards are going to be used in production for true "management" of an Istio mesh - most mesh deployments are going to be pushed out using things like CI automation tools or Ansible or something like that. People aren't going to be messing with production systems by clicking buttons in a Kiali wizard (especially when Kiali gives no indication of what the resources are that it is going to create). So these wizards may provide "management" of Istio in a development or tutorial environment, I doubt it is used that way in a production (or even QE) environment.

In other words, I don't think we should spend a lot of time and manpower building out these wizards - I don't think they impact our users much (but, again, since we don't have any data, I may be wrong even though I think I'm right :-)

[leandroberetta]

A survey would be very helpful, I think you're right when you're saying that a wizard wouldn't be used in production, all the production resources will be eventually created with a GitOps strategy I guess, but what it is also true is that this kind of tools are for developers and wizards allows developers (mainly newcomers to Istio) to quickly create what they want to do, test it on a non prod environment and finally promote them to production storing them in a Git repository for example.

Totally agree that wizards don't have to cover all the cases (that would require a great effort) but it sounds like a good investment to have wizards for high level and common scenarios to make the adoption easier (today we have wizards, but I think they need to be a little easier to understand, mainly speaking of the terminology, for this newcomer user).

[lucasponce]

I highly doubt these wizards are going to be used in production for true "management" of an Istio mesh - most mesh deployments are going to be pushed out using things like CI automation tools or Ansible or something like that.

Wizards have never been designed for "production" usage.
Those are designed to map a high level scenario into the specific details of Istio resources intially for educational purposes and help Istio users to use traffic management and securitty features.

Even if they help to "Developers" it's a goal, once created users typically use the YAML editor and validations.

But Kiali needs a way to generate configuration on key scenarios in a easy way as it's happening in other close areas [1] as networking.

spend a lot of time and manpower

I'd say that is best to have better impact, there are features that have a very good feedback (as combine telemetry with AuthorizationPolicies) and other forms that I agree that don't add too much value (some of the security forms are complex and don't help in high level scenarios).

The proposal of this design is to focus on the key ones: import/export about federated services, apply security rules for a given context, or simplify the Envoy configuration on given use cases.

[1] https://editor.cilium.io/?id=xpqGP3ct6oBJIXFf

[xeviknal]

we don't have any data, I may be wrong

Do we have data from other areas? We don't even know which appenders are worth and which aren't, whether all the metrics are useful or people might be using grafana instead. And so on. Same for tracing.

If we think data of issues / comments / talks, I'd say that @lucasponce was called from different places for showing the tutorial which uses a lot the wizzards. Plus, I remember a tweet where people from istio asked from more info about the Authz policy action.

There was an effort to create a survey to get more insight of kiali usage. It didn't go so far. The issue to make the survey more visible in both kiali.io and the console never made it to the backlog. I'd push to get more data from users (for example, capture usage metrics and give users a one-click solution to share them with us).

[xeviknal]

I like the idea of applying more transparency to the wizards. The idea of having a preview step before committing the changes where users can either check the integrity of the generated config, modify them to better approach their solution or even download it to commit it to their CI/CD pipeline.

I envision this like:

• Wizard form is show as it is today.
• Wizard has one more button to submit: 'Preview' next to the 'Submit' button.
• When the 'Preview' button is clicked, the kiali shows the ACEditor with the generated yaml.
• The editor lets users to:
  • edit the yaml
  • copy the yaml
  • showing validation (potentialy)
  • download the yaml file
• Once the user is done, three options are available: go back, quit the wizard or submit the changes..

This idea is aligned with the fact of that industry is using GitOps approach for managing all the configs of infrastructure. Some people might have a pre-production environment with the same exact config as prod which might use the wizard there. Otherwise, it seems a bit dangerous for production.
On the other side, wizards are kind of a blind choice. You submit them and you don't know the outcome: what object did it create? which types? If users already have plenty of istio configs in the namespace it might be difficult from them to detect which are the new ones created.

[lucasponce]

Totally +1.
Having a high level form to define the scenario and preview it for confirmation to create, modify and/or use the generated resources into other scenarios is a very good idea.

Also, it can go further, and perhaps the high level form can be used only for "creation" step, and delegate all "update" operations to the editors (perhaps combining editor and wizards together). With that change, forms can be focused to "added value" scenarios without spending too much effort in the "update" operation (which is the most challenge one).

[jshaughn]

Also big +1 to this idea, and I think tying/marketing it specifically for GitOps is an excellent idea.

[abonas]

I also +1 this. I would suggest also a "copy" yaml which is somewhat equivalent to download, but lets you copy the content without the download operation.
IIRC there's a similar UI in openshift with dual wizard/yaml preview, but I can't remember which screen it was. @andrew-ronaldson perhaps you remember?

[andrew-ronaldson]

We are working on designs now for a form/YAML split view but it is not implemented in console yet. Currently the behaviour is to toggle between the two views. We will hopefully have the first Split view use case with Network policies in either 4.9 or 4.10.

[sjbylo]

I am adding this comment in the hope it is useful. It's the first time I'm giving feedback, so please forgive me if such topics have already been discussed.

First, Kiali is excellent. It has a lot of killer features, thank you so much to the community for creating it!!! :)

I have been using Kiali mainly for learning and demo purposes for a while now and I can confirm 100% that Kiali's very useful to help with this and esp. generating yaml content, which I can later edit and refine. The wizards are great for this, although it took me a while to understand them. Tooltips in the wizards might help here or even a "new user" mode with some popups to give guidance/examples.

My biggest problem with Kiali so far was that, whilst I knew they were available, the wizards were very hard to find. When a new user has finally got to the point where the graph is visible and had that wonderful "ah-ha" moment, it then takes 4-5 exact clicks to find the wizards (via the Services Menu). Honestly, I never found the wizards until I went through the tutorial probably because I was always fixated on the excellent Graph. I would suggest adding a simple way to get directly to the wizards from the nodes in the graph.

I would prefer to have the graph "always in view", front and centre. I want to make config changes, view metrics etc, but never leave the graph view. The graph view can be temporarily covered by a popup, but still, I know the graph view is always there and I can see any changes to it.  Going back and forth between different views/menu items is something I'd like to do less of.

Not sure how feasible this is, but It would be great if, along with viewing tracing information, I could also view & compare all the service logs of all traced services at the exact time window of the requests, as they flow through each service. That would be very useful to debug issues with a specific request flowing through the mesh.

Lastly, if a user is running Istio/Kiali in production AND they are not using some form of GitOps, the ability to make quick changes to a production mesh might be very useful. If there is a problem with a prod workload, it might be necessary to temporarily add (or modify) e.g. a CB into the Istio config and NOT have to wait for a developer to do this. I would not assume that all users have a super sophisticated ci/cd/gitops etc in place, or even have the insight to add the CB in advance! And to add on, the people I've spoken to who have seen Istio/Kiali say that they expect the Graph and all the observability features to provide them with the biggest value, be it in non-prod or in prod.

[jshaughn]

@sjbylo Thanks so much for taking the time to give us this great feedback and to contribute to the discussion.

[abonas]

+1, thank you very much for your feedback. @jshaughn @lucasponce is there an existing gh issue to add opening wizards from topology? from a brief search I did not find

[lucasponce]

Thanks Steve :).
In the past there was some brainstorming around that idea but I don't think there is a fresh issue about that.

This discussion will act as a "source" for new issues, once all the discussions are consolidated, we will create proper "Epics" that can be added into the backlog.