[Feature] detect platform options when installing addons

[dhiguero]

Is your feature request related to a problem? Please describe.

While installing the flux addon, we came up with an issue that may be solved by introducing some level of parametrization. Currently, it is not possible to pass parameters when installing an addon which may result in inconsistent installations. As example use cases, consider:

• Multi-tenant installations: Some addons are by default not multi-tenant (e.g., fluxcd with the existing default configuration), while kubevela can be configured to be. In this situations installing an addon may introduce potential security risk enabling privilege escalation if not checked manually.
• Custom configurations: Some addons may provide extra parameters to alter their behavior. For example, installing the addon with debug flags enabled, a given certificate, or with a particular type of persistent volume.

Describe the solution you'd like

A potential solution could be to provide a mechanism to set parameters when installating addons, such as:

vela addon enable my-addon --set param=value --set another_param=value

And addionally, there could be a set of platform parameters that are configured when installing kubevela, that could be always forwarded to the addons, so the previous command would be equivalent to:

vela addon enable my-addon \
# Kubevela installation parameters
--set multitenant=true \ 
# User provided parameters
--set param=value --set another_param=value

Describe alternatives you've considered

In our case, we installed the addon manually without using vela addon enable to adapt its deployment to our needs, which requires knowledge of the addon internals, and its deployment on kubernetes.

Additional context

Another complementary approach could be to make the addon metadata include which platform parameters are supported:

name: example
version: 1.0.0
description: Example adddon.
...

# List of runtime configuration options that are supported by the addon. This will enable failing if a user tries to install an addon with an unsupported option.
addonRuntimeOptions:
  - multitenant
  - customCertificate
  - ...

Update

As commented by @charlie0129, addons can be already parametrized depending on how they are written. What remains to be addressed is the consistency of the kubevela install when new addons are installed in areas such as: security, multi-tenancy, etc.

• (11th August) Change title from [Feature] Addon parametrization to "[Feature] detect platform options when installing addons"

[charlie0129]

Would you take a look at addon parameters? From my first glance, it serves the same purpose as you suggested. Let me know if it suits your needs or is there anything needs to be improved.

For example, chartmuseum addon has several parameters that the user can customize:

vela addon enable chartmuseum debug=true persistence.enabled=true persistence.pvcName="myname"

[Somefive]

Unlike helm, vela addon enable currently does not use --set for setting parameters for now. The parametrization is implemented through cue rendering process. Therefore, not all addons support parametrization by now.

For example, the addon of fluxcd uses native YAML files, which means it does not support parametrization.
https://github.com/kubevela/catalog/tree/master/addons/fluxcd/resources

But, as @charlie0129 said, addons like chartmuseum uses CUE files for rendering, which allows parametrization.
https://github.com/kubevela/catalog/tree/master/addons/chartmuseum/resources

There are also further design for addon rendering, like making the rendering process, as a whole, as CUE rendering, which is still in development now. With this function, the parameterized rendering will not be limited to components but also works for policies and workflow steps.
#4401

[dhiguero]

Thanks @charlie0129, @Somefive for the extra information. I was not aware of the current parametrization capabilities for addons. With this I think the first problem of being able to pass parameters to addons is already solved.

I think what remains to be addressed is how to ensure the consistency of the installation with topics such as multi-tenancy, security, etc. once addons are installed.

[wonderflow]

Hi @dhiguero , can you share more about the remain topic, what affect do you want to provide for the addon user:

how to ensure the consistency of the installation with topics such as multi-tenancy, security, etc. once addons are installed.

[dhiguero]

Hi @wonderflow, I will to try rephrase the issue. The main problem I see with the current approach if we do not add any consistency mechanism is that by adding a new addon we may be conceptually breaking part of the installation. To illustrate this with an example, let us assume that we have installed Kubevela with multi-tenant, RBAC compliant options. As such, I would like to ensure that if I add a new addon later on, the installation remains with the previous multi-tenant, rbac compliant guarantees (i.e., a user can only create entities that are authorized to their assigned roles). If that is not possible, I would expect either the installation of the addon to fail, or to show a warning message so that the cluster administrator knows that the installation may break.

For example, with the existing addons and options, if we have a multi-tenant, secure Kubevela installation, and we install the default flux addon, that will conceptually break the installation since through the flux Helm extensions it will be possible to install cluster-wide resources, and creating kubernetes entities will no longer be restricted by the user role, but will take the flux admin one. This will result in the previous multi-tenant, rbac considerations not to be applied consistently to the all OAM applications. The same happens with other addons such as the terraform one, since the account credentials are stored in the kubevela addon namespace, not the one being used by the user, which could potentially translate in credentials leakage among cluster users.

To prevent that, I think we could include a method for the addon to define its capabilities in such a way that the cluster adminsitrator understand if those can be added to the existing installation.

Addon defining that it can work in a multi-tenant installation

name: example
version: 1.0.0
description: Example adddon.
...

# List of runtime configuration options that are supported by the addon. This will enable failing if a user tries to install an addon with an unsupported option.
addonRuntimeOptions:
  - multitenant

With this approach, if we store the options that have been used to install kubevela, it will be possible for the vela command to warn the user if the addon does not support any of the options used in the installation:

$ vela addon enable non-multitenan-addon
[WARN] This addon does not support the existing type of installation. Use --force if you can to proceed with the installation.
Conflicting options:
Multitenant

[wonderflow]

If I understand correctly, the addon with the multitenant option can only be installed by platform admin of kubevela, while others can be installed by any of the users, right? @dhiguero

[dhiguero]

I think that for a multitenant installation only admins should be able to install addons as creating cluster-wide entities and/or operators will be restricted.

[wonderflow]

@dhiguero According to the addonRuntimeOptions , do you mean we should also add these options on vela controller side to let vela addon command to know how to judge the addon metadata?

The process should be like:

1. Install kubevela core with multitenant runtime options, make it security enough.
2. Vela CLI will check runtime options when installing Addon, report warning.
3. webhook of kubevela-core will prevent the addon from installing when runtimeOption is not match.

Does the process work in your scenario?

BTW, please change the issue title as it will cause misleading to other community users.

[dhiguero]

I think that will work from the point of view of Kubevela core. From the point of view of the addon developer, i think we should make this type of platform options available to be consumed when the addon template is being processed. This could follow the existing approach that is used in component definitions, where the template has access through the context to the name of the application, name of the component, etc.

In a similar way, the addon template could have access to a platformContext to read metadata information about the platform (e.g., kubevela version, options installed, etc). With this information the addon template could automatically detect the kubevela installation options and adapt the addon installation to match those. This will improve consistency on the installation as addons will automatically detect the existing options and configure themselves. From the point of view of the users, this will also improve usability as the addons can automatically detect some of the installation options, and the users do not need to remember all the ones that should be set as parameters when executing vela addon enable myaddon.

[barnettZQG]

I think that will work from the point of view of Kubevela core. From the point of view of the addon developer, i think we should make this type of platform options available to be consumed when the addon template is being processed. This could follow the existing approach that is used in component definitions, where the template has access through the context to the name of the application, name of the component, etc.

In a similar way, the addon template could have access to a platformContext to read metadata information about the platform (e.g., kubevela version, options installed, etc). With this information the addon template could automatically detect the kubevela installation options and adapt the addon installation to match those. This will improve consistency on the installation as addons will automatically detect the existing options and configure themselves. From the point of view of the users, this will also improve usability as the addons can automatically detect some of the installation options, and the users do not need to remember all the ones that should be set as parameters when executing vela addon enable myaddon.

I agree with this proposal. We should add some useful information to the context, which currently only includes the metadata. There are some options:

1. Platform info(version, architecture ...)
2. Cluster info(version...)