Support debugging and settings propagations of reconciliations: add option to enable debug mode during runtime per Module CR

[tobiscr]

Description

The Kyma operator has to support debugging possibilities to enable analysis by on-call engineers and SREs during runtime.

It has to be possible to enable the debug mode for a whole Kyma instance or for particular Kyma modules. It's recommended to make the configuration quite simple, e.g by offering a debug field in the CR which accepts a boolean value.

The debug: true field should enable the debug mode. If the field is false, the debug mode is disabled.

AC:

• Labels are properly filled with necessary metadata to enable tracability (see comment below)
• The logger-instance of the module operator (respectively the logger used by operator responsible for the particular Kyma module) has to use the debug log-level for any printed log message. This is only possible if the module operator supports parsing the passed labels.
• If debug mode is disabled, the default log-level (probably info) has to be used by the logger-instance
• The debug mode can be configured Kyma wide by editing the Kyma CR. This will enable / disable the debug mode for all Kyma modules listed in the Kyma CR.
• The debug mode can be configured per Kyma module by editing the particular module CR. If the debug mode of the Kyma CR changes, the value of the Kyma CR has precedence and overwrite the module specific value.

[tobiscr]

We agreed to use following labels to track correlated IDs per resource:

• operator.kyma-project.io/managed-by (=Kyma oeprator, can be used for A/B testing)
• operator.kyma-project.io/controller-name (=Manifest operator, can be used for A/B testing)
• operator.kyma-project.io/kyma-name (name of Kyma installation, written to context)
• operator.kyma-project.io/module-name (name of module retrieved from OCI descriptor)
• operator.kyma-project.io/channel (release channel which includes the component and was parsed by Kyma operator)
• operator.kyma-project.io/profile (profile which included the Kyma component) (scoped out)

[jakobmoellerdev]

if we scope out profile like originally accounted for, it may make sense to instead propagate labels on a more generic scope.

[jakobmoellerdev]

@pbochynski this is what is currently planned for propagation of labels. Instead of hard-coded labels we can think of dynamic label configurations instead to make it possible to extend later on.

[pbochynski]

I had some time to think about the use cases (profile, debug), and to be honest I am not sure if it is the right way to do it. It makes module operators dependent on convention (not API). Also propagating flags to all modules is not the smartest solution (enabling debug level for all the modules at once). I think we should have the explicit configuration in the spec of module CR instead (resources, debug). We alredy have log configuration in the spec in our first module:
https://github.com/kyma-project/keda-manager
If we add label it will be confusing.

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity.
Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[jakobmoellerdev]

While this is fine for me I suggest we then leave this feature until we find a better way of propagation. Otherwise there will be no way to define a unified debugging spec field.

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity.
Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity.
Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale