[Telemetry Operator] Finalize fluent-bit managed by operator

[a-thaler]

Description
As part of #15955 the kyma fluent-bit instances are manageable by code already but the feature is not activated yet and will be still realized via a helm chart. Finalize the efforts by testing the migration fully and fixing leftovers.

Criterias:

• All resources of the old fluent-bit sub-chart are either managed by code (new telemetry-operator functionality), are moved to the umbrella telemetry chart (and will be managed by old logic) or are deleted (automated or manual migration script?)
  • Potential resources of interest are: Grafana Dashboards, ServiceMonitors, PeerAuthentication
  • Agreement for ServiceAccount is to also manage it by the operator with least privileges (remember to limit the namespace). It includes the ServiceAccount, a Role and a RoleBinding
• Feature flag got removed
• temporary integration test got merged into regular test suite
• Old fluent-bit sub-chart does not exist anymore
• Migration/Upgrade from helm chart approach to managed by code approach is tested
• Try to verify that automated upgrade tests will not fail
• add missing test cases
  • no logpipeline reveals in no fluentbit
  • deletion of logpipelines will remove fluentbit when reaching an amount pipelines of 0

Reasons

Attachments

[rakesh-garimella]

Following tasks have been performed:

• Removed telemetry alpha job: remove telemetry alpha job test-infra#6861
• Enable managed fluent-bit and removed the feature flag
• Deleted fluent bit charts
• Moved servicemonitor, peerauthentication and dashboards in flunet-bit folder
• Changed the logic to have fluent bit daemonset created only after the checksum is calculated.
• Fixed deletion logic
• Restricted the rights of telemetry-operator

The automated tests will be updated via a separated PR

[rakesh-garimella]

PR for automated tests have been added here: https://github.com/kyma-project/kyma/pull/16726/files

[a-thaler]

production profile is ignored, will re-open till it is respected

[a-thaler]

The containerPort 2021 is exposed by both containers, fluentbit and directory-size-exporter. WHich one will be used? I'm very sure that fluentbit should expose only the 2020 port and that this must be a mistake, please verify that

[a-thaler]

The generated luascript is broken.
The line local mapped_key = string.gsub(key, "[%/%.]", "_") contains escaping characters which breaks fluentbit startup local mapped_key = string.gsub(key, \"[%/%.]\", \"_\")

[a-thaler]

Mentioned issues are addressed