[Telemetry] mTLS support for TracePipeline OTLP output

[a-thaler]

Description
A security best practice is to use mTLS to establish a secure connection a a remote party. Especially when integrating with 3party systems this is a common practice which should be supported by a TracePipeline.

The OTLP output of the TracePipeline supports TLS in general and a header based authentication. However, it does not provide functionality for mutual TLS yet. Also the typical settings for verifying the server-side certificate are missing.

An API to support this scenarios could look like this, which adds a general tls section introducing the missing settings for TLS and mTLS.

The related files should be provided by secrets where the secret attributes need to get mounted as files into the gateway pod.

apiVersion: telemetry.kyma-project.io/v1alpha1
kind: TracePipeline
metadata:
  name: example
spec:
  output:
    otlp:
      endpoint:
        value: https://xxx
      headers:
        ...
      authentication:
        basic:
          ...
      # TLS settings
      tls:
        # if TLS is used, the optional CA that verifies the server certificate
        ca:
          valueFrom:
            secretKeyRef:
              key: x
              name: y
              namespace: z
        # if TLS is used, wether to skip verifying the certificate or not
        insecureSkipVerify: false
        # mTLS is used, the TLS cert to use for TLS required connections
        cert:
          valueFrom:
            secretKeyRef:
              key: x
              name: y
              namespace: z
        # mTLS is used, the TLS key to use for TLS required connections
        key:
          valueFrom:
            secretKeyRef:
              key: x
              name: y
              namespace: z

Criterias:

• Influencing the TLS Server-Side Certificate verification is possible by skipping it or providing a custom CA
• Providing certificate and key for mTLS is possible
• Needed Certs/keys can be provided by secret mappings
• Rotation of certs/keys is possible dynamically
• Documentation is updated
• Integration test assures functionality

Reasons
It should be possible to follow security best practices using the functionality

Attachments
supported settings of the otlpexporter: https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md

[chrkl]

mTLS for tracing has been implemented with kyma-project/telemetry-manager#347 and enabled for the telemetry component with #18035 and #18043. Support for logs and metrics will follow with separate PRs.