diff --git a/CHANGELOG.md b/CHANGELOG.md index c1ec1e47da..626b85abd3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed -- feat: add metadata.metrics.config.extraProcessors [#2724] +- feat(metrics): add `sumologic.metrics.otelcol.extraProcessors` [#2724] [#2780] - feat: add otellogs.additionDaemonSets configuration [#2750] - chore: upgrade Fluentd to v1.15.3-sumo-0 [#2745] - This also upgrades Ruby from `v2.7` to `v3.1` and some other dependencies. @@ -37,6 +37,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 [#2766]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2766 [#2754]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2754 [#2771]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2771 +[#2780]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2780 [v1.15.3-sumo-0]: https://github.com/SumoLogic/sumologic-kubernetes-fluentd/releases/tag/v1.15.3-sumo-0 [Unreleased]: https://github.com/SumoLogic/sumologic-kubernetes-collection/compare/v3.0.0-beta.0...main diff --git a/deploy/helm/sumologic/README.md b/deploy/helm/sumologic/README.md index b1374a9a8c..eb72fbc7a1 100644 --- a/deploy/helm/sumologic/README.md +++ b/deploy/helm/sumologic/README.md @@ -78,6 +78,7 @@ The following table lists the configurable parameters of the Sumo Logic chart an | `sumologic.logs.metadata.provider` | Set provider to use for logs forwarding and metadata enrichment. Can be either otelcol or fluentd. | `otelcol` | | `sumologic.metrics.enabled` | Set the enabled flag to false for disabling metrics ingestion altogether. | `true` | | `sumologic.metrics.metadata.provider` | Set provider to use for metrics forwarding and metadata enrichment. Can be either otelcol or fluentd. | `otelcol` | +| `sumologic.metrics.otelcol.extraProcessors` | Extra processors configuration for metrics pipeline. See [/docs/collecting-application-metrics.md#metrics-modifications](/docs/collecting-application-metrics.md#metrics-modifications) for more information. | `[]` | | `sumologic.metrics.remoteWriteProxy.enabled` | Enable a load balancing proxy for Prometheus remote writes. [See docs/best-practices.md for more information.](/docs/best-practices.md) | `true` | | `sumologic.metrics.remoteWriteProxy.config.clientBodyBufferSize` | See the [nginx documentation](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size). Increase if you've also increased samples per send in Prometheus remote write. | `64k` | | `sumologic.metrics.remoteWriteProxy.config.workerCountAutotune` | This feature autodetects how much CPU is assigned to the nginx instance and setsthe right amount of workers based on that. Disable to use the default of 8 workers. | `true` | @@ -490,7 +491,6 @@ The following table lists the configurable parameters of the Sumo Logic chart an | `metadata.metrics.logLevel` | Flag to control logging level for OpenTelemetry Collector for metrics. Can be `debug`, `info`, `warn`, `error`, `dpanic`, `panic`, `fatal`. | `info` | | `metadata.metrics.config.merge` | Configuration for metrics metadata otelcol, merged with defaults. See also https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/configuration.md. | {} | | `metadata.metrics.config.override` | Configuration for metrics metadata otelcol, replaces defaults.See also https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/configuration.md. | {} | -| `metadata.metrics.config.extraProcessors` | Extra processors configuration for metrics pipeline. See [/docs/collecting-application-metrics.md#metrics-modifications](/docs/collecting-application-metrics.md#metrics-modifications) for more information. | `[]` | | `metadata.metrics.statefulset.containers.otelcol.startupProbe` | Startup probe configuration for metrics otelcol container. | `{"periodSeconds": 3, "failureThreshold": 60}` | | `metadata.metrics.statefulset.nodeSelector` | Node selector for metrics metadata enrichment (otelcol) statefulset. [See docs/best-practices.md for more information.](/docs/best-practices.md) | `{}` | | `metadata.metrics.statefulset.tolerations` | Tolerations for metrics metadata enrichment (otelcol) statefulset. | `[]` | diff --git a/deploy/helm/sumologic/conf/metrics/otelcol/config.yaml b/deploy/helm/sumologic/conf/metrics/otelcol/config.yaml index 7a2244f508..bd067dbeeb 100644 --- a/deploy/helm/sumologic/conf/metrics/otelcol/config.yaml +++ b/deploy/helm/sumologic/conf/metrics/otelcol/config.yaml @@ -299,8 +299,8 @@ processors: ## ref: https://github.com/SumoLogic/sumologic-otel-collector/tree/main/pkg/processor/sourceprocessor source: collector: {{ .Values.sumologic.collectorName | default .Values.sumologic.clusterName | quote }} -{{- if .Values.metadata.metrics.config.extraProcessors }} -{{- range $processor := .Values.metadata.metrics.config.extraProcessors }} +{{- if .Values.sumologic.metrics.otelcol.extraProcessors }} +{{- range $processor := .Values.sumologic.metrics.otelcol.extraProcessors }} {{ toYaml $processor | indent 2}} {{- end }} {{- end }} @@ -324,8 +324,8 @@ service: - resource - k8s_tagger - source -{{- if .Values.metadata.metrics.config.extraProcessors }} -{{- range $processor := .Values.metadata.metrics.config.extraProcessors }} +{{- if .Values.sumologic.metrics.otelcol.extraProcessors }} +{{- range $processor := .Values.sumologic.metrics.otelcol.extraProcessors }} {{ printf "- %s" ( $processor | keys | first ) | indent 8 }} {{- end }} {{- end }} diff --git a/deploy/helm/sumologic/values.yaml b/deploy/helm/sumologic/values.yaml index 9804514b4f..1ee4de6b0e 100644 --- a/deploy/helm/sumologic/values.yaml +++ b/deploy/helm/sumologic/values.yaml @@ -380,6 +380,19 @@ sumologic: ## Set metadata provider service (either fluentd or otelcol). provider: otelcol + otelcol: + ## Includes additional processors into pipelines. + ## It can be used for filtering metrics, renaming, changing metadata and so on. + ## This is list of objects, for example: + ## extraProcessors: + ## - filterprocessor: + ## exclude: + ## match_type: strict + ## metric_names: + ## - hello_world + ## - hello/world + extraProcessors: [] + ### Enable a load balancing proxy for Prometheus remote writes. ## Prometheus remote write uses a single persistent HTTP connection per target, ## which interacts poorly with TCP load balancing with iptables that K8s Services do. @@ -3808,17 +3821,6 @@ metadata: ## The value of this key should be a dictionary, that will replace the normal configuration. ## This is an advanced feature, use with caution, and review the generated configuration first. override: {} - ## Includes additional processors into pipelines. - ## It can be used for filtering metrics, renaming, changing metadata and so on. - ## This is list of objects, for example: - ## extraProcessors: - ## - filterprocessor: - ## exclude: - ## match_type: strict - ## metric_names: - ## - hello_world - ## - hello/world - extraProcessors: [] statefulset: nodeSelector: {} diff --git a/docs/best-practices.md b/docs/best-practices.md index 28789c4abb..5060a02bb5 100644 --- a/docs/best-practices.md +++ b/docs/best-practices.md @@ -198,9 +198,9 @@ See [the Filtering metrics](/docs/collecting-application-metrics.md#filtering-me For example to filter out all metrics from `sumologic` namespace, you can use the following configuration: ```yaml -metadata: +sumologic: metrics: - config: + otelcol: extraProcessors: - filter/exclude_sumo_metrics: metrics: @@ -217,9 +217,9 @@ metadata: To exclude all metrics starting with `kube_`, you can use the following configuration: ```yaml -metadata: +sumologic: metrics: - config: + otelcol: extraProcessors: - filter/exclude_sumo_metrics: metrics: @@ -245,9 +245,9 @@ See [the Filtering metrics](/docs/collecting-application-metrics.md#adding-or-re For example to remove `namespace` and `deployment` dimensions, you can use the following configuration: ```yaml -metadata: +sumologic: metrics: - config: + otelcol: extraProcessors: - transform/remove_dimensions: metric_statements: diff --git a/docs/collecting-application-metrics.md b/docs/collecting-application-metrics.md index 8acacceb8c..3d6d4994f7 100644 --- a/docs/collecting-application-metrics.md +++ b/docs/collecting-application-metrics.md @@ -235,9 +235,9 @@ In order to filter in or out the metrics, you can add [filterprocessor] to metri Please see the following example: ```yaml -metadata: +sumologic: metrics: - config: + otelcol: extraProcessors: - filter/1: metrics: @@ -309,9 +309,9 @@ In order to rename metrics, the [transformprocessor] can be use. Please look at the following snippet: ```yaml -metadata: +sumologic: metrics: - config: + otelcol: extraProcessors: - transform/1: metric_statements: @@ -327,9 +327,9 @@ If you want to add or rename metadata, the [transformprocessor] can be use. Please look at the following snippet: ```yaml -metadata: +sumologic: metrics: - config: + otelcol: extraProcessors: - transform/1: metric_statements: diff --git a/docs/v3-migration-doc.md b/docs/v3-migration-doc.md index 2ff9ac32f7..b92416bdea 100644 --- a/docs/v3-migration-doc.md +++ b/docs/v3-migration-doc.md @@ -109,7 +109,7 @@ In this document we detail the changes as well as the exact steps for migration. - Adding `sumologic.metrics.serviceMonitors` to avoid copying values for `kube-prometheus-stack.prometheus.additionalServiceMonitors` configuration -- Adding `metadata.metrics.config.extraProcessors` to make metrics modification easy +- Adding `sumologic.metrics.otelcol.extraProcessors` to make metrics modification easy ## How to upgrade diff --git a/tests/helm/testdata/opentelemetry-metrics-extra-processors.yaml b/tests/helm/testdata/opentelemetry-metrics-extra-processors.yaml index 3dbd9b85de..136c196c8d 100644 --- a/tests/helm/testdata/opentelemetry-metrics-extra-processors.yaml +++ b/tests/helm/testdata/opentelemetry-metrics-extra-processors.yaml @@ -1,6 +1,6 @@ -metadata: +sumologic: metrics: - config: + otelcol: extraProcessors: - filter/1: metrics: