Skip to content

Commit

Permalink
feat(metrics): move metadata.metrics.config.extraProcessors to `sum…
Browse files Browse the repository at this point in the history 
…ologic.metrics.otelcol.extraProcessors`
  • Loading branch information
@andrzej-stencel
andrzej-stencel committed Jan 12, 2023
1 parent 3b7a609 commit a19d02a
Show file tree
Hide file tree
Showing 8 changed files with 35 additions and 32 deletions.
3 changes: 2 additions & 1 deletion CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- feat: add metadata.metrics.config.extraProcessors [#2724]
- feat(metrics): add `sumologic.metrics.otelcol.extraProcessors` [#2724] [#2780]
- feat: add otellogs.additionDaemonSets configuration [#2750]
- chore: upgrade Fluentd to v1.15.3-sumo-0 [#2745]
- This also upgrades Ruby from `v2.7` to `v3.1` and some other dependencies.
Expand Down Expand Up @@ -37,6 +37,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
[#2766]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2766
[#2754]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2754
[#2771]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2771
[#2780]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2780
[v1.15.3-sumo-0]: https://github.com/SumoLogic/sumologic-kubernetes-fluentd/releases/tag/v1.15.3-sumo-0
[Unreleased]: https://github.com/SumoLogic/sumologic-kubernetes-collection/compare/v3.0.0-beta.0...main

Expand Down
2 changes: 1 addition & 1 deletion deploy/helm/sumologic/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ The following table lists the configurable parameters of the Sumo Logic chart an
| `sumologic.logs.metadata.provider` | Set provider to use for logs forwarding and metadata enrichment. Can be either otelcol or fluentd. | `otelcol` |
| `sumologic.metrics.enabled` | Set the enabled flag to false for disabling metrics ingestion altogether. | `true` |
| `sumologic.metrics.metadata.provider` | Set provider to use for metrics forwarding and metadata enrichment. Can be either otelcol or fluentd. | `otelcol` |
| `sumologic.metrics.otelcol.extraProcessors` | Extra processors configuration for metrics pipeline. See [/docs/collecting-application-metrics.md#metrics-modifications](/docs/collecting-application-metrics.md#metrics-modifications) for more information. | `[]` |
| `sumologic.metrics.remoteWriteProxy.enabled` | Enable a load balancing proxy for Prometheus remote writes. [See docs/best-practices.md for more information.](/docs/best-practices.md) | `true` |
| `sumologic.metrics.remoteWriteProxy.config.clientBodyBufferSize` | See the [nginx documentation](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size). Increase if you've also increased samples per send in Prometheus remote write. | `64k` |
| `sumologic.metrics.remoteWriteProxy.config.workerCountAutotune` | This feature autodetects how much CPU is assigned to the nginx instance and setsthe right amount of workers based on that. Disable to use the default of 8 workers. | `true` |
Expand Down Expand Up @@ -490,7 +491,6 @@ The following table lists the configurable parameters of the Sumo Logic chart an
| `metadata.metrics.logLevel` | Flag to control logging level for OpenTelemetry Collector for metrics. Can be `debug`, `info`, `warn`, `error`, `dpanic`, `panic`, `fatal`. | `info` |
| `metadata.metrics.config.merge` | Configuration for metrics metadata otelcol, merged with defaults. See also https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/configuration.md. | {} |
| `metadata.metrics.config.override` | Configuration for metrics metadata otelcol, replaces defaults.See also https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/configuration.md. | {} |
| `metadata.metrics.config.extraProcessors` | Extra processors configuration for metrics pipeline. See [/docs/collecting-application-metrics.md#metrics-modifications](/docs/collecting-application-metrics.md#metrics-modifications) for more information. | `[]` |
| `metadata.metrics.statefulset.containers.otelcol.startupProbe` | Startup probe configuration for metrics otelcol container. | `{"periodSeconds": 3, "failureThreshold": 60}` |
| `metadata.metrics.statefulset.nodeSelector` | Node selector for metrics metadata enrichment (otelcol) statefulset. [See docs/best-practices.md for more information.](/docs/best-practices.md) | `{}` |
| `metadata.metrics.statefulset.tolerations` | Tolerations for metrics metadata enrichment (otelcol) statefulset. | `[]` |
Expand Down
8 changes: 4 additions & 4 deletions deploy/helm/sumologic/conf/metrics/otelcol/config.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -299,8 +299,8 @@ processors:
## ref: https://github.com/SumoLogic/sumologic-otel-collector/tree/main/pkg/processor/sourceprocessor
source:
collector: {{ .Values.sumologic.collectorName | default .Values.sumologic.clusterName | quote }}
{{- if .Values.metadata.metrics.config.extraProcessors }}
{{- range $processor := .Values.metadata.metrics.config.extraProcessors }}
{{- if .Values.sumologic.metrics.otelcol.extraProcessors }}
{{- range $processor := .Values.sumologic.metrics.otelcol.extraProcessors }}
{{ toYaml $processor | indent 2}}
{{- end }}
{{- end }}
Expand All @@ -324,8 +324,8 @@ service:
- resource
- k8s_tagger
- source
{{- if .Values.metadata.metrics.config.extraProcessors }}
{{- range $processor := .Values.metadata.metrics.config.extraProcessors }}
{{- if .Values.sumologic.metrics.otelcol.extraProcessors }}
{{- range $processor := .Values.sumologic.metrics.otelcol.extraProcessors }}
{{ printf "- %s" ( $processor | keys | first ) | indent 8 }}
{{- end }}
{{- end }}
Expand Down
24 changes: 13 additions & 11 deletions deploy/helm/sumologic/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,19 @@ sumologic:
## Set metadata provider service (either fluentd or otelcol).
provider: otelcol

otelcol:
## Includes additional processors into pipelines.
## It can be used for filtering metrics, renaming, changing metadata and so on.
## This is list of objects, for example:
## extraProcessors:
## - filterprocessor:
## exclude:
## match_type: strict
## metric_names:
## - hello_world
## - hello/world
extraProcessors: []

### Enable a load balancing proxy for Prometheus remote writes.
## Prometheus remote write uses a single persistent HTTP connection per target,
## which interacts poorly with TCP load balancing with iptables that K8s Services do.
Expand Down Expand Up @@ -3808,17 +3821,6 @@ metadata:
## The value of this key should be a dictionary, that will replace the normal configuration.
## This is an advanced feature, use with caution, and review the generated configuration first.
override: {}
## Includes additional processors into pipelines.
## It can be used for filtering metrics, renaming, changing metadata and so on.
## This is list of objects, for example:
## extraProcessors:
## - filterprocessor:
## exclude:
## match_type: strict
## metric_names:
## - hello_world
## - hello/world
extraProcessors: []

statefulset:
nodeSelector: {}
Expand Down
12 changes: 6 additions & 6 deletions docs/best-practices.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -198,9 +198,9 @@ See [the Filtering metrics](/docs/collecting-application-metrics.md#filtering-me
For example to filter out all metrics from `sumologic` namespace, you can use the following configuration:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- filter/exclude_sumo_metrics:
metrics:
Expand All @@ -217,9 +217,9 @@ metadata:
To exclude all metrics starting with `kube_`, you can use the following configuration:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- filter/exclude_sumo_metrics:
metrics:
Expand All @@ -245,9 +245,9 @@ See [the Filtering metrics](/docs/collecting-application-metrics.md#adding-or-re
For example to remove `namespace` and `deployment` dimensions, you can use the following configuration:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- transform/remove_dimensions:
metric_statements:
Expand Down
12 changes: 6 additions & 6 deletions docs/collecting-application-metrics.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -235,9 +235,9 @@ In order to filter in or out the metrics, you can add [filterprocessor] to metri
Please see the following example:
```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- filter/1:
metrics:
Expand Down Expand Up @@ -309,9 +309,9 @@ In order to rename metrics, the [transformprocessor] can be use.
Please look at the following snippet:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- transform/1:
metric_statements:
Expand All @@ -327,9 +327,9 @@ If you want to add or rename metadata, the [transformprocessor] can be use.
Please look at the following snippet:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- transform/1:
metric_statements:
Expand Down
2 changes: 1 addition & 1 deletion docs/v3-migration-doc.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ In this document we detail the changes as well as the exact steps for migration.
- Adding `sumologic.metrics.serviceMonitors` to avoid copying values for
`kube-prometheus-stack.prometheus.additionalServiceMonitors` configuration

- Adding `metadata.metrics.config.extraProcessors` to make metrics modification easy
- Adding `sumologic.metrics.otelcol.extraProcessors` to make metrics modification easy

## How to upgrade

Expand Down
4 changes: 2 additions & 2 deletions tests/helm/testdata/opentelemetry-metrics-extra-processors.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- filter/1:
metrics:
Expand Down

0 comments on commit a19d02a

Please sign in to comment.