Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(metrics): move metadata.metrics.config.extraProcessors to sumologic.metrics.otelcol.extraProcessors #2780

Merged
merged 1 commit into from
Jan 12, 2023
Merged

feat(metrics): move metadata.metrics.config.extraProcessors to sumologic.metrics.otelcol.extraProcessors #2780

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- feat: add metadata.metrics.config.extraProcessors [#2724]
- feat(metrics): add `sumologic.metrics.otelcol.extraProcessors` [#2724] [#2780]
- feat: add otellogs.additionDaemonSets configuration [#2750]
- chore: upgrade Fluentd to v1.15.3-sumo-0 [#2745]
- This also upgrades Ruby from `v2.7` to `v3.1` and some other dependencies.
Expand Down Expand Up @@ -37,6 +37,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
[#2766]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2766
[#2754]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2754
[#2771]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2771
[#2780]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2780
[v1.15.3-sumo-0]: https://github.com/SumoLogic/sumologic-kubernetes-fluentd/releases/tag/v1.15.3-sumo-0
[Unreleased]: https://github.com/SumoLogic/sumologic-kubernetes-collection/compare/v3.0.0-beta.0...main

Expand Down
2 changes: 1 addition & 1 deletion deploy/helm/sumologic/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ The following table lists the configurable parameters of the Sumo Logic chart an
| `sumologic.logs.metadata.provider` | Set provider to use for logs forwarding and metadata enrichment. Can be either otelcol or fluentd. | `otelcol` |
| `sumologic.metrics.enabled` | Set the enabled flag to false for disabling metrics ingestion altogether. | `true` |
| `sumologic.metrics.metadata.provider` | Set provider to use for metrics forwarding and metadata enrichment. Can be either otelcol or fluentd. | `otelcol` |
| `sumologic.metrics.otelcol.extraProcessors` | Extra processors configuration for metrics pipeline. See [/docs/collecting-application-metrics.md#metrics-modifications](/docs/collecting-application-metrics.md#metrics-modifications) for more information. | `[]` |
| `sumologic.metrics.remoteWriteProxy.enabled` | Enable a load balancing proxy for Prometheus remote writes. [See docs/best-practices.md for more information.](/docs/best-practices.md) | `true` |
| `sumologic.metrics.remoteWriteProxy.config.clientBodyBufferSize` | See the [nginx documentation](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size). Increase if you've also increased samples per send in Prometheus remote write. | `64k` |
| `sumologic.metrics.remoteWriteProxy.config.workerCountAutotune` | This feature autodetects how much CPU is assigned to the nginx instance and setsthe right amount of workers based on that. Disable to use the default of 8 workers. | `true` |
Expand Down Expand Up @@ -490,7 +491,6 @@ The following table lists the configurable parameters of the Sumo Logic chart an
| `metadata.metrics.logLevel` | Flag to control logging level for OpenTelemetry Collector for metrics. Can be `debug`, `info`, `warn`, `error`, `dpanic`, `panic`, `fatal`. | `info` |
| `metadata.metrics.config.merge` | Configuration for metrics metadata otelcol, merged with defaults. See also https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/configuration.md. | {} |
| `metadata.metrics.config.override` | Configuration for metrics metadata otelcol, replaces defaults.See also https://github.com/SumoLogic/sumologic-otel-collector/blob/main/docs/configuration.md. | {} |
| `metadata.metrics.config.extraProcessors` | Extra processors configuration for metrics pipeline. See [/docs/collecting-application-metrics.md#metrics-modifications](/docs/collecting-application-metrics.md#metrics-modifications) for more information. | `[]` |
| `metadata.metrics.statefulset.containers.otelcol.startupProbe` | Startup probe configuration for metrics otelcol container. | `{"periodSeconds": 3, "failureThreshold": 60}` |
| `metadata.metrics.statefulset.nodeSelector` | Node selector for metrics metadata enrichment (otelcol) statefulset. [See docs/best-practices.md for more information.](/docs/best-practices.md) | `{}` |
| `metadata.metrics.statefulset.tolerations` | Tolerations for metrics metadata enrichment (otelcol) statefulset. | `[]` |
Expand Down
8 changes: 4 additions & 4 deletions deploy/helm/sumologic/conf/metrics/otelcol/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -299,8 +299,8 @@ processors:
## ref: https://github.com/SumoLogic/sumologic-otel-collector/tree/main/pkg/processor/sourceprocessor
source:
collector: {{ .Values.sumologic.collectorName | default .Values.sumologic.clusterName | quote }}
{{- if .Values.metadata.metrics.config.extraProcessors }}
{{- range $processor := .Values.metadata.metrics.config.extraProcessors }}
{{- if .Values.sumologic.metrics.otelcol.extraProcessors }}
{{- range $processor := .Values.sumologic.metrics.otelcol.extraProcessors }}
{{ toYaml $processor | indent 2}}
{{- end }}
{{- end }}
Expand All @@ -324,8 +324,8 @@ service:
- resource
- k8s_tagger
- source
{{- if .Values.metadata.metrics.config.extraProcessors }}
{{- range $processor := .Values.metadata.metrics.config.extraProcessors }}
{{- if .Values.sumologic.metrics.otelcol.extraProcessors }}
{{- range $processor := .Values.sumologic.metrics.otelcol.extraProcessors }}
{{ printf "- %s" ( $processor | keys | first ) | indent 8 }}
{{- end }}
{{- end }}
Expand Down
24 changes: 13 additions & 11 deletions deploy/helm/sumologic/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,19 @@ sumologic:
## Set metadata provider service (either fluentd or otelcol).
provider: otelcol

otelcol:
## Includes additional processors into pipelines.
## It can be used for filtering metrics, renaming, changing metadata and so on.
## This is list of objects, for example:
## extraProcessors:
## - filterprocessor:
## exclude:
## match_type: strict
## metric_names:
## - hello_world
## - hello/world
extraProcessors: []

### Enable a load balancing proxy for Prometheus remote writes.
## Prometheus remote write uses a single persistent HTTP connection per target,
## which interacts poorly with TCP load balancing with iptables that K8s Services do.
Expand Down Expand Up @@ -3808,17 +3821,6 @@ metadata:
## The value of this key should be a dictionary, that will replace the normal configuration.
## This is an advanced feature, use with caution, and review the generated configuration first.
override: {}
## Includes additional processors into pipelines.
## It can be used for filtering metrics, renaming, changing metadata and so on.
## This is list of objects, for example:
## extraProcessors:
## - filterprocessor:
## exclude:
## match_type: strict
## metric_names:
## - hello_world
## - hello/world
extraProcessors: []

statefulset:
nodeSelector: {}
Expand Down
12 changes: 6 additions & 6 deletions docs/best-practices.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,9 +198,9 @@ See [the Filtering metrics](/docs/collecting-application-metrics.md#filtering-me
For example to filter out all metrics from `sumologic` namespace, you can use the following configuration:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- filter/exclude_sumo_metrics:
metrics:
Expand All @@ -217,9 +217,9 @@ metadata:
To exclude all metrics starting with `kube_`, you can use the following configuration:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- filter/exclude_sumo_metrics:
metrics:
Expand All @@ -245,9 +245,9 @@ See [the Filtering metrics](/docs/collecting-application-metrics.md#adding-or-re
For example to remove `namespace` and `deployment` dimensions, you can use the following configuration:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- transform/remove_dimensions:
metric_statements:
Expand Down
12 changes: 6 additions & 6 deletions docs/collecting-application-metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,9 +235,9 @@ In order to filter in or out the metrics, you can add [filterprocessor] to metri
Please see the following example:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- filter/1:
metrics:
Expand Down Expand Up @@ -309,9 +309,9 @@ In order to rename metrics, the [transformprocessor] can be use.
Please look at the following snippet:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- transform/1:
metric_statements:
Expand All @@ -327,9 +327,9 @@ If you want to add or rename metadata, the [transformprocessor] can be use.
Please look at the following snippet:

```yaml
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- transform/1:
metric_statements:
Expand Down
2 changes: 1 addition & 1 deletion docs/v3-migration-doc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ In this document we detail the changes as well as the exact steps for migration.
- Adding `sumologic.metrics.serviceMonitors` to avoid copying values for
`kube-prometheus-stack.prometheus.additionalServiceMonitors` configuration

- Adding `metadata.metrics.config.extraProcessors` to make metrics modification easy
- Adding `sumologic.metrics.otelcol.extraProcessors` to make metrics modification easy

## How to upgrade

Expand Down
4 changes: 2 additions & 2 deletions tests/helm/testdata/opentelemetry-metrics-extra-processors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
metadata:
sumologic:
metrics:
config:
otelcol:
extraProcessors:
- filter/1:
metrics:
Expand Down