Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(metadata/otc): upgrade to v0.50.0-sumo-0 #2251

Merged
merged 9 commits into from
May 5, 2022
Merged

feat(metadata/otc): upgrade to v0.50.0-sumo-0 #2251

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8878ecc
feat(metadata/otc): upgrade to v0.49.0-sumo-0
sumo-drosiek Apr 28, 2022
558c598
chore(changelog): update
sumo-drosiek Apr 28, 2022
8f48e28
feat(metrics/opentelemetry): remove http_listener_v2_path in routing …
sumo-drosiek Apr 28, 2022
e9d98fd
fix(otelcol/systemd): fix systemd pipeline
Apr 28, 2022
227fd0b
tests(templates): update tests for metadata_logs_otc
sumo-drosiek Apr 29, 2022
0135eb1
fix: fix syslog and kubelet filtering according to integration tests
sumo-drosiek Apr 29, 2022
8b1f368
fix: drop Pod annotation attributes for container logs
May 4, 2022
afd246a
chore(otelcol): upgrade to v0.50.0-sumo-0
sumo-drosiek May 4, 2022
947f6e1
fix: revert unwanted change
sumo-drosiek May 4, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- chore: change Fluent Bit image to `public.ecr.aws/sumologic/fluent-bit:1.6.10-sumo-2`,
it is Fluent Bit 1.6.10 with updated dependencies,
image repository: https://github.com/SumoLogic/fluent-bit-docker-image [#2254][#2254]
- feat(metadata/otc): upgrade to v0.49.0-sumo-0 [#2251][#2251]
sumo-drosiek marked this conversation as resolved.
Show resolved Hide resolved

### Fixed

Expand All @@ -36,6 +37,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
[#2246]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2246
[#2254]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2254
[#2255]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2255
[#2251]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2251
[Unreleased]: https://github.com/SumoLogic/sumologic-kubernetes-collection/compare/v2.6.0...main

## [v2.7.0][v2.7.0]
Expand Down
5 changes: 1 addition & 4 deletions deploy/docs/Best_Practices.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,9 +214,6 @@ metadata:
- action: extract
key: fluent.tag
pattern: ^containers\.var\.log\.pods\.(?P<k8s_namespace>[^_]+)_(?P<k8s_pod_name>[^_]+)_(?P<k8s_uid>[a-f0-9\-]{36})\.(?P<k8s_container_name>[^\._]+)\.(?P<k8s_run_id>\d+)\.log$
- action: insert
key: k8s.pod.uid
from_attribute: k8s_uid
- action: delete
key: k8s_uid
- action: delete
Expand Down Expand Up @@ -1058,7 +1055,7 @@ For [sumologic exporter][sumologic_exporter]:
**because once filled in PVC never reduces its fill.**

[batch_processor]: https://github.com/open-telemetry/opentelemetry-collector/tree/v0.47.0/processor/batchprocessor#batch-processor
[sumologic_exporter]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/exporter/sumologicexporter#sumo-logic-exporter
[sumologic_exporter]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/exporter/sumologicexporter#sumo-logic-exporter
[filling_up_alert]: https://runbooks.prometheus-operator.dev/runbooks/kubernetes/kubepersistentvolumefillingup/

### Compaction
Expand Down
44 changes: 22 additions & 22 deletions deploy/docs/fluentd_otc_comparison.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@
| `disable_cookies` | Cookies are not used in Opentelemetry Collector |
| `compress` | [exporters.sumologic.compress_encoding][otelcol_sumologic_config] set to `""` |
| `compress_encoding` | [exporters.sumologic.compress_encoding][otelcol_sumologic_config] |
| `custom_fields` | [Resource processor][resource_processor] combined with [exporters.sumologic.metadata_attributes][otelcol_sumologic_config] |
| `custom_dimensions` | [Resource processor][resource_processor] combined with [exporters.sumologic.metadata_attributes][otelcol_sumologic_config] |
| `custom_fields` | [Resource processor][resource_processor] |
| `custom_dimensions` | [Resource processor][resource_processor] |

Additional behavior:

Expand All @@ -50,14 +50,14 @@ Additional behavior:
| [record[_sumo_metadata][source_name]][source_name_precedence] taking precedence over `source_name` | Can be achieved by separate pipelines |
| [record[_sumo_metadata][source_host]][source_host_precedence] taking precedence over `source_host` | Can be achieved by separate pipelines |
| [record[_sumo_metadata][source_category]][source_category_precedence] taking precedence over `source_category` | Can be achieved by separate pipelines |
| [record[_sumo_metadata][fields]][fields_base] being base for fields | Can be achieved using [resource processor][resource_processor], separate pipelines and [exporter.sumologic.metadata_attributes][otelcol_sumologic_config] |
| [record[_sumo_metadata][fields]][fields_base] being base for fields | Can be achieved using [resource processor][resource_processor] and separate pipelines |

[fields_base]: https://github.com/SumoLogic/fluentd-output-sumologic/blob/1.7.2/lib/fluent/plugin/out_sumologic.rb#L284-L285
[fluentd_output_plugin]: https://github.com/sumologic/fluentd-output-sumologic/tree/1.7.2#configuration
[otelcol_basic_confg]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.47.0-sumo-0/docs/Configuration.md#basic-configuration
[otelcol_proxy]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.47.0-sumo-0/docs/Configuration.md#proxy-support
[otelcol_source_config]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/processor/sourceprocessor#config
[otelcol_sumologic_config]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.47.0-sumo-0/pkg/exporter/sumologicexporter/README.md#sumo-logic-exporter
[otelcol_basic_confg]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.49.0-sumo-0/docs/Configuration.md#basic-configuration
[otelcol_proxy]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.49.0-sumo-0/docs/Configuration.md#proxy-support
[otelcol_source_config]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/processor/sourceprocessor#config
[otelcol_sumologic_config]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.49.0-sumo-0/pkg/exporter/sumologicexporter/README.md#sumo-logic-exporter
[otelocl_tls_config]: https://github.com/open-telemetry/opentelemetry-collector/blob/v0.47.0/config/configtls/README.md#tls--mtls-configuration
[resource_processor]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.47.0/processor/resourceprocessor#resource-processor
[source_category_precedence]: https://github.com/SumoLogic/fluentd-output-sumologic/blob/1.7.2/lib/fluent/plugin/out_sumologic.rb#L278-L279
Expand All @@ -70,15 +70,15 @@ In order to receive prometheus data and for their initial processing [telegrafre
It should cover [fluent-plugin-datapoint][fluent_plugin_datapoint] functionality and more.

[fluent_plugin_datapoint]: https://github.com/SumoLogic/sumologic-kubernetes-fluentd/tree/v1.12.2-sumo-4/fluent-plugin-datapoint
[telegrafreceiver]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/receiver/telegrafreceiver
[telegrafreceiver]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/receiver/telegrafreceiver

### fluent-plugin-protobuf

In order to receive prometheus data and for their initial processing [telegrafreceiver][telegrafreceiver] is being used.
It should cover [fluent_plugin_protobuf][fluent_plugin_protobuf] functionality and more.

[fluent_plugin_protobuf]: https://github.com/SumoLogic/sumologic-kubernetes-fluentd/tree/v1.12.2-sumo-4/fluent-plugin-protobuf
[telegrafreceiver]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/receiver/telegrafreceiver
[telegrafreceiver]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/receiver/telegrafreceiver

### fluent-plugin-prometheus-format

Expand Down Expand Up @@ -162,17 +162,17 @@ Sanitized pod name is name portion of the pod. Please consider following example
[kube_daemonset]: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
[kube_deployment]: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
[kube_statefulset]: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset
[otelcol_annotations]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/processor/sourceprocessor#pod-annotations
[otelcol_annotations]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/processor/sourceprocessor#pod-annotations
[otelcol_annotations_exclude]: https://github.com/SumoLogic/sumologic-kubernetes-fluentd/blob/v1.12.2-sumo-4/fluent-plugin-kubernetes-sumologic/lib/fluent/plugin/filter_kubernetes_sumologic.rb#L171-L183
[otelcol_source_templates]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/exporter/sumologicexporter#source-templates
[otelcol_undefined]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.47.0-sumo-0/pkg/processor/sourceprocessor/attribute_filler.go#L113-L123
[source_containers]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/processor/sourceprocessor#container-level-pod-annotations
[source_filtering]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/processor/sourceprocessor#filtering-section
[source_keys]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/processor/sourceprocessor#keys-section
[source_processor]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/processor/sourceprocessor#config
[source_processor_source_templates]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/processor/sourceprocessor#name-translation-and-template-keys
[otelcol_source_templates]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/exporter/sumologicexporter#source-templates
[otelcol_undefined]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.49.0-sumo-0/pkg/processor/sourceprocessor/attribute_filler.go#L113-L123
[source_containers]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/processor/sourceprocessor#container-level-pod-annotations
[source_filtering]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/processor/sourceprocessor#filtering-section
[source_keys]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/processor/sourceprocessor#keys-section
[source_processor]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/processor/sourceprocessor#config
[source_processor_source_templates]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/processor/sourceprocessor#name-translation-and-template-keys
[sumo_metadata]: https://github.com/SumoLogic/sumologic-kubernetes-fluentd/tree/v1.12.2-sumo-4/fluent-plugin-kubernetes-sumologic#fluent-plugin-kubernetes-sumologic
[sumologic_exporter]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.47.0-sumo-0/pkg/exporter/sumologicexporter/README.md#sumo-logic-exporter
[sumologic_exporter]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.49.0-sumo-0/pkg/exporter/sumologicexporter/README.md#sumo-logic-exporter

### fluent-plugin-kubernetes-metadata-filter

Expand All @@ -190,8 +190,8 @@ Sanitized pod name is name portion of the pod. Please consider following example
| `cache_ttl` | N/A |

[fluent_plugin_k8s_metadata]: https://github.com/SumoLogic/sumologic-kubernetes-fluentd/tree/v1.12.2-sumo-4/fluent-plugin-kubernetes-metadata-filter#configuration
[k8sprocessor]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.47.0-sumo-0/pkg/processor/k8sprocessor
[k8sprocessor_field_extract]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/processor/k8sprocessor#field-extract-config
[k8sprocessor]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.49.0-sumo-0/pkg/processor/k8sprocessor
[k8sprocessor_field_extract]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/processor/k8sprocessor#field-extract-config
[kubeconfig_auth_type]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.47.0/internal/k8sconfig/config.go#L53-L60

### fluent-plugin-enhance-k8s-metadata
Expand All @@ -209,7 +209,7 @@ Sanitized pod name is name portion of the pod. Please consider following example
| `data_type` | N/A |

[fluent_plugin_enhance_k8s_metadata]: https://github.com/SumoLogic/sumologic-kubernetes-fluentd/tree/v1.12.2-sumo-4/fluent-plugin-enhance-k8s-metadata#configuration
[pod_association]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.47.0-sumo-0/pkg/processor/k8sprocessor/doc.go#L17-L46
[pod_association]: https://github.com/SumoLogic/sumologic-otel-collector/blob/v0.49.0-sumo-0/pkg/processor/k8sprocessor/doc.go#L17-L46

### fluent-plugin-events

Expand Down Expand Up @@ -358,7 +358,7 @@ Events are not supported by `Opentelemetry Collector`

[filter_processor]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.47.0/processor/filterprocessor#filter-processor
[readme]: ../helm/sumologic/README.md
[source_containers]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.47.0-sumo-0/pkg/processor/sourceprocessor#container-level-pod-annotations
[source_containers]: https://github.com/SumoLogic/sumologic-otel-collector/tree/v0.49.0-sumo-0/pkg/processor/sourceprocessor#container-level-pod-annotations
[persistent_queue]: https://github.com/open-telemetry/opentelemetry-collector/tree/release/v0.37.x/exporter/exporterhelper#persistent-queue
[file_storage_extension]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/release/v0.37.x/extension/storage/filestorage

Expand Down
Loading