Skip to content

Commit

Permalink
feat: add experimental OT event collection
Browse files Browse the repository at this point in the history
  • Loading branch information
Mikołaj Świątek committed Jul 4, 2022
1 parent 22e9328 commit c712362
Show file tree
Hide file tree
Showing 27 changed files with 1,164 additions and 10 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ This change affects you only if you have enabled traces with `sumologic.traces.e
### Added

- feat(metrics): add service metrics [#2367]
- feat(events): add experimental OT event collection [#2379]

### Changed

Expand All @@ -37,6 +38,7 @@ This change affects you only if you have enabled traces with `sumologic.traces.e
[#2397]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2397
[#2325]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2325
[#2390]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2390
[#2379]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2379
[Unreleased]: https://github.com/SumoLogic/sumologic-kubernetes-collection/compare/v2.10.0...main

## [v2.10.0]
Expand Down
29 changes: 29 additions & 0 deletions deploy/docs/opentelemetry_collector.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -594,3 +594,32 @@ Refs:

- [Trace ID aware load balancing](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/loadbalancingexporter/README.md)
- [Using cascading_filter](https://help.sumologic.com/Traces/03Advanced_Configuration/What_if_I_don't_want_to_send_all_the_tracing_data_to_Sumo_Logic%3F)

## Kubernetes Events

Opentelemetry Collector can be used to collect and enrich Kubernetes Events in lieu of FluentD.
This is a drop-in replacement, and currently the OT configuration directly uses some of the FluentD configuration values.
It can be enabled by setting:

```yaml
sumologic:
fluentd:
events:
enabled: false

otelevents:
enabled: true
```

For configurations which don't modify `sumologic.fluentd.events.overrideOutputConf`, this should be enough. See the configuration
options under `sumologic.otelevents` in [values.yaml](../helm/sumologic/values.yaml).

### Customizing OpenTelemetry Collector configuration

If the configuration options present under the `sumologic.otelevents` key aren't sufficient for your needs, you can override
the OT configuration directly. Be aware that doing this isn't subject to normal backwards-compatibility guarantees offered by
this chart, so you'll need to be more careful during upgrades. The exact OT configuration emitted by the chart may change
even in minor releases.

In order to override the configuration, use the `sumologic.otelevents.config.override` key. This key takes a yaml object, whose
value is merged with the configuration generated by the Chart.
13 changes: 13 additions & 0 deletions deploy/helm/sumologic/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -377,6 +377,19 @@ The following table lists the configurable parameters of the Sumo Logic chart an
| `metadata.logs.autoscaling.targetCPUUtilizationPercentage` | The desired target CPU utilization for autoscaling. | `50` |
| `metadata.logs.autoscaling.targetMemoryUtilizationPercentage` | The desired target memory utilization for autoscaling. | `Nil` |
| `metadata.logs.podDisruptionBudget` | Pod Disruption Budget for logs metadata enrichment (otelcol) statefulset. | `{"minAvailable": 2}` |
| `otelevents.enabled` | Enable event collection using the OpenTelemetry Collector. Be sure to disable `fluentd.events.enabled` when enabling this. | `false` |
| `otelevents.image.repository` | Image repository for otelcol docker container. | `public.ecr.aws/sumologic/sumologic-otel-collector` |
| `otelevents.image.tag` | Image tag for otelcol docker container. | `0.0.18` |
| `otelevents.image.pullPolicy` | Image pullPolicy for otelcol docker container. | `IfNotPresent` |
| `otelevents.logLevel` | Log level for the OpenTelemtry Collector. Can be `debug`, `info`, `warn`, `error`, `dpanic`, `panic`, `fatal`. | `info` |
| `otelevents.persistence.enabled` | Enable persistence for OpenTelemetry Collector. | `true` |
| `otelevents.persistence.storageClass` | Defines storageClassName for the PersistentVolumeClaim which is used to provide persistence for OpenTelemetry Collector. | `Nil` |
| `otelevents.persistence.accessMode` | The accessMode for the volume which is used to provide persistence for OpenTelemetry Collector. | `ReadWriteOnce` |
| `otelevents.persistence.size` | Size of the volume which is used to provide persistence for OpenTelemetry Collector. | `10Gi` |
| `otelevents.persistence.pvcLabels` | Additional PersistentVolumeClaim labels for all OpenTelemetry Collector pods. | `{}` |
| `otelevents.config.override` | Override configuration for OpenTelemetry Collector. See [the documentation](../../docs/opentelemetry_collector.md#customizing-opentelemetry-collector-configuration) for more details. | `{}` |
| `otelevents.statefulset` | OpenTelemetry Collector StatefulSet customization options. See values.yaml for more details. | See values.yaml | | `Nil` | | `Nil`
| `otelevents.podDisruptionBudget` | Pod Disruption Budget for the OpenTelemetry Collector StatefulSet. | See values.yaml |
| `tailing-sidecar-operator.enabled` | Flag to control deploying Tailing Sidecar Operator Helm sub-chart. | `false` |
| `tailing-sidecar-operator.fullnameOverride` | Used to override the chart's full name. | `Nil` |
| `prometheus.prometheusSpec.nodeSelector` | Node selector for prometheus. [See docs/Best_Practices.md for more information.](../../docs/Best_Practices.md) | `{}` |
Expand Down
0 ...oy/helm/sumologic/conf/events/events.conf → ...sumologic/conf/events/fluentd/events.conf
View file
File renamed without changes.
63 changes: 63 additions & 0 deletions deploy/helm/sumologic/conf/events/otelcol/config.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
receivers:
raw_k8s_events: {}
extensions:
health_check: {}
{{- if .Values.otelevents.persistence.enabled }}
file_storage:
directory: /var/lib/storage/otc
timeout: 10s
{{- end }}
pprof: {}
exporters:
sumologic:
log_format: json
json_logs:
add_timestamp: true
timestamp_key: timestamp
endpoint: ${SUMO_ENDPOINT_DEFAULT_EVENTS_SOURCE}
source_name: "%{_sourceName}"
source_category: "%{_sourceCategory}"
sending_queue:
enabled: true
persistent_storage_enabled: {{ .Values.otelevents.persistence.enabled }}
processors:
memory_limiter:
check_interval: 1s
limit_percentage: 70
spike_limit_percentage: 20
batch:
send_batch_size: 1024
send_batch_max_size: 2_048
timeout: 1s
resource/add_cluster:
attributes:
- key: cluster
value: {{ .Values.sumologic.clusterName | quote }}
action: upsert
source:
collector: {{ .Values.sumologic.collectorName | default .Values.sumologic.clusterName | quote }}
source_name: {{ .Values.fluentd.events.sourceName | quote}}
source_category: {{ .Values.fluentd.events.sourceCategory | default (printf "%s/%s" (include "sumologic.clusterNameReplaceSpaceWithDash" .) (.Values.fluentd.events.sourceName )) | quote}}
source_category_prefix: ""

service:
telemetry:
logs:
level: {{ .Values.otelevents.logLevel }}
extensions:
- health_check
{{- if .Values.otelevents.persistence.enabled }}
- file_storage
{{- end }}
- pprof
pipelines:
logs/events:
receivers:
- raw_k8s_events
processors:
- memory_limiter
- resource/add_cluster
- source
- batch
exporters:
- sumologic
48 changes: 47 additions & 1 deletion deploy/helm/sumologic/templates/_helpers.tpl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,11 @@ We truncate at 63 chars because some Kubernetes name fields are limited to this
{{- end -}}

{{- define "sumologic.labels.app.events" -}}
{{- template "sumologic.labels.app.fluentd" . }}-events
{{- if eq (include "events.fluentd.enabled" .) "true" -}}
{{ template "sumologic.labels.app.fluentd" . }}-events
{{- else if eq (include "events.otelcol.enabled" .) "true" -}}
{{ template "sumologic.labels.app.otelcol" . }}-events
{{- end -}}
{{- end -}}

{{- define "sumologic.labels.app.events.pod" -}}
Expand Down Expand Up @@ -472,7 +476,11 @@ helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
{{- end -}}

{{- define "sumologic.metadata.name.events" -}}
{{- if eq (include "events.fluentd.enabled" .) "true" -}}
{{ template "sumologic.metadata.name.fluentd" . }}-events
{{- else if eq (include "events.otelcol.enabled" .) "true" -}}
{{ template "sumologic.metadata.name.otelcol" . }}-events
{{- end -}}
{{- end -}}

{{- define "sumologic.metadata.name.events.service" -}}
Expand Down Expand Up @@ -1339,6 +1347,44 @@ Example Usage:
{{ $enabled }}
{{- end -}}

{{/*
Check if any events provider is enabled
Example Usage:
{{- if eq (include "metrics.enabled" .) "true" }}

*/}}
{{- define "events.enabled" -}}
{{- $enabled := false -}}
{{- if eq (include "events.otelcol.enabled" .) "true" }}
{{- $enabled = true -}}
{{- end -}}
{{- if eq (include "events.fluentd.enabled" .) "true" }}
{{- $enabled = true -}}
{{- end -}}
{{ $enabled }}
{{- end -}}


{{/*
Check if otelcol events provider is enabled
Example Usage:
{{- if eq (include "metrics.otelcol.enabled" .) "true" }}

*/}}
{{- define "events.otelcol.enabled" -}}
{{- eq .Values.otelevents.enabled true -}}
{{- end -}}


{{/*
Check if fluentd events provider is enabled
Example Usage:
{{- if eq (include "metrics.fluentd.enabled" .) "true" }}

*/}}
{{- define "events.fluentd.enabled" -}}
{{- eq .Values.fluentd.events.enabled true -}}
{{- end -}}

{{/*
Add service labels
Expand Down
8 changes: 6 additions & 2 deletions ...ates/events/fluentd/service-headless.yaml → ...lates/events/common/service-headless.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if eq .Values.fluentd.events.enabled true }}
{{- if eq (include "events.enabled" .) "true" }}
apiVersion: v1
kind: Service
metadata:
Expand All @@ -18,4 +18,8 @@ spec:
port: 24231
targetPort: 24231
protocol: TCP
{{- end }}
- name: otelcol-metrics
port: 8888
targetPort: 8888
protocol: TCP
{{- end }}
8 changes: 6 additions & 2 deletions ...gic/templates/events/fluentd/service.yaml → ...ogic/templates/events/common/service.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if eq .Values.fluentd.events.enabled true }}
{{- if eq (include "events.enabled" .) "true" }}
apiVersion: v1
kind: Service
metadata:
Expand All @@ -18,4 +18,8 @@ spec:
port: 24231
targetPort: 24231
protocol: TCP
{{- end }}
- name: otelcol-metrics
port: 8888
targetPort: 8888
protocol: TCP
{{- end }}
6 changes: 3 additions & 3 deletions deploy/helm/sumologic/templates/events/fluentd/configmap.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if eq .Values.fluentd.events.enabled true }}
{{- if eq (include "events.fluentd.enabled" .) "true" }}
apiVersion: v1
kind: ConfigMap
metadata:
Expand All @@ -9,6 +9,6 @@ metadata:
data:
fluent.conf: |-
@include events.conf
{{- (tpl (.Files.Glob "conf/events/*.conf").AsConfig .) | nindent 2 }}
{{- (tpl (.Files.Glob "conf/events/fluentd/*.conf").AsConfig .) | nindent 2 }}
{{- (tpl (.Files.Glob "conf/fluentd/buffer.output.conf").AsConfig .) | nindent 2 }}
{{- end }}
{{- end }}
4 changes: 2 additions & 2 deletions deploy/helm/sumologic/templates/events/fluentd/statefulset.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if eq .Values.fluentd.events.enabled true }}
{{- if eq (include "events.fluentd.enabled" .) "true" }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
Expand Down Expand Up @@ -125,4 +125,4 @@ spec:
requests:
storage: {{ .Values.fluentd.persistence.size }}
{{- end }}
{{- end }}
{{- end }}
15 changes: 15 additions & 0 deletions deploy/helm/sumologic/templates/events/otelcol/configmap.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{{- if eq (include "events.otelcol.enabled" .) "true" }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "sumologic.metadata.name.events.configmap" . }}
labels:
app: {{ template "sumologic.labels.app.events.configmap" . }}
{{- include "sumologic.labels.common" . | nindent 4 }}
data:
{{- $baseConfig := (tpl (.Files.Get "conf/events/otelcol/config.yaml") .) | fromYaml -}}
{{- $overrideConfig := .Values.otelevents.config.override -}}
{{- $finalConfig := mergeOverwrite $baseConfig $overrideConfig }}
config.yaml: |
{{- $finalConfig | toYaml | nindent 4 }}
{{- end }}
Loading

0 comments on commit c712362

Please sign in to comment.