From a0ba310ce22e70b7b0c02a6085dc445c1fd46dc6 Mon Sep 17 00:00:00 2001 From: Raj Nishtala Date: Thu, 23 Jan 2025 11:43:52 -0500 Subject: [PATCH 1/3] Add note to update CRDs before upgrading to helm chart version 4.12.0 --- .../send-data/kubernetes/v4/how-to-upgrade.md | 39 +++++++++++++++++++ .../kubernetes/v4/important-changes.md | 30 +++++++++----- 2 files changed, 60 insertions(+), 9 deletions(-) diff --git a/docs/send-data/kubernetes/v4/how-to-upgrade.md b/docs/send-data/kubernetes/v4/how-to-upgrade.md index fe9fbfae84..65e84087fb 100644 --- a/docs/send-data/kubernetes/v4/how-to-upgrade.md +++ b/docs/send-data/kubernetes/v4/how-to-upgrade.md @@ -88,6 +88,23 @@ kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry- kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetrycollector.yaml` ``` +:::note +Starting v4.12.0, please use the CRDs below +::: + +```shell +kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/refs/tags/v4.12.0/deploy/helm/sumologic/crds/crd-opentelemetry.io_opampbridges.yaml --force-conflicts + +kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/refs/tags/v4.12.0/deploy/helm/sumologic/crds/crd-opentelemetrycollector.yaml --force-conflicts + +kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/refs/tags/v4.12.0/deploy/helm/sumologic/crds/crd-opentelemetryinstrumentation.yaml --force-conflicts + +kubectl annotate crds instrumentations.opentelemetry.io opentelemetrycollectors.opentelemetry.io opampbridges.opentelemetry.io \ + meta.helm.sh/release-name=${RELEASE_NAME} \ + meta.helm.sh/release-namespace=${RELEASE_NAMESPACE} +kubectl label crds instrumentations.opentelemetry.io opentelemetrycollectors.opentelemetry.io opampbridges.opentelemetry.io app.kubernetes.io/managed-by=Helm +``` + ### How to revert to the v3 defaults Set the following in your configuration: @@ -108,6 +125,28 @@ kube-prometheus-stack: enabled: true ``` +Starting v4.12.0, please use the configuration below + +```yaml +sumologic: + metrics: + collector: + otelcol: + enabled: false + remoteWriteProxy: + enabled: true + +kube-prometheus-stack: + prometheus: + enabled: true + prometheusOperator: + enabled: true + +opentelemetry-operator: + crds: + create: true +``` + ## Remove remaining Fluent Bit and Fluentd configuration If you've already switched to OTel, skip straight to the [next major section](#switch-to-otlp-sources). diff --git a/docs/send-data/kubernetes/v4/important-changes.md b/docs/send-data/kubernetes/v4/important-changes.md index 799c97bf0a..35249a22d0 100644 --- a/docs/send-data/kubernetes/v4/important-changes.md +++ b/docs/send-data/kubernetes/v4/important-changes.md @@ -33,21 +33,33 @@ By default, the OpenTelemetry Collector is now used for metrics collection inste Ensure that the following CRDs from the OpenTelemetry operator are installed and updated using the following commands. -##### Instrumentation CRD (relevant for metrics and traces) +:::note +Please follow instructions below to install the appropriate CRD versions +::: -``` -kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetryinstrumentation.yaml -``` +#### CRDS to install (v4.12.0 and later) -##### OpenTelemetry collector CRD +```shell +kubectl delete crd instrumentations.opentelemetry.io -``` -kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetrycollector.yaml -``` +kubectl delete crd opentelemetrycollectors.opentelemetry.io -##### OpAMP Bridge CRD +kubectl delete crd opampbridges.opentelemetry.io +kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/refs/tags/v4.12.0/deploy/helm/sumologic/crds/crd-opentelemetry.io_opampbridges.yaml --force-conflicts + +kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/refs/tags/v4.12.0/deploy/helm/sumologic/crds/crd-opentelemetrycollector.yaml --force-conflicts + +kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/refs/tags/v4.12.0/deploy/helm/sumologic/crds/crd-opentelemetryinstrumentation.yaml --force-conflicts ``` + +#### CRDs to install (before v4.12.0) + +```shell +kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetryinstrumentation.yaml + +kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetrycollector.yaml + kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetry.io_opampbridges.yaml ``` From 47b521525cdb8ff47e9d0f5c12ca1b5c2b6a781c Mon Sep 17 00:00:00 2001 From: Raj Nishtala Date: Thu, 13 Feb 2025 11:29:05 -0500 Subject: [PATCH 2/3] Add a note about recovering from hung CRDs --- .../kubernetes/troubleshoot-collection.md | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/docs/send-data/kubernetes/troubleshoot-collection.md b/docs/send-data/kubernetes/troubleshoot-collection.md index 92549662b8..896c2a214d 100644 --- a/docs/send-data/kubernetes/troubleshoot-collection.md +++ b/docs/send-data/kubernetes/troubleshoot-collection.md @@ -1005,6 +1005,32 @@ It means that Custom Resource Definition has not been applied by Helm. It is [He kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.44.0/charts/opentelemetry-operator/crds/crd-opentelemetry.io_opampbridges.yaml ``` +### Hung/Stuck OpenTelemetryCollector K8s CRD + +To patch and delete CustomResourceDefinitions in Kubernetes + +```shell +kubectl patch crd/opentelemetrycollectors.opentelemetry.io -p '{"metadata":{"finalizers":[]}}' --type=merge +``` + +Confirm that the finalizer has been removed + +```shell +kubectl get opentelemetrycollectors.opentelemetry.io -o yaml > my-resource.yaml +``` + +If the patch command doesn't work, please edit the CRD to remove the finalizer + +```shell +kubectl edit crd opentelemetrycollectors.opentelemetry.io +``` + +Finally, delete the CRD + +```shell +kubectl delete crd/opampbridges.opentelemetry.io +``` + ## Using Sumo Logic Mock Sumo Logic Mock is debugging tool, which helps to see what exactly is being sent from the Sumo Logic Collection to Sumo Logic. It may help with finding if there are any missing metrics, logs, traces, or parts of them like labels or metadata. From daedffa73c52021c91da8c090b8980c9281b382a Mon Sep 17 00:00:00 2001 From: Raj Nishtala Date: Thu, 13 Feb 2025 13:11:53 -0500 Subject: [PATCH 3/3] Re-organize the notes about upgrading the CRDs --- .../send-data/kubernetes/v4/how-to-upgrade.md | 24 ++++++++++++------- .../kubernetes/v4/important-changes.md | 11 ++++++++- 2 files changed, 26 insertions(+), 9 deletions(-) diff --git a/docs/send-data/kubernetes/v4/how-to-upgrade.md b/docs/send-data/kubernetes/v4/how-to-upgrade.md index 65e84087fb..000419a8bf 100644 --- a/docs/send-data/kubernetes/v4/how-to-upgrade.md +++ b/docs/send-data/kubernetes/v4/how-to-upgrade.md @@ -80,14 +80,6 @@ kube-prometheus-stack: #### Update custom resource definition for OpenTelemetry operator -```shell -kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetry.io_opampbridges.yaml - -kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetryinstrumentation.yaml - -kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetrycollector.yaml` -``` - :::note Starting v4.12.0, please use the CRDs below ::: @@ -98,13 +90,29 @@ kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumol kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/refs/tags/v4.12.0/deploy/helm/sumologic/crds/crd-opentelemetrycollector.yaml --force-conflicts kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/refs/tags/v4.12.0/deploy/helm/sumologic/crds/crd-opentelemetryinstrumentation.yaml --force-conflicts +``` +Then, annotate and label these CRDs as below + +```shell kubectl annotate crds instrumentations.opentelemetry.io opentelemetrycollectors.opentelemetry.io opampbridges.opentelemetry.io \ meta.helm.sh/release-name=${RELEASE_NAME} \ meta.helm.sh/release-namespace=${RELEASE_NAMESPACE} kubectl label crds instrumentations.opentelemetry.io opentelemetrycollectors.opentelemetry.io opampbridges.opentelemetry.io app.kubernetes.io/managed-by=Helm ``` +:::note +CRDs prior to v4.12.0 are below +::: + +```shell +kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetry.io_opampbridges.yaml + +kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetryinstrumentation.yaml + +kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-helm-charts/opentelemetry-operator-0.56.1/charts/opentelemetry-operator/crds/crd-opentelemetrycollector.yaml` +``` + ### How to revert to the v3 defaults Set the following in your configuration: diff --git a/docs/send-data/kubernetes/v4/important-changes.md b/docs/send-data/kubernetes/v4/important-changes.md index 35249a22d0..2f2fb78494 100644 --- a/docs/send-data/kubernetes/v4/important-changes.md +++ b/docs/send-data/kubernetes/v4/important-changes.md @@ -37,7 +37,7 @@ Ensure that the following CRDs from the OpenTelemetry operator are installed and Please follow instructions below to install the appropriate CRD versions ::: -#### CRDS to install (v4.12.0 and later) +#### CRDs to install (v4.12.0 and later) ```shell kubectl delete crd instrumentations.opentelemetry.io @@ -53,6 +53,15 @@ kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumol kubectl apply --server-side -f https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/refs/tags/v4.12.0/deploy/helm/sumologic/crds/crd-opentelemetryinstrumentation.yaml --force-conflicts ``` +Then, annotate and label these CRDs as below + +```shell +kubectl annotate crds instrumentations.opentelemetry.io opentelemetrycollectors.opentelemetry.io opampbridges.opentelemetry.io \ + meta.helm.sh/release-name=${RELEASE_NAME} \ + meta.helm.sh/release-namespace=${RELEASE_NAMESPACE} +kubectl label crds instrumentations.opentelemetry.io opentelemetrycollectors.opentelemetry.io opampbridges.opentelemetry.io app.kubernetes.io/managed-by=Helm +``` + #### CRDs to install (before v4.12.0) ```shell