-
Notifications
You must be signed in to change notification settings - Fork 414
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is there a way with the OpenTelemetry Operator Auto-Instrumentation to instrument multiple languages in the same Pod #1717
Comments
Unfortunately the Instrumentation resource cannot do what you need, but I think it is a great use case we should allow. Feels like the Instrumentation should be able to handle language-specific container names like |
I started work on this today but ran into a snag with the initContainers. We restrict ourselves to 1 right now, which isn't a strict requirement, but before I dive down that rabbit hole: @jack78901 have you tried adding the auto-instrumentation to your pod manually? Instead of using the operator, have you tried setting up your deployment so it has java initConatiners and nodejs initContainers and all the proper volume mounts and env vars? I want to confirm that what we're trying to do here is even possible with the way the different language auto-instrumentation work with initContainers. |
@TylerHelmuth I had a similar issue in the past. As far as I can remember, sdkInjector.inject in instPodMutator.Mutate cannot be executed twice. This is caused by a few append calls that do not check for an existing entry. E.g.: opentelemetry-operator/pkg/instrumentation/javaagent.go Lines 55 to 58 in 87d4e10
Once this method can be executed multiple times and leads to the same result we can also continue with #553 😮💨 |
@frzifus I think we can definitely handle that blocker, especially if we gave each language its own volume mount and path. I'm more concerned tho that the languages will interfere with each other if we try to auto-instrument multiple languages. I'll try it out myself soon. |
@TylerHelmuth, I apologize. I missed in your message about testing the manual configuration to see if the multiple instrumentation will work. I will see if we can try this in our dev environment and see what results. Thank you for looking into this! |
Hey @jack78901 @TylerHelmuth , |
Closed by #1901, please let me know if that solution is not sufficient! |
I have an application running in Kubernetes that has a pod that has multiple containers. Each container performs part of a task. Namely, one is the API (written in node.js), a backend transport (written in java), and a drive connector (written in C).
When attempting to add multiple annotations for instrumentation, we get the error:
44s Warning FailedCreate statefulset/my-pod-with-multiple-containers create Pod my-pod-with-multiple-containers-1 in StatefulSet my-pod-with-multiple-containers failed error: Pod “my-pod-with-multiple-containers-1” is invalid: spec.containers[1].volumeMounts[2].mountPath: Invalid value: "/otel-auto-instrumentation": must be unique
I know that with OpenTelemetry, you can annotate a pod to do auto-instrumentation, such as:
apiVersion: apps/v1
kind: Statefulset
metadata:
name: my-deployment-with-multiple-containers
spec:
selector:
matchLabels:
app: my-pod-with-multiple-containers
replicas: 1
template:
metadata:
labels:
app: my-pod-with-multiple-containers
annotations:
instrumentation.opentelemetry.io/inject-java: "true"
instrumentation.opentelemetry.io/container-names: "myapp,myapp2"
spec:
containers:
- name: myapp
image: myImage1
- name: myapp2
image: myImage2
- name: myapp3
image: myImage3
The above (taken from https://github.com/open-telemetry/opentelemetry-operator/blob/main/README.md#multi-container-pods), myapp and myapp2 would get java auto-instrumentation injected.
We tried doing:
We were hoping this would make it so that myapp would get Java instrumentation and myapp2 would get node.js instrumentation.
I was informed to move this here from Stack Overflow: https://stackoverflow.com/questions/75943971/is-there-a-way-with-the-opentelemetry-operator-auto-instrumentation-to-instrumen
The text was updated successfully, but these errors were encountered: