fix(telemetry): merge resource attrs onto wrapper-installed TracerProvider#82
Merged
LucasAlvesSoares merged 1 commit intoSAP:mainfrom Apr 27, 2026
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Description
When
auto_instrument()runs on a Python process whose OTelTracerProviderwas already installed by an upstream auto-instrumentation wrapper (e.g. an OpenTelemetry-Operator-injected init-container, common on managed-Kubernetes runtimes — SAP App Foundation's runtime is one such environment, see App Foundation runtime trigger path below for the trigger path),Traceloop.init'sset_tracer_providercall silently no-ops because OTel's_TRACER_PROVIDER_SET_ONCEguard is already tripped (first set wins, nooverrideparameter exists upstream). The resource attrs we pass toTraceloop.initnever reach the globally active provider, sosap.cloud_sdk.*,sap.solution_area,mlflow.experiment_id,sap.cld.*,deployment.environment.name, andcloud.regionare missing from emitted spans.This PR adds a
_merge_resource_attrs_into_active_provider_if_wrapper_installed(...)helper called fromauto_instrument()immediately afterTraceloop.init. It detects the wrapper-installed provider via the standard upstream markertelemetry.auto.versionon itsResource, and merges the sap-cloud-sdk resource attrs onto it viaprovider._resource = provider.resource.merge(Resource.create(sap_attrs)). Right-side wins on collisions, so SAP-supplied keys (e.g.service.namefromAPPFND_CONHOS_APP_NAME) override colliding operator-supplied keys.When no wrapper is detected (proxy default, or a self-installed
TracerProviderwithout the marker), the helper is a no-op — existing single-tenant flows are unchanged. No new public API; no parameter additions toauto_instrument(). Existing callers pick up the fix transparently on upgrade.App Foundation runtime trigger path
The wrapper injection that triggers this bug for SAP App Foundation tenants follows this chain. Tracking ticket on the platform side: AFSDK-2840.
ci-cd-workflow/.github/actions/detect-otel-runtime/detect.pyauto-detects Python frompyproject.toml/requirements.txt, theninject-otel-app-yaml/inject.py:39-47stampsotel.instrumentation/enabled: pythononto the workload CR'smetadata.labels. Default-on for every Python workload.helm-templates/charts/agent/templates/_helpers.tpl:170-193propagates the CR label onto both the renderedDeployment.metadata.labelsand theDeployment.spec.template.metadata.labels. The chart comment explicitly says "Kyverno matches Deployment labels".otel-inject-python-podmatches the pod label and triggers the OpenTelemetry-Operator webhook to attach an init-container that copies its bundled OTel Python SDK to/otel-auto-instrumentation-python, prepends that path toPYTHONPATH, and stampstelemetry.auto.versionon the activeTracerProvider'sResource. By the timeauto_instrument()runs,set_tracer_providerhas already been called — and the bug above kicks in.The fix in this PR neutralises step 3's effect on resource attrs without requiring any change at steps 1–2 (which are platform-owned and out of scope for this repo).
Related Issue
Closes #81
Type of Change
Please check the relevant option:
How to Test
Unit tests:
git checkout fix/telemetry-merge-resource-with-wrapper-provideruv run pytest tests/core/unit/telemetry/test_auto_instrument.py -vtest_auto_instrument_merges_resource_when_wrapper_installed,test_auto_instrument_skips_merge_when_no_wrapper_marker,test_auto_instrument_merge_overrides_colliding_service_name).End-to-end on a wrapper-injected pod (e.g. App Foundation runtime):
instrumentation.opentelemetry.io/inject-pythoninit-container.sap_cloud_sdk.core.telemetry.auto_instrument()from the agent's startup path (as usual).trace.get_tracer_provider().resource.attributes— e.g. via a/_debug/tracingroute or a print statement.Resourcecarries both the operator-supplied attrs (telemetry.auto.version,k8s.*,service.namespace, …) and sap-cloud-sdk's full enrichment (sap.cloud_sdk.*,sap.solution_area,mlflow.experiment_id,sap.cld.*,deployment.environment.name,cloud.region). Collidingservice.nameis won by SAP side. Validated on a deployed App Foundation buyer-agent — see Additional Notes below for the before/after JSON.Lint, format, type-check, coverage:
uv run ruff check .— clean.uv run ruff format --check --diff .— clean (tests/excluded per the existingpyproject.toml [tool.ruff].exclude).uv run ty check --output-format github --python-version 3.11 .— exit 0.uv run pytest -m "not integration" --cov=src/sap_cloud_sdk— 1160 passed, 80 deselected, total coverage 93%;auto_instrument.pyat 93%.Checklist
Before submitting your PR, please review and check the following:
test_auto_instrument.py)sap_attrs: dict -> None)fix(telemetry): merge resource attrs onto wrapper-installed TracerProviderBreaking Changes
None. The merge is auto-detected via the upstream
telemetry.auto.versionmarker, which is only present when an OTel-Operator-class wrapper has installed the active provider. Self-installed user providers don't carry this marker, so the helper short-circuits and behaviour is identical to today.auto_instrument()'s signature is unchanged.Additional Notes
Validation evidence — before/after on a wrapper-active pod
Resource Attributes on a deployed App Foundation Kyma pod with OpenTelemetry-Operator-injected auto-instrumentation active.
Before —
auto_instrument()ran but resource attrs were dropped (15 attrs, no SAP enrichment):{ "resource_attribute_count": 15, "resource_attributes": { "telemetry.sdk.language": "python", "telemetry.sdk.name": "opentelemetry", "telemetry.sdk.version": "1.41.1", "service.version": "0.0.1", "sap.service.display_name": "buyer-agent-evals-fina", "k8s.container.name": "buyer-agent-evals-fina", "k8s.deployment.name": "buyer-agent-evals-fina-deployment", "k8s.namespace.name": "buyer-agent-evals-fsmcba", "k8s.node.name": "ip-10-250-152-52.eu-central-1.compute.internal", "k8s.pod.name": "buyer-agent-evals-fina-deployment-97d4f6795-7r2vz", "k8s.replicaset.name": "buyer-agent-evals-fina-deployment-97d4f6795", "service.instance.id": "buyer-agent-evals-fsmcba.buyer-agent-evals-fina-deployment-97d4f6795-7r2vz.buyer-agent-evals-fina", "service.namespace": "buyer-agent-evals-fsmcba", "service.name": "buyer-agent-evals-fina-deployment", "telemetry.auto.version": "0.62b1" } }mlflow.experiment_id,sap.cloud_sdk.*,sap.solution_area,sap.cld.*,deployment.environment.name,cloud.region— all absent.service.nameis the operator'sOTEL_SERVICE_NAMEvalue (<appname>-deployment) rather thanAPPFND_CONHOS_APP_NAME.After — same pod, with this PR's merge fix applied (24 attrs, full SAP enrichment):
{ "resource_attribute_count": 24, "resource_attributes": { "telemetry.sdk.language": "python", "telemetry.sdk.name": "opentelemetry", "telemetry.sdk.version": "1.41.1", "service.version": "0.0.1", "sap.service.display_name": "buyer-agent-evals-fina", "k8s.container.name": "buyer-agent-evals-fina", "k8s.deployment.name": "buyer-agent-evals-fina-deployment", "k8s.namespace.name": "buyer-agent-evals-fsmcba", "k8s.node.name": "ip-10-250-84-103.eu-central-1.compute.internal", "k8s.pod.name": "buyer-agent-evals-fina-deployment-7746997958-rbh6z", "k8s.replicaset.name": "buyer-agent-evals-fina-deployment-7746997958", "service.instance.id": "buyer-agent-evals-fina-deployment-7746997958-rbh6z", "service.namespace": "buyer-agent-evals-fsmcba", "service.name": "buyer-agent-evals-fina", "telemetry.auto.version": "0.62b1", "deployment.environment.name": "canary", "cloud.region": "eu12", "sap.cld.subaccount_id": "ac5d447e-af4b-4c64-abf5-90e71f770c07", "sap.cld.system_role": "unknown", "sap.cloud_sdk.name": "SAP Cloud SDK for Python", "sap.cloud_sdk.language": "python", "sap.cloud_sdk.version": "0.11.6", "sap.solution_area": "fina", "mlflow.experiment_id": "1635264705567712" } }Operator-supplied
k8s.*,service.namespace,service.version,sap.service.display_name,telemetry.*all preserved. SAP attrs added on top.service.namecollision won by SAP side —buyer-agent-evals-finafromAPPFND_CONHOS_APP_NAMEdrops the redundant-deploymentsuffix the operator picked up from the k8s deployment name.Design notes for reviewers
Traceloop.initstill runs. Even when the merge will fire, we keep theTraceloop.initcall above untouched — it's the only way to install Traceloop's third-party autoinstrumentation hooks (langchain, openai, …) which emit spans viatrace.get_tracer(...)and flow through whichever provider is globally active. Cost in the wrapper-active case is one orphanedTracerProviderallocation at startup; acceptable.Files changed
src/sap_cloud_sdk/core/telemetry/auto_instrument.py— addsMappingandResourceimports; calls the new helper afterTraceloop.init; defines_merge_resource_attrs_into_active_provider_if_wrapper_installed.tests/core/unit/telemetry/test_auto_instrument.py— three new tests covering merge-fires, marker-absent-skip, and SAP-side-wins-on-collision.pyproject.toml+uv.lock— version0.11.6→0.11.7(required bycheck-version-bump.yamlwhensrc/changes).