feat(tracing): propagate OTel trace context across Temporal boundaries

[NiteshDhanpal]

Problem

Temporal breaks OpenTelemetry context propagation. A start_workflow / execute_activity call is serialized and dispatched to a (potentially different) worker process, and the active W3C traceparent is not carried across that boundary. So any span created inside a workflow or activity becomes a new detached root — the trace shatters at every Temporal hop.

This bites agentex directly: adk.tracing.span branches on in_temporal_workflow() and, when true, creates the business span as a Temporal activity (TracingActivityName.START_SPAN). Without propagation, those business spans (and any downstream spans) detach from the turn's obs trace.

The existing ContextInterceptor threads business context (task_id) across the boundary — but nothing threads the obs-trace (W3C) context.

Change

Wire temporalio's first-party temporalio.contrib.opentelemetry.TracingInterceptor onto our Temporal client factories and worker, so client → workflow → activity is one trace. It injects the active span context into Temporal headers on outbound calls, and extracts + roots the workflow/activity execution spans under it, using the global OpenTelemetry propagator.

Wired at three points (both client factories + the worker):

• core/temporal/workers/worker.py — get_temporal_client (worker's client) and the Worker(interceptors=...) (inbound execution, tracing outermost so business interceptors nest under it).
• core/clients/temporal/utils.py — get_temporal_client used by TemporalClient (the ACP's workflow-starting client — the critical outbound start_workflow propagation point).
• New core/tracing/temporal.py — temporal_tracing_interceptors() helper so every client/worker wires it identically.

Default-on with an opt-out

Enabled by default. Opt out with AGENTEX_TEMPORAL_TRACE_INTERCEPTOR_ENABLED=false (also accepts 0 / no / off). It also degrades to a no-op and never raises if temporalio's OpenTelemetry contrib isn't importable — so default-on can't break a worker. (adk already depends on opentelemetry-api/sdk, so the contrib is available in practice.)

Ordering (important)

On the worker the tracing interceptor is outermost:

TracingInterceptor          ← continues/roots the workflow/activity span first
  └─ ContextInterceptor     ← threads task_id, creates business spans as children
       └─ handler

If reversed, business spans would be created before the trace context is active → back to detached roots.

Tests

tests/lib/core/tracing/test_temporal_interceptor.py — default-on returns a TracingInterceptor, env opt-out returns [], and the contrib-missing fallback returns [] (verified against the real temporalio contrib).

Note: a full workflow→activity propagation assertion (same trace_id across the boundary) belongs in a Temporal-test-env integration test — noted as a follow-up; this PR unit-tests the wiring/flag/fallback logic.

Risk

Low — additive, config-gated, no-op when disabled or when the contrib is absent; no workflow-determinism concern (TracingInterceptor is designed for the workflow sandbox). Independent of PR #484 (correlation edge) — branched off main.

Relation to platform tracing

This is the Temporal-boundary piece of "W3C propagation everywhere": once it lands, an agent turn (or any workflow) that fans out through Temporal activities stays one trace end-to-end, and the business/anchor spans created inside activities attach to the turn.

🤖 Generated with Claude Code

Greptile Summary

The PR enables OpenTelemetry trace-context propagation across Temporal boundaries.

• Adds a shared, default-on tracing-interceptor factory with an environment-variable opt-out and no-op fallback.
• Installs tracing on both Temporal client factories and ahead of business interceptors on workers.
• Adds unit coverage for default enablement, opt-out values, and unavailable-contrib fallback.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the constructor now executes inside the guarded try block, so ordinary constructor failures follow the documented no-op fallback instead of aborting Temporal startup.

Important Files Changed

Filename	Overview
src/agentex/lib/core/tracing/temporal.py	Adds the shared interceptor factory and now correctly catches both import and constructor failures, resolving the prior startup-failure concern.
src/agentex/lib/core/clients/temporal/utils.py	Installs the tracing interceptor on the application Temporal client factory.
src/agentex/lib/core/temporal/workers/worker.py	Installs tracing on the worker client and places the worker tracing interceptor before existing business interceptors.
tests/lib/core/tracing/test_temporal_interceptor.py	Covers default enablement, supported opt-out values, non-falsy values, and unavailable-contrib fallback behavior.

Sequence Diagram

sequenceDiagram
    participant C as Temporal Client
    participant TW as Temporal Workflow Worker
    participant A as Temporal Activity
    C->>TW: start_workflow + injected OTel context
    activate TW
    TW->>A: execute_activity + propagated OTel context
    activate A
    A-->>TW: activity result
    deactivate A
    TW-->>C: workflow result
    deactivate TW

Loading

_{Reviews (2): Last reviewed commit: "feat(tracing): propagate OTel trace cont..." | Re-trigger Greptile}

[harvhan]

This double installs the tracing interceptor on the worker. Worker.init prepends client interceptors that also implement temporalio.worker.Interceptor, and TracingInterceptor implements both, so the instance already on this worker's client (from get_temporal_client above) applies to the worker automatically. With this line the worker runs two instances. Verified with a local repro on temporalio 1.26.0: every worker side span (RunWorkflow, CompleteWorkflow, StartActivity, RunActivity) is emitted twice, while client only wiring emits each span once with propagation intact.

Suggest interceptors=self.interceptors. The inherited instance is prepended, so tracing still sits outermost ahead of the business interceptors.