Skip to content

user.id silently dropped when GENERATE_CONTENT_EVENT_ONLY_EXTRA_ATTRIBUTES_CONTEXT_KEY is missing #6361

Description

@DmitriiK

Summary

When opentelemetry-instrumentation-google-genai is installed (Vertex Agent Engine / Gemini path), ADK collects session.user_id into log_only_extra_attributes but never emits user.id on spans or GenAI log events. The hand-off depends on a context key that the instrumentation package does not export (through 1.0b0). ADK catches the import failure and silently continues.

This contradicts the documented expectation that user.id appears on generate_content telemetry when content capture is enabled (OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, with OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental).

Environment

  • google-adk 2.1.0 and 2.4.0 (same code path; also present on main as of 2026-07-10)
  • opentelemetry-instrumentation-google-genai 0.7b1 and 1.0b0
  • Vertex AI Agent Engine + local runs with GoogleGenAiSdkInstrumentor active

Call chain

Runner.run_async(user_id=...)
  → Session.user_id / InvocationContext
  → use_inference_span(...)
       log_only_common_attributes[USER_ID] = session.user_id
  → _use_extra_generate_content_attributes(..., log_only_extra_attributes=...)
       try:
         from opentelemetry.instrumentation.google_genai import
             GENERATE_CONTENT_EVENT_ONLY_EXTRA_ATTRIBUTES_CONTEXT_KEY
       except (ImportError, AttributeError):
         pass   # ← user.id discarded with no warning
  → GoogleGenAiSdkInstrumentor
       only reads GENERATE_CONTENT_EXTRA_ATTRIBUTES_CONTEXT_KEY
  → generate_content {model} span / Cloud Trace   ← no user.id

Relevant code: src/google/adk/telemetry/tracing.py_use_extra_generate_content_attributes.

Root cause

opentelemetry-instrumentation-google-genai exports only:

  • GENERATE_CONTENT_EXTRA_ATTRIBUTES_CONTEXT_KEY

It does not export GENERATE_CONTENT_EVENT_ONLY_EXTRA_ATTRIBUTES_CONTEXT_KEY (proposed in open-telemetry/opentelemetry-python-genai#24, closed without merge).

ADK puts user.id only in the EVENT_ONLY channel (intentionally “log-only” / PII-sensitive), so there is no fallback onto the working span extras key.

Expected

user.id should appear on GenAI telemetry (at least on allowlisted log/events, and ideally discoverable in Cloud Trace) when session.user_id is set and content capture is enabled.

Actual

user.id is absent from every span/event produced via the instrumentor path. Failure is silent (except: pass).

Suggested fix (ADK-side, until instrumentation ships the key)

In _use_extra_generate_content_attributes, when the EVENT_ONLY import fails, merge log_only_extra_attributes into the existing span extras dict (GENERATE_CONTENT_EXTRA_ATTRIBUTES_CONTEXT_KEY) and/or log a clear warning that user.id could not be propagated.

Trade-off: putting user.id on spans is slightly broader than “event-only”, but is far better than silently dropping identity for all Agent Engine Trace users. Longer-term, either:

  1. instrumentation exports and honors the EVENT_ONLY key, or
  2. ADK adopts a LogRecordProcessor-based propagation (related attempt: fix(otel): Save user.id in opentelemetry-instrumentation-google-genai emitted logs #6217, not on a released tag / main as of this report).

Repro

  1. Install ADK ≥ 2.1 + opentelemetry-instrumentation-google-genai.
  2. Enable instrumentor; set OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental and OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=EVENT_ONLY.
  3. Run an agent with a non-default user_id; export spans (e.g. InMemorySpanExporter).
  4. Observe generate_content … spans: agent/conversation attrs present, user.id missing.

We verified a one-line-style fallback (merge into span extras on ImportError) restores user.id on generate_content spans in production Cloud Trace.

Happy to open a PR with that fallback if maintainers prefer an ADK-side mitigation while the instrumentation gap remains.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions