feat(gemini): migrate google-generativeai to latest OTel GenAI semantic conventions#3840
Merged
OzBenSimhonTraceloop merged 24 commits intomainfrom Apr 9, 2026
Merged
feat(gemini): migrate google-generativeai to latest OTel GenAI semantic conventions#3840OzBenSimhonTraceloop merged 24 commits intomainfrom
OzBenSimhonTraceloop merged 24 commits intomainfrom
Conversation
|
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR updates semantic attributes across Google Generative AI and Vertex AI instrumentations by replacing Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
…semantic conventions
avivhalfon
force-pushed
the
ah/gemini-sem-conv
branch
from
7c8d572 to
f839c29
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py (1)
418-433:⚠️ Potential issue | 🟠 MajorAdd required
gen_ai.operation.nameattribute to token usage histogram.The
gen_ai.client.token.usagemetric requiresgen_ai.operation.namealongside provider and token type per the OpenTelemetry GenAI specification. Without it, token usage from all operations collapses into the same metric series. AddGenAIAttributes.GEN_AI_OPERATION_NAME: "generate_content"to both histogram record calls.Diff
attributes={ + GenAIAttributes.GEN_AI_OPERATION_NAME: "generate_content", GenAIAttributes.GEN_AI_PROVIDER_NAME: "gcp.gen_ai", GenAIAttributes.GEN_AI_TOKEN_TYPE: "input", GenAIAttributes.GEN_AI_RESPONSE_MODEL: llm_model, } @@ attributes={ + GenAIAttributes.GEN_AI_OPERATION_NAME: "generate_content", GenAIAttributes.GEN_AI_PROVIDER_NAME: "gcp.gen_ai", GenAIAttributes.GEN_AI_TOKEN_TYPE: "output", GenAIAttributes.GEN_AI_RESPONSE_MODEL: llm_model, },🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py` around lines 418 - 433, The token usage histogram calls in span_utils.py are missing the required GenAIAttributes.GEN_AI_OPERATION_NAME attribute; update both token_histogram.record invocations (the ones using response.usage_metadata.prompt_token_count and response.usage_metadata.candidates_token_count) to include GenAIAttributes.GEN_AI_OPERATION_NAME: "generate_content" alongside the existing provider, token type and GenAIAttributes.GEN_AI_RESPONSE_MODEL (llm_model) attributes so metrics are separated per operation.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/__init__.py`:
- Around line 206-209: Update the GenAI operation attribute from "chat" to the
semconv name "generate_content" and add that operation attribute to the duration
metric attributes so the histogram includes both gen_ai.operation.name and
gen_ai.system; specifically change the attributes dict where
GenAIAttributes.GEN_AI_OPERATION_NAME is set (currently "chat") to
"generate_content", and when building the gen_ai.client.operation.duration
histogram attributes include
GenAIAttributes.GEN_AI_OPERATION_NAME="generate_content" alongside
GenAIAttributes.GEN_AI_SYSTEM; apply the same fixes for the
AsyncModels.generate_content* and Models.generate_content* wrappers (the other
occurrences referenced near the existing attributes and the sync wrapper
sections).
In
`@packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/event_emitter.py`:
- Line 31: EVENT_ATTRIBUTES currently only sets
GenAIAttributes.GEN_AI_PROVIDER_NAME and the code emits deprecated event names;
update the non-legacy path to emit the new semantic event
"gen_ai.client.inference.operation.details" instead of "gen_ai.{role}.message"
or "gen_ai.choice" and ensure EVENT_ATTRIBUTES (and the event emission call
sites) include the required attributes: GenAIAttributes.GEN_AI_PROVIDER_NAME
(value "gcp.gen_ai") and GenAIAttributes.GEN_AI_OPERATION_NAME (attribute key
"gen_ai.operation.name") plus any other required fields from the GenAI spec;
locate EVENT_ATTRIBUTES and the emit/send calls in event_emitter.py and replace
the deprecated event names with "gen_ai.client.inference.operation.details" and
populate the attribute map with the operation name and provider before emitting.
In
`@packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py`:
- Around line 205-235: The messages payload currently uses a top-level "content"
field and JSON-encodes parts (producing double-serialization) and uses
{"type":"text","text":...} instead of the GenAI semantic convention; update the
three builders (the async input block that calls _process_content_item and
_process_argument, the sync input block with the same pattern, and the output
builder that uses response.text) to construct each message as {"role": <role>,
"parts": [ ... ]} where each text part is {"type":"text","content": "<plain
string>"} (do not json.dumps the inner parts — only json.dumps the final
messages list when calling _set_span_attribute). Locate and change the code that
currently creates {"role": ..., "content": json.dumps(...)} to instead build a
parts list from processed_content (convert strings to [{"type":"text","content":
...}] and ensure processed_content lists/dicts map to parts with "content"
fields) and pass that structure into _set_span_attribute with
GEN_AI_INPUT_MESSAGES (and make the analogous change for output messages that
used response.text).
In
`@packages/opentelemetry-instrumentation-google-generativeai/tests/test_generate_content.py`:
- Around line 49-59: The current test only asserts message role and misses
validating the required 'parts' structure; update the assertions after loading
input_messages and output_messages (the variables in this diff) to assert that
each message has a 'parts' field that is a non-empty list and that parts[0]
contains 'type' and 'content' keys (e.g., assert parts exists, len(parts) > 0,
parts[0]["type"] and parts[0]["content"] are present/non-empty) for both
input_messages[0] and output_messages[0] to enforce the OpenTelemetry GenAI
semantic convention.
In
`@packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.py`:
- Around line 239-240: Update the GenAI semantic attributes to set
GenAIAttributes.GEN_AI_PROVIDER_NAME to "gcp.vertex_ai" instead of "vertex_ai",
and compute GenAIAttributes.GEN_AI_OPERATION_NAME dynamically in both the async
wrapper and the sync wrapper (where these attributes are currently hard-coded)
by mapping the wrapped API/method name to the correct operation token
("generate_content", "text_completion", or "chat") rather than always using
"chat"; locate the assignment sites that set
GenAIAttributes.GEN_AI_PROVIDER_NAME and GenAIAttributes.GEN_AI_OPERATION_NAME
in the async wrapper and the sync wrapper and replace the hard-coded operation
with a small function or conditional that inspects the wrapped method name
(e.g., method.name or the API call identifier) to choose the correct operation.
In
`@packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/event_emitter.py`:
- Line 32: Update EVENT_ATTRIBUTES so GenAIAttributes.GEN_AI_PROVIDER_NAME is
set to the canonical value "gcp.vertex_ai" (replace the current "vertex_ai");
then remove or stop emitting the deprecated per-message event attributes (the
per-event keys like gen_ai.user.message, gen_ai.assistant.message,
gen_ai.choice) and instead ensure the instrumentation records chat history using
span attributes such as gen_ai.input.messages and gen_ai.output.messages where
the relevant code constructs span attributes for requests/responses (look for
usages in event_emitter.py that emit per-message events and change them to
populate the span attribute arrays accordingly).
In
`@packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/span_utils.py`:
- Around line 255-258: The current span_utils.py path only checks
kwargs.get("prompt") and misses when callers pass message=..., so modify the
logic in the span-attribute population to also check kwargs.get("message") and
normalize it the same way as the positional args handling (reuse the same
parts/normalization code path used for args) before serializing; specifically
update the block that builds messages and calls _set_span_attribute(span,
GEN_AI_INPUT_MESSAGES, json.dumps(messages)) to accept either prompt or message
(or fall back to the existing args-derived parts) so ChatSession.send_message*
invocations with message=... produce the same parts schema and populate
gen_ai.input.messages consistently.
- Around line 218-227: The span attribute payloads emitted by
_process_vertexai_argument/_set_span_attribute must follow GenAI semconv:
replace {"role":"user","content":...} with
{"role":"user","parts":[{"type":"text","content":...}]} (avoid double-encoding
JSON), ensure assistant outputs include a required "finish_reason" field
alongside {"role":"assistant","parts":[...,"finish_reason":...]} and that any
plain string content is wrapped into a parts array with
{"type":"text","content":...}; also update set_model_input_attributes to check
both kwargs.get("prompt") and kwargs.get("message") (the
ChatSession.send_message() keyword) so message= inputs get recorded into
GEN_AI_INPUT_MESSAGES. Locate and adjust the logic in
_process_vertexai_argument, the blocks that append messages (currently building
{"role","content"}), the output handling that emits plain strings, and the
set_model_input_attributes function to implement these changes.
---
Outside diff comments:
In
`@packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py`:
- Around line 418-433: The token usage histogram calls in span_utils.py are
missing the required GenAIAttributes.GEN_AI_OPERATION_NAME attribute; update
both token_histogram.record invocations (the ones using
response.usage_metadata.prompt_token_count and
response.usage_metadata.candidates_token_count) to include
GenAIAttributes.GEN_AI_OPERATION_NAME: "generate_content" alongside the existing
provider, token type and GenAIAttributes.GEN_AI_RESPONSE_MODEL (llm_model)
attributes so metrics are separated per operation.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: d4c2161f-a9bd-4e62-aa08-e6b3f2ae5221
⛔ Files ignored due to path filters (2)
packages/opentelemetry-instrumentation-google-generativeai/uv.lockis excluded by!**/*.lockpackages/opentelemetry-instrumentation-vertexai/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (12)
packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/__init__.pypackages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/event_emitter.pypackages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.pypackages/opentelemetry-instrumentation-google-generativeai/pyproject.tomlpackages/opentelemetry-instrumentation-google-generativeai/tests/test_generate_content.pypackages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.pypackages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/event_emitter.pypackages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/span_utils.pypackages/opentelemetry-instrumentation-vertexai/pyproject.tomlpackages/opentelemetry-instrumentation-vertexai/tests/disabled_test_bison.pypackages/opentelemetry-instrumentation-vertexai/tests/disabled_test_gemini.pypackages/opentelemetry-instrumentation-vertexai/tests/test_role_attributes.py
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.py (1)
239-240:⚠️ Potential issue | 🟠 MajorUse semconv-defined provider and operation mapping instead of hard-coded values.
GEN_AI_PROVIDER_NAMEshould use the Vertex AI well-known token, andGEN_AI_OPERATION_NAMEshould be derived from the wrapped method (generate_content*/predict*/send_message*) instead of always"chat". Current values collapse distinct operation types and skew telemetry dimensions. This also affects updated assertions inpackages/opentelemetry-instrumentation-vertexai/tests/disabled_test_bison.pyandpackages/opentelemetry-instrumentation-vertexai/tests/disabled_test_gemini.py.Proposed fix
+def _resolve_genai_operation_name(to_wrap): + method = (to_wrap or {}).get("method", "") + if method.startswith("generate_content"): + return "generate_content" + if method.startswith("predict"): + return "text_completion" + if method.startswith("send_message"): + return "chat" + return "chat" + @@ attributes={ - GenAIAttributes.GEN_AI_PROVIDER_NAME: "vertex_ai", - GenAIAttributes.GEN_AI_OPERATION_NAME: "chat", + GenAIAttributes.GEN_AI_PROVIDER_NAME: "gcp.vertex_ai", + GenAIAttributes.GEN_AI_OPERATION_NAME: _resolve_genai_operation_name( + to_wrap + ), }, @@ attributes={ - GenAIAttributes.GEN_AI_PROVIDER_NAME: "vertex_ai", - GenAIAttributes.GEN_AI_OPERATION_NAME: "chat", + GenAIAttributes.GEN_AI_PROVIDER_NAME: "gcp.vertex_ai", + GenAIAttributes.GEN_AI_OPERATION_NAME: _resolve_genai_operation_name( + to_wrap + ), },OpenTelemetry GenAI semantic conventions (latest): what are the well-known values for `gen_ai.provider.name` and `gen_ai.operation.name`, and which operation values should map to Vertex AI APIs `generate_content*`, `predict*`, and `send_message*`?Based on learnings: Follow the OpenTelemetry GenAI semantic specification at https://opentelemetry.io/docs/specs/semconv/gen-ai/
Also applies to: 288-289
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.py` around lines 239 - 240, Replace the hard-coded GenAIAttributes.GEN_AI_PROVIDER_NAME and GEN_AI_ATTRIBUTES.GEN_AI_OPERATION_NAME values with the semconv-defined provider token for Vertex AI and a derived operation mapping based on the wrapped method name: map methods starting with "generate_content" -> operation "generate", "predict" -> "predict", and "send_message" -> "chat" (use the exact OpenTelemetry GenAI semantic-convention tokens for provider and operations). Implement this logic where GenAIAttributes.GEN_AI_PROVIDER_NAME and GenAIAttributes.GEN_AI_OPERATION_NAME are set (the block that currently sets "vertex_ai" and "chat") so provider uses the canonical semconv constant and operation is chosen by checking the wrapped method name (e.g., wrapped_method_name.startswith("generate_content"), .startswith("predict"), .startswith("send_message")); update the related assertions in the disabled_test_bison.py and disabled_test_gemini.py fixtures to expect the semconv provider token and the mapped operation values. Ensure all references to the old hard-coded strings are replaced.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py`:
- Around line 209-210: The branch that handles a plain string (currently
appending {"role": "user", "content": contents}) is inconsistent with other
branches that JSON-encode an array of parts; update that branch so it
JSON-encodes the string into an array-of-parts before appending (e.g., use
json.dumps([...]) to produce the same array-of-parts shape used in the other
branches), ensuring the messages list always has content as a JSON-encoded
array; locate this change where the variable contents is handled and messages is
appended in span_utils.py and mirror the encoding logic used in the other
branches.
---
Duplicate comments:
In
`@packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.py`:
- Around line 239-240: Replace the hard-coded
GenAIAttributes.GEN_AI_PROVIDER_NAME and GEN_AI_ATTRIBUTES.GEN_AI_OPERATION_NAME
values with the semconv-defined provider token for Vertex AI and a derived
operation mapping based on the wrapped method name: map methods starting with
"generate_content" -> operation "generate", "predict" -> "predict", and
"send_message" -> "chat" (use the exact OpenTelemetry GenAI semantic-convention
tokens for provider and operations). Implement this logic where
GenAIAttributes.GEN_AI_PROVIDER_NAME and GenAIAttributes.GEN_AI_OPERATION_NAME
are set (the block that currently sets "vertex_ai" and "chat") so provider uses
the canonical semconv constant and operation is chosen by checking the wrapped
method name (e.g., wrapped_method_name.startswith("generate_content"),
.startswith("predict"), .startswith("send_message")); update the related
assertions in the disabled_test_bison.py and disabled_test_gemini.py fixtures to
expect the semconv provider token and the mapped operation values. Ensure all
references to the old hard-coded strings are replaced.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 45b5433d-73a3-4460-bc3f-35d3bba6e3af
⛔ Files ignored due to path filters (2)
packages/opentelemetry-instrumentation-google-generativeai/uv.lockis excluded by!**/*.lockpackages/opentelemetry-instrumentation-vertexai/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (12)
packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/__init__.pypackages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/event_emitter.pypackages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.pypackages/opentelemetry-instrumentation-google-generativeai/pyproject.tomlpackages/opentelemetry-instrumentation-google-generativeai/tests/test_generate_content.pypackages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.pypackages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/event_emitter.pypackages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/span_utils.pypackages/opentelemetry-instrumentation-vertexai/pyproject.tomlpackages/opentelemetry-instrumentation-vertexai/tests/disabled_test_bison.pypackages/opentelemetry-instrumentation-vertexai/tests/disabled_test_gemini.pypackages/opentelemetry-instrumentation-vertexai/tests/test_role_attributes.py
✅ Files skipped from review due to trivial changes (4)
- packages/opentelemetry-instrumentation-vertexai/pyproject.toml
- packages/opentelemetry-instrumentation-google-generativeai/pyproject.toml
- packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/event_emitter.py
- packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/event_emitter.py
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/init.py
- packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/span_utils.py
doronkopit5
reviewed
Mar 23, 2026
| GenAIAttributes.GEN_AI_SYSTEM: "Google", | ||
| SpanAttributes.LLM_REQUEST_TYPE: LLMRequestTypeValues.COMPLETION.value, | ||
| GenAIAttributes.GEN_AI_PROVIDER_NAME: "gcp.gen_ai", | ||
| GenAIAttributes.GEN_AI_OPERATION_NAME: "chat", |
Member
There was a problem hiding this comment.
use GenAiOperationNameValues.TEXT_COMPLETION.value instead of hardcoded value, and why change to chat form completion? a mistake from the beginning?
doronkopit5
reviewed
Mar 23, 2026
| attributes={ | ||
| GenAIAttributes.GEN_AI_SYSTEM: "Google", | ||
| SpanAttributes.LLM_REQUEST_TYPE: LLMRequestTypeValues.COMPLETION.value, | ||
| GenAIAttributes.GEN_AI_PROVIDER_NAME: "gcp.gen_ai", |
Member
There was a problem hiding this comment.
cant you import from GenAiSystemValues.<gcp>.value?
doronkopit5
reviewed
Mar 23, 2026
Comment on lines
+22
to
+23
| GEN_AI_INPUT_MESSAGES = "gen_ai.input.messages" | ||
| GEN_AI_OUTPUT_MESSAGES = "gen_ai.output.messages" |
Member
There was a problem hiding this comment.
import from
GenAIAttributes.GEN_AI_INPUT_MESSAGES
doronkopit5
reviewed
Mar 23, 2026
Member
There was a problem hiding this comment.
do you know why the revision changed and upload time added in many places?
doronkopit5
reviewed
Mar 23, 2026
Comment on lines
+239
to
+240
| GenAIAttributes.GEN_AI_PROVIDER_NAME: "vertex_ai", | ||
| GenAIAttributes.GEN_AI_OPERATION_NAME: "chat", |
Member
There was a problem hiding this comment.
same comments from google genai
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/span_utils.py (1)
291-298:⚠️ Potential issue | 🟠 MajorHandle keyword-only
message=inputs here.Line 291 only checks
prompt, soChatSession.send_message*calls made asmessage=...never populategen_ai.input.messageswhenargsis empty. The disabled Bison suite already exercises that form at Line 242, so this still drops user input for a common chat path. Reusing_process_vertexai_argument_sync()here also keepsmessage=normalization aligned with the new positional-input path.Minimal fix
- prompt = kwargs.get("prompt") - if prompt: - messages = [ - {"role": "user", "parts": [{"type": "text", "content": prompt}]}, - ] - _set_span_attribute( - span, GenAIAttributes.GEN_AI_INPUT_MESSAGES, json.dumps(messages) - ) + prompt_or_message = kwargs.get("prompt") + if prompt_or_message is None: + prompt_or_message = kwargs.get("message") + if prompt_or_message: + parts = _process_vertexai_argument_sync(prompt_or_message, span) + if parts: + _set_span_attribute( + span, + GenAIAttributes.GEN_AI_INPUT_MESSAGES, + json.dumps([{"role": "user", "parts": parts}]), + )#!/bin/bash set -euo pipefail echo "== current kwargs handling in set_model_input_attributes ==" sed -n '286,300p' packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/span_utils.py echo echo "== keyword-only chat call in disabled_test_bison.py ==" sed -n '238,245p' packages/opentelemetry-instrumentation-vertexai/tests/disabled_test_bison.py echo echo "== call sites of set_model_input_attributes ==" rg -n -C3 '\bset_model_input_attributes\s*\(' packages/opentelemetry-instrumentation-vertexai --type py🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/span_utils.py` around lines 291 - 298, The current handling in set_model_input_attributes only looks for kwargs.get("prompt") and misses keyword-only chat inputs passed as message=..., so update the function to also check for kwargs.get("message") (or prefer using the same normalization path) by reusing the existing _process_vertexai_argument_sync(...) logic used for positional inputs; ensure when a message keyword is present you call _process_vertexai_argument_sync(span, kwargs.get("message")) (or otherwise normalize into the same messages structure) and then set the GenAIAttributes.GEN_AI_INPUT_MESSAGES attribute via _set_span_attribute(span, GenAIAttributes.GEN_AI_INPUT_MESSAGES, json.dumps(messages)) so message=... flows produce the same gen_ai.input.messages output as prompt/positional paths.
🧹 Nitpick comments (1)
packages/opentelemetry-instrumentation-vertexai/tests/disabled_test_bison.py (1)
43-46: Decodegen_ai.input.messagesbefore asserting.These checks only look for a substring in the serialized JSON, so they still pass if the prompt lands under the wrong role/message shape. Parsing the attribute and asserting
role == "user"plus a textpartwould make these migration tests actually catch schema regressions.Small helper to make the assertions structural
+def assert_user_text_input(messages_json: str, expected_text: str): + messages = json.loads(messages_json) + assert any( + msg.get("role") == "user" + and any( + part.get("type") == "text" and part.get("content") == expected_text + for part in msg.get("parts", []) + ) + for msg in messages + ) + ... - assert ( - "Give me ten interview questions for the role of program manager." - in vertexai_span.attributes["gen_ai.input.messages"] - ) + assert_user_text_input( + vertexai_span.attributes["gen_ai.input.messages"], + "Give me ten interview questions for the role of program manager.", + )Also applies to: 84-87, 118-121, 160-163, 209-212
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/__init__.py`:
- Around line 229-235: The duration histogram is missing the required
gen_ai.operation.name attribute; update the attributes passed to
duration_histogram.record in both async and sync paths to include
GenAIAttributes.GEN_AI_OPERATION_NAME with the standard value "chat" alongside
GenAIAttributes.GEN_AI_PROVIDER_NAME (_GCP_GEN_AI) and
GenAIAttributes.GEN_AI_RESPONSE_MODEL (llm_model), ensuring the same change is
applied in the async duration_histogram.record call and the sync
duration_histogram.record call.
In
`@packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py`:
- Around line 641-657: The token counts um.prompt_token_count and
um.candidates_token_count may be None, so before calling
token_histogram.record(...) (inside the token_histogram and um is not None
block) validate each value is not None and only record when present;
specifically, check um.prompt_token_count is not None before the input-token
record and check um.candidates_token_count is not None before the output-token
record, keeping the same attributes
(GenAIAttributes.GEN_AI_PROVIDER_NAME/_GCP_GEN_AI,
GenAIAttributes.GEN_AI_TOKEN_TYPE,
GenAIAttributes.GEN_AI_RESPONSE_MODEL/llm_model).
---
Duplicate comments:
In
`@packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/span_utils.py`:
- Around line 291-298: The current handling in set_model_input_attributes only
looks for kwargs.get("prompt") and misses keyword-only chat inputs passed as
message=..., so update the function to also check for kwargs.get("message") (or
prefer using the same normalization path) by reusing the existing
_process_vertexai_argument_sync(...) logic used for positional inputs; ensure
when a message keyword is present you call _process_vertexai_argument_sync(span,
kwargs.get("message")) (or otherwise normalize into the same messages structure)
and then set the GenAIAttributes.GEN_AI_INPUT_MESSAGES attribute via
_set_span_attribute(span, GenAIAttributes.GEN_AI_INPUT_MESSAGES,
json.dumps(messages)) so message=... flows produce the same
gen_ai.input.messages output as prompt/positional paths.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b490ba55-8d7c-4495-95db-22703b9559bf
📒 Files selected for processing (10)
packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/__init__.pypackages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/event_emitter.pypackages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.pypackages/opentelemetry-instrumentation-google-generativeai/tests/test_generate_content.pypackages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.pypackages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/event_emitter.pypackages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/span_utils.pypackages/opentelemetry-instrumentation-vertexai/tests/disabled_test_bison.pypackages/opentelemetry-instrumentation-vertexai/tests/disabled_test_gemini.pypackages/opentelemetry-instrumentation-vertexai/tests/test_role_attributes.py
✅ Files skipped from review due to trivial changes (1)
- packages/opentelemetry-instrumentation-vertexai/tests/test_role_attributes.py
🚧 Files skipped from review as they are similar to previous changes (4)
- packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/event_emitter.py
- packages/opentelemetry-instrumentation-vertexai/tests/disabled_test_gemini.py
- packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/init.py
- packages/opentelemetry-instrumentation-google-generativeai/tests/test_generate_content.py
Comment on lines
+641
to
+657
| if token_histogram and um is not None: | ||
| token_histogram.record( | ||
| response.usage_metadata.prompt_token_count, | ||
| um.prompt_token_count, | ||
| attributes={ | ||
| GenAIAttributes.GEN_AI_PROVIDER_NAME: "Google", | ||
| GenAIAttributes.GEN_AI_PROVIDER_NAME: _GCP_GEN_AI, | ||
| GenAIAttributes.GEN_AI_TOKEN_TYPE: "input", | ||
| GenAIAttributes.GEN_AI_RESPONSE_MODEL: llm_model, | ||
| } | ||
| }, | ||
| ) | ||
| token_histogram.record( | ||
| response.usage_metadata.candidates_token_count, | ||
| attributes={ | ||
| GenAIAttributes.GEN_AI_PROVIDER_NAME: "Google", | ||
| GenAIAttributes.GEN_AI_TOKEN_TYPE: "output", | ||
| GenAIAttributes.GEN_AI_RESPONSE_MODEL: llm_model, | ||
| }, | ||
| ) | ||
| um.candidates_token_count, | ||
| attributes={ | ||
| GenAIAttributes.GEN_AI_PROVIDER_NAME: _GCP_GEN_AI, | ||
| GenAIAttributes.GEN_AI_TOKEN_TYPE: "output", | ||
| GenAIAttributes.GEN_AI_RESPONSE_MODEL: llm_model, | ||
| }, | ||
| ) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check if there are any existing None guards for usage_metadata fields in the codebase
rg -n "prompt_token_count|candidates_token_count" --type=py -C2Repository: traceloop/openllmetry
Length of output: 16250
🏁 Script executed:
# Find the import statement and type definitions for usage_metadata
rg -n "usage_metadata|UsageMetadata" packages/opentelemetry-instrumentation-google-generativeai/ --type=py -A2 -B2 | head -50Repository: traceloop/openllmetry
Length of output: 813
🏁 Script executed:
# Check the type of um (usage_metadata) in the span_utils file context
rg -n "def.*um" packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py | head -20Repository: traceloop/openllmetry
Length of output: 206
🏁 Script executed:
# Look at the function signature around line 641 to understand um type
sed -n '600,660p' packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.pyRepository: traceloop/openllmetry
Length of output: 1977
🏁 Script executed:
# Check if there are any tests covering the histogram recording with None values
find packages/opentelemetry-instrumentation-google-generativeai/tests -name "*.py" -type f -exec grep -l "token_histogram\|prompt_token_count" {} \;Repository: traceloop/openllmetry
Length of output: 47
🏁 Script executed:
# Check imports and type hints in the span_utils file
head -50 packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py | grep -E "^import|^from|typing"Repository: traceloop/openllmetry
Length of output: 439
🏁 Script executed:
# Check histogram.record method - find its implementation
rg -n "histogram.record\|class.*Histogram" packages/opentelemetry-instrumentation-google-generativeai/ --type=py -B3 -A3Repository: traceloop/openllmetry
Length of output: 47
🏁 Script executed:
# Check if google-generativeai is specified and what version
find packages/opentelemetry-instrumentation-google-generativeai -name "pyproject.toml" -o -name "setup.py" -o -name "requirements*.txt" | xargs cat 2>/dev/null | grep -A5 -B5 "google-generativeai"Repository: traceloop/openllmetry
Length of output: 712
🏁 Script executed:
# Search the broader codebase for how histogram.record is used elsewhere and if None values are ever passed
rg -n "histogram\.record" --type=py -B5 -A2Repository: traceloop/openllmetry
Length of output: 50377
🏁 Script executed:
# Search for type information about usage_metadata in the codebase
rg -n "from google" packages/opentelemetry-instrumentation-google-generativeai/ --type=py | head -20Repository: traceloop/openllmetry
Length of output: 684
🏁 Script executed:
# Check if there are any error handling or try-catch blocks around histogram recording in google-generativeai
sed -n '630,670p' packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py | cat -nRepository: traceloop/openllmetry
Length of output: 1286
🏁 Script executed:
# Check if there are any test failures or comments about None values in the tests
find packages/opentelemetry-instrumentation-google-generativeai/tests -type f -name "*.py" -exec cat {} \; | grep -i "none\|null\|token" | head -30Repository: traceloop/openllmetry
Length of output: 959
🏁 Script executed:
# Check if google-genai is a stub or if we can find type hints for UsageMetadata
find . -name "*.pyi" -o -name "py.typed" | xargs grep -l "prompt_token_count" 2>/dev/null || echo "No type stubs found"Repository: traceloop/openllmetry
Length of output: 84
🏁 Script executed:
# Look for the actual function that calls set_model_response_attributes to understand response handling
rg -n "set_model_response_attributes" packages/opentelemetry-instrumentation-google-generativeai/ --type=py -B5 -A2Repository: traceloop/openllmetry
Length of output: 6377
🌐 Web query:
google-genai GenerateContentResponse usage_metadata prompt_token_count type
💡 Result:
In the Google GenAI SDK (google-genai Python package), GenerateContentResponse has an attribute usage_metadata which provides token usage details from the Gemini API response. prompt_token_count is an integer field within usage_metadata representing the number of tokens in the input prompt. Access it as: response.usage_metadata.prompt_token_count This is an optional field (may be None if not present), of type int when available. Example usage: from google import genai client = genai.Client response = client.models.generate_content(model='gemini-2.5-flash', contents='Your prompt here') if response.usage_metadata: print(response.usage_metadata.prompt_token_count) usage_metadata type is a Pydantic model (from google.genai.types) with fields like: - prompt_token_count: int (input tokens) - candidates_token_count: int (output tokens) - total_token_count: int - thoughts_token_count: int (for thinking models) - cached_content_token_count: int (if using caching) Confirmed in official Gemini API docs and SDK references. Vertex AI REST API equivalent is UsageMetadata with promptTokenCount (camelCase).
Citations:
- 1: https://ai.google.dev/gemini-api/docs/tokens
- 2: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/reference/rest/v1/GenerateContentResponse
- 3: https://googleapis-python-genai-70.mintlify.app/api/models/generate-content
- 4: Mismatch between
client.count_tokens()andresponse.usage_metadata.prompt_token_countin Vertex AI googleapis/python-genai#1284
Add None checks before recording token counts to histogram.
The fields um.prompt_token_count and um.candidates_token_count are optional and may be None. Passing None to histogram.record() could cause runtime errors. Other instrumentation packages in this codebase (ollama, groq, writer) validate token values before recording; apply the same pattern here.
Suggested fix
if token_histogram and um is not None:
+ if um.prompt_token_count is not None:
token_histogram.record(
um.prompt_token_count,
attributes={
GenAIAttributes.GEN_AI_PROVIDER_NAME: _GCP_GEN_AI,
GenAIAttributes.GEN_AI_TOKEN_TYPE: "input",
GenAIAttributes.GEN_AI_RESPONSE_MODEL: llm_model,
},
)
+ if um.candidates_token_count is not None:
token_histogram.record(
um.candidates_token_count,
attributes={
GenAIAttributes.GEN_AI_PROVIDER_NAME: _GCP_GEN_AI,
GenAIAttributes.GEN_AI_TOKEN_TYPE: "output",
GenAIAttributes.GEN_AI_RESPONSE_MODEL: llm_model,
},
)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/span_utils.py`
around lines 641 - 657, The token counts um.prompt_token_count and
um.candidates_token_count may be None, so before calling
token_histogram.record(...) (inside the token_histogram and um is not None
block) validate each value is not None and only record when present;
specifically, check um.prompt_token_count is not None before the input-token
record and check um.candidates_token_count is not None before the output-token
record, keeping the same attributes
(GenAIAttributes.GEN_AI_PROVIDER_NAME/_GCP_GEN_AI,
GenAIAttributes.GEN_AI_TOKEN_TYPE,
GenAIAttributes.GEN_AI_RESPONSE_MODEL/llm_model).
- Resolve semconv-ai dep: use >=0.5.1,<0.6.0; regenerate google-genai and vertex uv.lock - Fix test_generate_metrics: depend on exporter fixture; aggregate metrics across scopes (SDK vs instrumentation) Made-with: Cursor
- BlobPart: use "data" key instead of "content" for base64 payloads (OTel schema) - OutputMessage: always set finish_reason (required by schema), use None when unavailable - Event emitters: map finish reasons to OTel canonical values (stop, length, content_filter, error) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verified against canonical JSON schema (gen-ai-input-messages.json): BlobPart.content is the correct field name for base64 data. The original code was already correct. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Vertex AI SDK (google-cloud-aiplatform) generative AI modules are deprecated (June 2025) and being removed June 2026. The unified google-genai SDK covers both Gemini and Vertex AI, so the google-generativeai instrumentation is sufficient going forward. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
OzBenSimhonTraceloop
changed the title
nina-kollman
approved these changes
Apr 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related to #3836
Summary by CodeRabbit
Release Notes
Refactor