-
Notifications
You must be signed in to change notification settings - Fork 744
feat(vertexai): implement emitting events in addition to current beha… #2942
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
LuizDMM
wants to merge
3
commits into
traceloop:main
Choose a base branch
from
LuizDMM:implement-events-vertexai
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…vior - Add "use_legacy_attributes" to Config and the Instrumentor constructor, defaulting to True; - emit events for user prompts and AI responses, following [OpenTelemetry semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/\\\); - implement comprehensive tests to verify the new functionality and ensure that, when use_legacy_attributes == True, the existing behavior remains unchanged;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Caution
Changes requested ❌
Reviewed everything up to e023a64 in 2 minutes and 15 seconds. Click for details.
- Reviewed
1154
lines of code in10
files - Skipped
0
files when reviewing. - Skipped posting
5
draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/utils.py:10
- Draft comment:
The default logic in should_send_prompts uses 'os.getenv(...) or "true"'; confirm that this default of 'true' is intended, as it may always enable sending prompt contents. - Reason this comment was not posted:
Confidence changes required:50%
<= threshold50%
None
2. packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.py:298
- Draft comment:
Typographical error: The docstring in the VertexAIInstrumentor class contains a typo ('VertextAI'). It should be corrected to 'VertexAI' to maintain consistency. - Reason this comment was not posted:
Comment was on unchanged code.
3. packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/event_emitter.py:125
- Draft comment:
Typo in comment: The phrase '... if it is the same as the in the event name.' could be reworded to '... if it is the same as in the event name.' for clarity. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% While the comment is technically correct about the grammatical error, it's an extremely minor issue in an internal comment. The meaning is still clear despite the typo. According to the rules, we should not make purely informative comments or comments that are unimportant. Documentation typos that don't impact understanding fall into this category. The typo could potentially cause slight confusion for future maintainers. Documentation quality is important for long-term maintainability. While documentation quality matters, this particular typo is so minor that it doesn't impact comprehension. The rules explicitly state not to make unimportant comments. Delete this comment as it addresses an unimportant documentation issue that doesn't impact code functionality or comprehension.
4. packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/event_emitter.py:51
- Draft comment:
Potential typo in finish_reason_map: the value for key 8 is set to 'spii'. Please verify if 'spii' is intended or if it should be corrected (e.g. to 'spi' or another appropriate term). - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% SPII typically stands for "Sensitive Personal Identifiable Information" which is a common term in data privacy. Given this is a finish reason in a Vertex AI context, it's likely this is the intended meaning. The comment suggests changing it without any evidence that 'spii' is incorrect. We should not suggest changes to API-defined constants without strong evidence. I could be wrong about SPII being the intended meaning. Maybe there's documentation from Vertex AI that specifies a different term. Even if I'm not 100% sure about SPII being the meaning, we shouldn't suggest changing API-defined constants without clear evidence of an error. The burden of proof is on showing it's wrong, not speculating about typos. Delete the comment. It suggests changing what appears to be an API-defined constant without any evidence that it's incorrect, and 'spii' is likely the intended value representing "Sensitive Personal Identifiable Information".
5. packages/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/utils.py:32
- Draft comment:
Typographical error: 'OpenLLMetry' appears in the logging message. Please correct it to 'OpenTelemetry' to ensure consistency in our telemetry naming. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
Workflow ID: wflow_zwjfmbz3uB3Zy3qH
You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.
...es/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.py
Outdated
Show resolved
Hide resolved
...es/opentelemetry-instrumentation-vertexai/opentelemetry/instrumentation/vertexai/__init__.py
Outdated
Show resolved
Hide resolved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
✅ PR Requirements
📌 Issue Requirements
Important
Adds event emission to Vertex AI instrumentation with a configuration option to toggle between legacy and event-based attributes.
use_legacy_attributes
config inconfig.py
.emit_prompt_events
andemit_response_events
inevent_emitter.py
to handle event logging.VertexAIInstrumentor
in__init__.py
to support event logging based on configuration.should_emit_events
andshould_send_prompts
inutils.py
to control event emission and prompt logging.span_utils.py
.disabled_test_bison.py
anddisabled_test_gemini.py
to verify event emission and legacy behavior.conftest.py
to support testing with different configurations.This description was created by
for e023a64. You can customize this summary. It will automatically update as commits are pushed.