-
Notifications
You must be signed in to change notification settings - Fork 752
feat(transformers): implement the support to emitting events in addition to current behavior #2940
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
LuizDMM
wants to merge
3
commits into
traceloop:main
Choose a base branch
from
LuizDMM:implement-events-transformers
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Implement event emition logic in addition to current behavior; - Add "use_legacy_attributes" to Config and the Instrumentor constructor, defaulting to True; - emit events for user prompts and AI responses, following [OpenTelemetry semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/); - fix prompt span attribute settings; - implement testing framework.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Caution
Changes requested ❌
Reviewed everything up to b5a43cb in 2 minutes and 12 seconds. Click for details.
- Reviewed
788
lines of code in11
files - Skipped
1
files when reviewing. - Skipped posting
9
draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. packages/opentelemetry-instrumentation-transformers/opentelemetry/instrumentation/transformers/__init__.py:36
- Draft comment:
Consider avoiding direct modification of the global Config in the constructor. Using instance-level config may prevent unexpected behavior when multiple instrumentors are active concurrently. - Reason this comment was not posted:
Confidence changes required:50%
<= threshold50%
None
2. packages/opentelemetry-instrumentation-transformers/opentelemetry/instrumentation/transformers/utils.py:44
- Draft comment:
Typo in the debug log message of the dont_throw decorator ('OpenLLMetry' likely should be 'OpenTelemetry'). - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
3. packages/opentelemetry-instrumentation-transformers/tests/test_placeholder.py:12
- Draft comment:
This test prints debug information and then forces an AssertionError. If this is a placeholder, consider removing debug prints and the forced failure before merging. - Reason this comment was not posted:
Confidence changes required:33%
<= threshold50%
None
4. packages/opentelemetry-instrumentation-transformers/opentelemetry/instrumentation/transformers/event_emitter.py:96
- Draft comment:
Typo: The comment says "if it is the same as the in the event name." It seems like a word is missing. Consider revising to "if it is the same as that in the event name." - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% While the comment correctly identifies a grammatical error, it's a very minor issue in an internal comment that doesn't affect code functionality. The meaning is still clear despite the typo. According to the rules, we should not make purely informative comments or comments that are unimportant. Documentation typos that don't impact understanding fall into this category. The typo does make the comment slightly less professional, and fixing it would improve code quality marginally. Some might argue that maintaining high-quality documentation is important. While documentation quality matters, this is an extremely minor issue that doesn't impact code understanding or functionality. The rules explicitly state not to make unimportant comments. This comment should be deleted as it addresses a minor documentation issue that doesn't significantly impact code quality or understanding.
5. packages/opentelemetry-instrumentation-transformers/opentelemetry/instrumentation/transformers/utils.py:44
- Draft comment:
There is a typographical error in the log message at line 44. The string 'OpenLLMetry failed to trace in %s, error: %s' appears to contain a typo ('OpenLLMetry' instead of 'OpenTelemetry'). Please correct this for clarity and consistency. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
6. packages/opentelemetry-instrumentation-transformers/pyproject.toml:18
- Draft comment:
It looks like the repository URL on line 18 contains 'openllmetry'. This may be a typographical error; perhaps it should be 'opentelemetry'? Please double-check and correct as necessary. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
7. packages/opentelemetry-instrumentation-transformers/tests/test_pipeline.py:10
- Draft comment:
Typo in function name: 'test_tranformers_pipeline' should be renamed to 'test_transformers_pipeline' to maintain consistency with parameter names and correct spelling. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% While the typo is real, it's consistent across all test functions and doesn't affect functionality. The code will still work perfectly fine. This is more of a cosmetic issue. According to the rules, we should not make comments that are purely informative or unimportant. The build system would likely catch this if it was a real issue. The typo could lead to confusion when searching for these test functions in the codebase. It might also indicate a lack of attention to detail. While consistency in naming is good, this is a minor cosmetic issue that doesn't affect functionality. The rules explicitly state not to comment on things that would be caught by the build or that are purely informative. The comment should be deleted as it points out a minor cosmetic issue that doesn't affect functionality and falls under the category of purely informative comments.
8. packages/opentelemetry-instrumentation-transformers/tests/test_pipeline.py:34
- Draft comment:
Typo in function name: 'test_tranformers_pipeline_with_events_with_content' should be corrected to 'test_transformers_pipeline_with_events_with_content'. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50% While the typo exists, fixing it would require changing all three function names for consistency. The typo doesn't affect functionality since these are just test function names. The comment is about code style rather than logic or functionality. According to the rules, we should not make purely informative comments or comments about obvious issues. The typo could potentially cause confusion when searching for these test functions. Also, maintaining good code quality includes proper spelling. However, this is a minor stylistic issue that doesn't affect functionality, and the typo is consistent across all test functions, suggesting it might even be intentional. The comment should be removed as it's a minor stylistic issue that doesn't affect functionality and doesn't require immediate action.
9. packages/opentelemetry-instrumentation-transformers/tests/test_pipeline.py:62
- Draft comment:
Typo in function name: 'test_tranformers_pipeline_with_events_with_no_content' should be updated to 'test_transformers_pipeline_with_events_with_no_content'. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% The typo appears in all three test function names consistently. This suggests it might be intentional or matching some naming convention from the code being tested. Since it's consistent across all test functions, changing just one would actually make the naming less consistent. Additionally, function names in tests are internal implementation details that don't affect functionality. The typo could make the code slightly harder to read or search for. It might also indicate a deeper issue if it's reflecting a typo in the actual implementation being tested. While readability is important, the consistency across all test functions suggests this isn't a simple typo. If there was an issue with the spelling, it should be addressed across all instances, not just one. The comment should be deleted. The consistent spelling across all test functions suggests this isn't a simple mistake, and changing just one instance would reduce consistency.
Workflow ID: wflow_LGf1iyKpZIK2tlS6
You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.
...etry-instrumentation-transformers/opentelemetry/instrumentation/transformers/event_models.py
Show resolved
Hide resolved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
✅ PR Requirements
feat(instrumentation): ...
orfix(instrumentation): ....
📌 Issue Requirements
Important
Adds event emission support to transformers instrumentation with a toggle for legacy attributes, including tests for new functionality.
TransformersInstrumentor
in__init__.py
.use_legacy_attributes
config inconfig.py
to toggle between legacy attributes and event-based logging.event_emitter.py
andspan_utils.py
.text_generation_pipeline_wrapper.py
to handle both span attributes and event emissions.PromptEvent
andCompletionEvent
inevent_models.py
for structured event data.should_emit_events()
andshould_send_prompts()
inutils.py
to control event emission behavior.test_pipeline.py
to verify event emission with and without content.conftest.py
for different instrumentation setups.This description was created by
for b5a43cb. You can customize this summary. It will automatically update as commits are pushed.