Skip to content

feat(transformers): implement the support to emitting events in addition to current behavior #2940

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

LuizDMM
Copy link

@LuizDMM LuizDMM commented May 13, 2025

✅ PR Requirements

  • I have added tests that cover my changes.
  • If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.
  • PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
  • (If applicable) I have updated the documentation accordingly.

📌 Issue Requirements

  • Make sure to keep the current way of emitting events and the new (event-based) way with the official semantic
  • Add a config to each instrumentation called use_legacy_attributes which defaults to true, and if set to false emit events instead of the prompt / completion attributes.
  • Propagate a new initialization parameter of the SDK called use_legacy_attributes that sets this config in all instrumentations.

Important

Adds event emission support to transformers instrumentation with a toggle for legacy attributes, including tests for new functionality.

  • Behavior:
    • Adds event emission capability to TransformersInstrumentor in __init__.py.
    • Introduces use_legacy_attributes config in config.py to toggle between legacy attributes and event-based logging.
    • Implements event emission logic in event_emitter.py and span_utils.py.
    • Updates text_generation_pipeline_wrapper.py to handle both span attributes and event emissions.
  • Models:
    • Defines PromptEvent and CompletionEvent in event_models.py for structured event data.
  • Utilities:
    • Adds should_emit_events() and should_send_prompts() in utils.py to control event emission behavior.
  • Testing:
    • Adds tests in test_pipeline.py to verify event emission with and without content.
    • Configures test fixtures in conftest.py for different instrumentation setups.

This description was created by Ellipsis for b5a43cb. You can customize this summary. It will automatically update as commits are pushed.

- Implement event emition logic in addition to current behavior;
- Add "use_legacy_attributes" to Config and the Instrumentor constructor, defaulting to True;
- emit events for user prompts and AI responses, following [OpenTelemetry semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/);
- fix prompt span attribute settings;
- implement testing framework.
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Changes requested ❌

Reviewed everything up to b5a43cb in 2 minutes and 12 seconds. Click for details.
  • Reviewed 788 lines of code in 11 files
  • Skipped 1 files when reviewing.
  • Skipped posting 9 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. packages/opentelemetry-instrumentation-transformers/opentelemetry/instrumentation/transformers/__init__.py:36
  • Draft comment:
    Consider avoiding direct modification of the global Config in the constructor. Using instance-level config may prevent unexpected behavior when multiple instrumentors are active concurrently.
  • Reason this comment was not posted:
    Confidence changes required: 50% <= threshold 50% None
2. packages/opentelemetry-instrumentation-transformers/opentelemetry/instrumentation/transformers/utils.py:44
  • Draft comment:
    Typo in the debug log message of the dont_throw decorator ('OpenLLMetry' likely should be 'OpenTelemetry').
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
3. packages/opentelemetry-instrumentation-transformers/tests/test_placeholder.py:12
  • Draft comment:
    This test prints debug information and then forces an AssertionError. If this is a placeholder, consider removing debug prints and the forced failure before merging.
  • Reason this comment was not posted:
    Confidence changes required: 33% <= threshold 50% None
4. packages/opentelemetry-instrumentation-transformers/opentelemetry/instrumentation/transformers/event_emitter.py:96
  • Draft comment:
    Typo: The comment says "if it is the same as the in the event name." It seems like a word is missing. Consider revising to "if it is the same as that in the event name."
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% While the comment correctly identifies a grammatical error, it's a very minor issue in an internal comment that doesn't affect code functionality. The meaning is still clear despite the typo. According to the rules, we should not make purely informative comments or comments that are unimportant. Documentation typos that don't impact understanding fall into this category. The typo does make the comment slightly less professional, and fixing it would improve code quality marginally. Some might argue that maintaining high-quality documentation is important. While documentation quality matters, this is an extremely minor issue that doesn't impact code understanding or functionality. The rules explicitly state not to make unimportant comments. This comment should be deleted as it addresses a minor documentation issue that doesn't significantly impact code quality or understanding.
5. packages/opentelemetry-instrumentation-transformers/opentelemetry/instrumentation/transformers/utils.py:44
  • Draft comment:
    There is a typographical error in the log message at line 44. The string 'OpenLLMetry failed to trace in %s, error: %s' appears to contain a typo ('OpenLLMetry' instead of 'OpenTelemetry'). Please correct this for clarity and consistency.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
6. packages/opentelemetry-instrumentation-transformers/pyproject.toml:18
  • Draft comment:
    It looks like the repository URL on line 18 contains 'openllmetry'. This may be a typographical error; perhaps it should be 'opentelemetry'? Please double-check and correct as necessary.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
7. packages/opentelemetry-instrumentation-transformers/tests/test_pipeline.py:10
  • Draft comment:
    Typo in function name: 'test_tranformers_pipeline' should be renamed to 'test_transformers_pipeline' to maintain consistency with parameter names and correct spelling.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% While the typo is real, it's consistent across all test functions and doesn't affect functionality. The code will still work perfectly fine. This is more of a cosmetic issue. According to the rules, we should not make comments that are purely informative or unimportant. The build system would likely catch this if it was a real issue. The typo could lead to confusion when searching for these test functions in the codebase. It might also indicate a lack of attention to detail. While consistency in naming is good, this is a minor cosmetic issue that doesn't affect functionality. The rules explicitly state not to comment on things that would be caught by the build or that are purely informative. The comment should be deleted as it points out a minor cosmetic issue that doesn't affect functionality and falls under the category of purely informative comments.
8. packages/opentelemetry-instrumentation-transformers/tests/test_pipeline.py:34
  • Draft comment:
    Typo in function name: 'test_tranformers_pipeline_with_events_with_content' should be corrected to 'test_transformers_pipeline_with_events_with_content'.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50% While the typo exists, fixing it would require changing all three function names for consistency. The typo doesn't affect functionality since these are just test function names. The comment is about code style rather than logic or functionality. According to the rules, we should not make purely informative comments or comments about obvious issues. The typo could potentially cause confusion when searching for these test functions. Also, maintaining good code quality includes proper spelling. However, this is a minor stylistic issue that doesn't affect functionality, and the typo is consistent across all test functions, suggesting it might even be intentional. The comment should be removed as it's a minor stylistic issue that doesn't affect functionality and doesn't require immediate action.
9. packages/opentelemetry-instrumentation-transformers/tests/test_pipeline.py:62
  • Draft comment:
    Typo in function name: 'test_tranformers_pipeline_with_events_with_no_content' should be updated to 'test_transformers_pipeline_with_events_with_no_content'.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% The typo appears in all three test function names consistently. This suggests it might be intentional or matching some naming convention from the code being tested. Since it's consistent across all test functions, changing just one would actually make the naming less consistent. Additionally, function names in tests are internal implementation details that don't affect functionality. The typo could make the code slightly harder to read or search for. It might also indicate a deeper issue if it's reflecting a typo in the actual implementation being tested. While readability is important, the consistency across all test functions suggests this isn't a simple typo. If there was an issue with the spelling, it should be addressed across all instances, not just one. The comment should be deleted. The consistent spelling across all test functions suggests this isn't a simple mistake, and changing just one instance would reduce consistency.

Workflow ID: wflow_LGf1iyKpZIK2tlS6

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant