Skip to content

Improve OTLP telemetry span names with dataset context #58

@Dongbumlee

Description

@Dongbumlee

Objective

Improve the readability of OTLP evaluation spans by including dataset row context (input text) in span names and attributes.

Current Behavior

Item spans are named generically:

  • eval_item 1, eval_item 2, eval_item 3

This makes it hard to identify which dataset row a span corresponds to when viewing traces in AI Toolkit, Jaeger, or other collectors.

Desired Behavior

Item spans should include a truncated version of the input query:

  • eval_item 1: What is the weather in Seattle today?
  • eval_item 2: Convert 100 USD to EUR

And the input_text and expected_text attributes should be populated on item spans.

Implementation Notes

  • Thread dataset rows (or at minimum input_field values) from the runner to _emit_item_spans()
  • Pass input_text to eval_item_span() — the parameter already exists but is not used by the caller
  • Truncate input text in span name to ~80 chars to avoid excessively long span names
  • The eval_item_span() function in utils/telemetry.py already accepts input_text and expected_text — only the caller in services/runner.py needs updating

Files to Change

  • src/agentops/services/runner.py — pass dataset row data to _emit_item_spans
  • tests/unit/test_telemetry.py — update tests if span name format changes

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions