Objective
Improve the readability of OTLP evaluation spans by including dataset row context (input text) in span names and attributes.
Current Behavior
Item spans are named generically:
eval_item 1, eval_item 2, eval_item 3
This makes it hard to identify which dataset row a span corresponds to when viewing traces in AI Toolkit, Jaeger, or other collectors.
Desired Behavior
Item spans should include a truncated version of the input query:
eval_item 1: What is the weather in Seattle today?
eval_item 2: Convert 100 USD to EUR
And the input_text and expected_text attributes should be populated on item spans.
Implementation Notes
- Thread dataset rows (or at minimum
input_field values) from the runner to _emit_item_spans()
- Pass
input_text to eval_item_span() — the parameter already exists but is not used by the caller
- Truncate input text in span name to ~80 chars to avoid excessively long span names
- The
eval_item_span() function in utils/telemetry.py already accepts input_text and expected_text — only the caller in services/runner.py needs updating
Files to Change
src/agentops/services/runner.py — pass dataset row data to _emit_item_spans
tests/unit/test_telemetry.py — update tests if span name format changes
Related
Objective
Improve the readability of OTLP evaluation spans by including dataset row context (input text) in span names and attributes.
Current Behavior
Item spans are named generically:
eval_item 1,eval_item 2,eval_item 3This makes it hard to identify which dataset row a span corresponds to when viewing traces in AI Toolkit, Jaeger, or other collectors.
Desired Behavior
Item spans should include a truncated version of the input query:
eval_item 1: What is the weather in Seattle today?eval_item 2: Convert 100 USD to EURAnd the
input_textandexpected_textattributes should be populated on item spans.Implementation Notes
input_fieldvalues) from the runner to_emit_item_spans()input_texttoeval_item_span()— the parameter already exists but is not used by the callereval_item_span()function inutils/telemetry.pyalready acceptsinput_textandexpected_text— only the caller inservices/runner.pyneeds updatingFiles to Change
src/agentops/services/runner.py— pass dataset row data to_emit_item_spanstests/unit/test_telemetry.py— update tests if span name format changesRelated