Skip to content

doc: add KQL query library section to telemetry.md #89

@placerda

Description

@placerda

Objective

Add a ready-to-use KQL query section to docs/telemetry.md so users who send eval traces to Azure Monitor / Application Insights can immediately query their data.

Background

When eval traces land in App Insights (via OTLP ingestion — see #88), all span attributes are available as custom dimensions in the dependencies and requests tables. Users need practical KQL examples to extract value from this data.

Proposed Queries

Add a new section between 'Sending Traces to Azure Monitor' and 'Evaluation Tracing vs. Agent Execution Tracing' with queries like:

  1. Slowest evaluation rows — top N eval_item spans by duration
  2. Failed evaluators — filter by agentops.eval.evaluator.passed == false
  3. Pass rate over time — trend of agentops.eval.pass_rate from root spans
  4. Token usage per run — sum of gen_ai.usage.input_tokens + output_tokens
  5. Evaluator score distribution — histogram of agentops.eval.evaluator.score grouped by evaluator name

Format

Each query should include:

  • A short description of what it answers
  • The KQL query itself (ready to paste into App Insights → Logs)
  • Expected table/column names based on OTLP → App Insights mapping

Acceptance Criteria

Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions