Skip to content

Sentry skill ignores TELEMETRY.md before querying #330

@sentry-junior

Description

@sentry-junior

The sentry skill's workflow says to check TELEMETRY.md in the configured repo before forming queries, but in practice the agent skips this step and falls back to searching Sentry issues/errors instead of using the repo's telemetry recipes.

Observed behavior

  • When asked about warden security-review findings (a usage/telemetry question), the agent searched Sentry error issues instead of loading TELEMETRY.md from the configured github.repo (getsentry/warden).
  • The agent only consulted TELEMETRY.md after being explicitly prompted, at which point the correct query recipe was immediately obvious.
  • The skill instruction "check the repo root for TELEMETRY.md before forming queries" was not followed on either the first or second turn.

Likely areas to investigate

  • The sentry skill workflow step that says to check TELEMETRY.md may not be prominent enough relative to the default error-searching behavior. The agent defaulted to sentry issue list and sentry explore --dataset errors before considering logs, spans, or metrics recipes.
  • The skill may need a stronger gate or ordering that forces TELEMETRY.md lookup before any Sentry CLI query when a github.repo is configured.
  • Separately, the agent initially tried to resolve security-review as a Junior skill rather than recognizing it as a warden skill emitting telemetry — the configured repo context (getsentry/warden) should have been a stronger signal.

Evidence

  • Thread: agent was asked "how many findings from the security-review skill in the last two weeks?"
  • getsentry/warden/TELEMETRY.md contains a "Total findings for a skill" recipe and log completion patterns that answered the question directly once consulted.
  • The correct answer (7 findings from Sentry logs) required the TELEMETRY.md recipes, not the errors dataset.

Action taken on behalf of David Cramer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions