Skip to content

Add daily Sentry OTel token consumption report workflow#27916

Merged
pelikhan merged 4 commits intomainfrom
copilot/create-token-consumption-report
Apr 22, 2026
Merged

Add daily Sentry OTel token consumption report workflow#27916
pelikhan merged 4 commits intomainfrom
copilot/create-token-consumption-report

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 22, 2026

This adds a new daily agentic workflow that reports token consumption across all repository workflows using OTel telemetry stored in Sentry. The workflow produces a rolling daily issue with top consumers, aggregate metrics, and data-quality gaps.

  • Workflow added: daily token consumption reporting

    • Introduced .github/workflows/daily-token-consumption-report.md (+ compiled lockfile) with weekday-daily scheduling and strict mode.
    • Configured Sentry MCP + OTLP reporting imports (shared/mcp/sentry.md, shared/reporting-otlp.md).
    • Publishes a single rolling issue via safe outputs ([token-consumption] prefix, close-older-issues: true, expires: 1d).
  • Telemetry query and aggregation contract

    • Pulls last 24h telemetry from Sentry spans, with fallback to transactions only when spans are unusable.
    • Defines workflow/run field extraction fallbacks to keep attribution stable even with sparse payloads.
    • Requires top-10 workflow breakdown plus distribution metrics (avg, p95, coverage/missing counts).
  • Token field normalization to prevent overcounting

    • Added explicit precedence rules for total, input, and output token aliases across heterogeneous telemetry schemas.
    • Specifies reconstruction logic when only partial token fields exist (derive total from input/output when needed).
safe-outputs:
  create-issue:
    title-prefix: "[token-consumption] "
    close-older-issues: true
    expires: 1d

Copilot AI and others added 4 commits April 22, 2026 22:30
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/e33d7875-691f-4198-a3a0-b1002afc6bc5

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/e33d7875-691f-4198-a3a0-b1002afc6bc5

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
This reverts commit 7966b31.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan marked this pull request as ready for review April 22, 2026 22:46
Copilot AI review requested due to automatic review settings April 22, 2026 22:46
@pelikhan pelikhan merged commit bb11b8b into main Apr 22, 2026
18 of 19 checks passed
@pelikhan pelikhan deleted the copilot/create-token-consumption-report branch April 22, 2026 22:46
@github-actions github-actions Bot mentioned this pull request Apr 22, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new scheduled Agentic Workflow that queries Sentry OTel telemetry to produce a daily token-consumption report and publishes it as a rolling GitHub issue.

Changes:

  • Introduces a new weekday-daily workflow definition for querying Sentry spans/transactions and aggregating token usage by workflow.
  • Configures safe-outputs to publish a single rolling issue with a [token-consumption] prefix and 1-day expiry.
  • Adds the compiled workflow lockfile that wires up MCP Gateway + Sentry MCP server and safe-outputs handling.
Show a summary per file
File Description
.github/workflows/daily-token-consumption-report.md New agentic workflow prompt/spec for token consumption reporting via Sentry OTel + safe-outputs issue publishing.
.github/workflows/daily-token-consumption-report.lock.yml Compiled workflow YAML/lockfile including MCP Gateway setup, Sentry MCP config, and safe-outputs handlers.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment on lines +670 to +674
"env": {
"OPENAI_API_KEY": "${{ secrets.SENTRY_OPENAI_API_KEY }}",
"SENTRY_ACCESS_TOKEN": "${{ secrets.SENTRY_ACCESS_TOKEN }}",
"SENTRY_HOST": "${{ env.SENTRY_HOST || 'https://sentry.io' }}"
},
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the generated MCP config heredoc, the sentry.env values are embedded using GitHub expression syntax (${{ secrets.* }}, ${{ env.* }}), which results in secrets/URLs being interpolated directly into the JSON at runtime. This deviates from other workflows’ lockfiles that keep placeholders (e.g. \${SENTRY_ACCESS_TOKEN}) and pass the actual values via environment variables, reducing the risk of accidental logging/exfiltration. Prefer placeholder-style variables here and ensure the corresponding env vars are set for the step.

Copilot uses AI. Check for mistakes.
"type": "http",
"url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT",
"headers": {
"Authorization": "$GH_AW_SAFE_OUTPUTS_API_KEY"
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The safeoutputs MCP server config uses $GH_AW_SAFE_OUTPUTS_API_KEY directly in the JSON heredoc, which will expand into the config content and may be printed by gateway tooling (e.g., during debugging/connectivity checks). Other lockfiles typically escape this as a placeholder (e.g. \${GH_AW_SAFE_OUTPUTS_API_KEY}) to avoid writing sensitive values into config text. Update the generated config to avoid inlining the key.

Suggested change
"Authorization": "$GH_AW_SAFE_OUTPUTS_API_KEY"
"Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}"

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants