Add daily Sentry OTel token consumption report workflow#27916
Conversation
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/e33d7875-691f-4198-a3a0-b1002afc6bc5 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/e33d7875-691f-4198-a3a0-b1002afc6bc5 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
This reverts commit 7966b31. Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/e33d7875-691f-4198-a3a0-b1002afc6bc5 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a new scheduled Agentic Workflow that queries Sentry OTel telemetry to produce a daily token-consumption report and publishes it as a rolling GitHub issue.
Changes:
- Introduces a new weekday-daily workflow definition for querying Sentry spans/transactions and aggregating token usage by workflow.
- Configures safe-outputs to publish a single rolling issue with a
[token-consumption]prefix and 1-day expiry. - Adds the compiled workflow lockfile that wires up MCP Gateway + Sentry MCP server and safe-outputs handling.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/daily-token-consumption-report.md | New agentic workflow prompt/spec for token consumption reporting via Sentry OTel + safe-outputs issue publishing. |
| .github/workflows/daily-token-consumption-report.lock.yml | Compiled workflow YAML/lockfile including MCP Gateway setup, Sentry MCP config, and safe-outputs handlers. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 2
| "env": { | ||
| "OPENAI_API_KEY": "${{ secrets.SENTRY_OPENAI_API_KEY }}", | ||
| "SENTRY_ACCESS_TOKEN": "${{ secrets.SENTRY_ACCESS_TOKEN }}", | ||
| "SENTRY_HOST": "${{ env.SENTRY_HOST || 'https://sentry.io' }}" | ||
| }, |
There was a problem hiding this comment.
In the generated MCP config heredoc, the sentry.env values are embedded using GitHub expression syntax (${{ secrets.* }}, ${{ env.* }}), which results in secrets/URLs being interpolated directly into the JSON at runtime. This deviates from other workflows’ lockfiles that keep placeholders (e.g. \${SENTRY_ACCESS_TOKEN}) and pass the actual values via environment variables, reducing the risk of accidental logging/exfiltration. Prefer placeholder-style variables here and ensure the corresponding env vars are set for the step.
| "type": "http", | ||
| "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT", | ||
| "headers": { | ||
| "Authorization": "$GH_AW_SAFE_OUTPUTS_API_KEY" |
There was a problem hiding this comment.
The safeoutputs MCP server config uses $GH_AW_SAFE_OUTPUTS_API_KEY directly in the JSON heredoc, which will expand into the config content and may be printed by gateway tooling (e.g., during debugging/connectivity checks). Other lockfiles typically escape this as a placeholder (e.g. \${GH_AW_SAFE_OUTPUTS_API_KEY}) to avoid writing sensitive values into config text. Update the generated config to avoid inlining the key.
| "Authorization": "$GH_AW_SAFE_OUTPUTS_API_KEY" | |
| "Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}" |
This adds a new daily agentic workflow that reports token consumption across all repository workflows using OTel telemetry stored in Sentry. The workflow produces a rolling daily issue with top consumers, aggregate metrics, and data-quality gaps.
Workflow added: daily token consumption reporting
.github/workflows/daily-token-consumption-report.md(+ compiled lockfile) with weekday-daily scheduling and strict mode.shared/mcp/sentry.md,shared/reporting-otlp.md).[token-consumption]prefix,close-older-issues: true,expires: 1d).Telemetry query and aggregation contract
avg,p95, coverage/missing counts).Token field normalization to prevent overcounting
total,input, andoutputtoken aliases across heterogeneous telemetry schemas.