Skip to content

feat(telemetry): normalize agent tag into structured agent/version/role fields#896

Merged
BYK merged 1 commit intomainfrom
byk/normalize-agent-telemetry-tag
Apr 30, 2026
Merged

feat(telemetry): normalize agent tag into structured agent/version/role fields#896
BYK merged 1 commit intomainfrom
byk/normalize-agent-telemetry-tag

Conversation

@BYK
Copy link
Copy Markdown
Member

@BYK BYK commented Apr 30, 2026

Summary

  • Adds normalizeAgent() to parse compound slash-separated agent values (e.g. claude-code/2.1.123/agent), resolve aliases (claude-codeclaude), and handle boolean-ish garbage values
  • Changes detectAgent() and detectAgentFromProcessTree() to return structured AgentInfo { name, version?, role? } instead of raw strings
  • Telemetry now sets three Sentry tags: agent (canonical family name), agent.version, agent.role

Motivation

The agent Sentry tag was receiving raw, unvalidated values from AI_AGENT and AGENT env vars. This caused fragmented tag data — claude, claude-code, and claude-code/2.1.123/agent all appeared as separate tag values.

Boolean-ish garbage handling

Boolean-ish values are split into two classes:

Class Values Result Rationale
Truthy 1, true, yes, on { name: "unknown" } Agent is active but didn't identify itself
Falsy 0, false, no, off undefined (no agent) Explicit opt-out — "no agent present"

For AI_AGENT, falsy values fall through to the next detection level (e.g. AI_AGENT=false + CLAUDE_CODE=1claude). For AGENT (lowest priority), falsy values result in no detection, which then falls through to process tree detection in initSentry.

Example transformations

Input agent tag agent.version agent.role
claude-code/2.1.123/agent claude 2.1.123 agent
claude-code claude
claude claude
1 unknown
false (not set)
cowork cowork

Test plan

  • Updated all existing detectAgent and detectAgentFromProcessTree test assertions for AgentInfo return type
  • Added unit tests for normalizeAgent(): compound values, alias resolution, truthy garbage → unknown, falsy garbage → undefined
  • Added edge case tests: CLAUDE_CODE_IS_COWORK alone → undefined, AI_AGENT=false falls through to next level
  • Added AGENT_ALIASES structural tests
  • Added property-based tests (detect-agent.property.test.ts) for 11 invariants: lowercase, non-empty, truthy garbage → unknown, falsy garbage → undefined, version format, idempotency, alias resolution, compound extraction
  • Full unit suite passes (6344+ tests, 0 failures)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 30, 2026

PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://cli.sentry.dev/_preview/pr-896/

Built to branch gh-pages at 2026-04-30 23:07 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 30, 2026

Codecov Results 📊

6461 passed | Total: 6461 | Pass Rate: 100% | Execution Time: 0ms

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +51
Passed Tests 📈 +51
Failed Tests
Skipped Tests

All tests are passing successfully.

❌ Patch coverage is 63.75%. Project has 13258 uncovered lines.
❌ Project coverage is 75.95%. Comparing base (base) to head (head).

Files with missing lines (2)
File Patch % Lines
src/lib/detect-agent.ts 71.01% ⚠️ 20 Missing
src/lib/telemetry.ts 18.18% ⚠️ 9 Missing
Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
- Coverage    75.98%    75.95%    -0.03%
==========================================
  Files          295       295         —
  Lines        55045     55120       +75
  Branches         0         0         —
==========================================
+ Hits         41819     41862       +43
- Misses       13226     13258       +32
- Partials         0         0         —

Generated by Codecov Action

Comment thread src/lib/detect-agent.ts
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c848aba. Configure here.

expect(result.version).toBe(version);
}),
{ numRuns: DEFAULT_NUM_RUNS }
);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Property tests flake when arbitrary generates garbage names

Medium Severity

The agentNameArb arbitrary can generate strings that match GARBAGE_NAME_RE (e.g., "no", "on", "off", "yes", "true", "false") since they are 2–5 lowercase letters satisfying the filter. When such a name appears in the compound property tests, normalizeAgent short-circuits to { name: "unknown" } without extracting version or role, causing the unconditional expect(result.version).toBe(version) and expect(result.role).toBe(role) assertions to fail. These tests will flake in CI whenever fast-check happens to generate a garbage name.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit c848aba. Configure here.

@BYK BYK force-pushed the byk/normalize-agent-telemetry-tag branch from c848aba to 24cd11f Compare April 30, 2026 20:50
…le fields

Raw AI_AGENT and AGENT env var values like 'claude-code/2.1.123/agent'
or '1' were passed through to the Sentry 'agent' tag without
normalization. This caused fragmented data — 'claude', 'claude-code',
and 'claude-code/2.1.123/agent' all appeared as separate tag values.

Add normalizeAgent() that parses compound slash-separated values,
resolves aliases (claude-code → claude), and detects garbage values
(1, true, yes → 'unknown'). Change detectAgent() and
detectAgentFromProcessTree() to return structured AgentInfo with
name, version, and role fields. Telemetry now sets three tags:
agent, agent.version, agent.role.
@BYK BYK force-pushed the byk/normalize-agent-telemetry-tag branch from 24cd11f to 5620fc7 Compare April 30, 2026 23:07
@BYK BYK enabled auto-merge (squash) April 30, 2026 23:07
@BYK BYK merged commit f4730f7 into main Apr 30, 2026
26 checks passed
@BYK BYK deleted the byk/normalize-agent-telemetry-tag branch April 30, 2026 23:12
BYK added a commit that referenced this pull request Apr 30, 2026
## Summary

- Excludes boolean-ish garbage names (`no`, `on`, `off`, `yes`, `true`,
`false`, `0`, `1`) from `agentNameArb` in
`detect-agent.property.test.ts`

## Problem

Follow-up to #896. The `agentNameArb` arbitrary could randomly generate
short strings like `"no"` or `"on"` that match `FALSY_GARBAGE_RE` /
`TRUTHY_GARBAGE_RE`. When these appeared in compound property tests
(e.g. `"no/1.2.3/agent"`), `normalizeAgent` would short-circuit without
extracting version/role, causing the unconditional
`expect(result!.version).toBe(version)` assertion to fail.

Caught by [Cursor
BugBot](#896 (comment)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant