Skip to content

feat(tracking): identify CLI and detect AI agent harness#175

Merged
jomifepe merged 4 commits intomainfrom
aa/ai-harness-tracking
May 8, 2026
Merged

feat(tracking): identify CLI and detect AI agent harness#175
jomifepe merged 4 commits intomainfrom
aa/ai-harness-tracking

Conversation

@angeloashmore
Copy link
Copy Markdown
Member

@angeloashmore angeloashmore commented May 5, 2026

Description

  • Adds a detectAgent helper that identifies the AI coding agent (Claude Code, Codex, Cursor, Cline, Copilot, etc.) invoking the CLI based on environment variables. Groundwork for harness-aware tracking.
  • createRepository now sends ?app=cli (and &agent=<harness> when detected) on POST /app/dashboard/repositories. On the API side (paired PR: prismicio/wroom#3189), app=cli maps to a new ExternalApp.CLI case so the Repository Created Segment event reports source=cli, and the agent query param is propagated as an agent trait on the same event.
  • Prismic CLI Start and Prismic CLI End events now carry an agent property as well, so harness usage can be analyzed across the full CLI lifecycle, not only repo creation.

How to QA 1

Deployed to dev-tools

  • Use an agent to create a new repository with the CLI, initialize, etc:
    • PRISMIC_HOST=dev-tools-wroom.com npx prismic@pr-175 repo create -n my-repo
    • PRISMIC_HOST=dev-tools-wroom.com npx prismic@pr-175 init -r my-repo
  • Check Segment live debugger to see the new source and agent properties

View in Codesmith
Need help on this PR? Tag @codesmith with what you need.

  • Let Codesmith autofix CI failures and bot reviews

Note

Low Risk
Low risk: adds a new standalone helper that only reads environment variables (and optionally checks a Linux file path) without changing existing execution paths.

Overview
Introduces src/lib/ai.ts with a new async detectAgent() helper that identifies the active AI agent by checking a prioritized set of environment variables (Claude Code/Cowork, Codex, Cursor, Cline, Copilot, etc.).

On Linux it additionally probes for a Devin marker at file:///opt/.devin, and it returns unknown-sandbox when IS_SANDBOX=yes.

Reviewed by Cursor Bugbot for commit cd2983c. Bugbot is set up for automated code reviews on this repo. Configure here.

Footnotes

  1. Please use these labels when submitting a review:
    ❓ #ask: Ask a question.
    💡 #idea: Suggest an idea.
    ⚠️ #issue: Strongly suggest a change.
    🎉 #nice: Share a compliment.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit cd2983c. Configure here.

Comment thread src/lib/ai.ts
@angeloashmore angeloashmore marked this pull request as draft May 5, 2026 20:52
jomifepe added 2 commits May 7, 2026 10:02
Pass app=cli and detected AI agent (Claude Code, Cursor, etc.) as query
params when creating a repository so the server-side Repository Created
event can distinguish CLI from UI and attribute which harness triggered it.
Attach the agent to Prismic CLI Start/End events as well.
@jomifepe jomifepe self-assigned this May 7, 2026
@jomifepe jomifepe changed the title feat: detect AI agent harness feat(tracking): identify CLI and detect AI agent harness May 7, 2026
@jomifepe jomifepe marked this pull request as ready for review May 7, 2026 14:22
@jomifepe jomifepe merged commit 1d5c9ed into main May 8, 2026
12 checks passed
@jomifepe jomifepe deleted the aa/ai-harness-tracking branch May 8, 2026 12:27
@github-actions github-actions Bot mentioned this pull request May 8, 2026
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants