Classify agents on every command invocation#296
Conversation
|
Cc @carole-lavillonniere @anisaoshafi because #58, #127, etc. |
| TypeHuman Type = "human" | ||
| TypeAgent Type = "agent" | ||
| TypeCI Type = "ci" | ||
| ) |
There was a problem hiding this comment.
question: why do we have a type with mutually exclusive values (ci can also be agent but we can't represent this here), when instead we could have 2 binary fields: agent=yes/no ci=yes/no? Data model could become something like:
is_agent / agent_identity (cursor, claude-code, …)
is_ci / ci_identity (github-actions, jenkins, …)
detection_method
Then we could also remove the other lookup in internal/telemetry/client.go line 77 but instead use the value from internal/caller/caller.go
There was a problem hiding this comment.
Nice catch, @carole-lavillonniere! ⚾
I was trying to mirror the approach from localstack/localstack-cli-standalone#19, but you're right! Made some changes in 27b8021, and updated PR description. Could you take another look and take it over from here? 🙏
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
8d4d900 to
27b8021
Compare
Tags every command invocation in CLI telemetry so we can answer "are agents using lstk?". Detection is environment-based (the strongest signal) with a TTY fallback for humans, and is attached to every telemetry event. A detected agent is also forwarded into the emulator container as
AI_AGENT.Agent and CI are tracked as independent axes — an agent can run inside CI, so each is recorded separately rather than collapsed into one mutually exclusive value. Every event carries
is_ciplusagent_identity/ci_identitywhen detected, and a derivedcaller_type(agent/ci/human) for single-label segmentation. This mirrors the legacy CLI (localstack/localstack-cli-standalone#19), whereis_ciis its own field independent of agent detection.See PRO-276 for more context.
Notes
agent_identityandis_ci/ci_identity);caller_typederives toagent(agent > ci > human) so segmentation stays single-label without discarding the CI signal.is_ciis sourced from the single caller classifier (14 known CI providers), not a separate bareCIenv lookup.AI_AGENTon the container is session-origin — it reflects who started the container, not who makes later calls into it, and is forwarded whenever an agent is detected (including inside CI).