feat: opt-in anonymous CLI telemetry (closes #80)#91
Merged
Conversation
raid now ships an opt-in, anonymous CLI telemetry pipeline so the project can prioritize features against real usage signal without ever capturing what users actually run. Off by default. First interactive run prompts ([y/N/?] — capital N default). Non-interactive contexts (no TTY on stdin, --yes/--headless, --json, DO_NOT_TRACK=1, or a `raid telemetry ...` invocation) skip the prompt entirely and persist consent=off so we never re-prompt. Build-time PostHog API key injection via ldflags; dev builds have no key, telemetry is dead code in them. New `raid telemetry on / off / status / purge / preview` subcommand: status shows current state + anonymous ID; preview renders the exact JSON payload raid would post (with the API key redacted) so the user can audit before opting in; purge deletes the anonymous machine ID so future events can't be linked to past ones. Event hooks fire from ExecuteCommand, ExecuteRepoCommand, and ExecuteTask. The per-task event is sampled at 10% to bound volume for commands with many tasks. Every event property is sanitized: command names and task types only, never cmd: bodies, paths, env values, or stdout/stderr. Tests pin the contracts: zero network calls when opted out, zero network calls under DO_NOT_TRACK, every event builder is free of forbidden content, UUIDv4 format is correct, the preview output redacts the API key. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #91 +/- ##
==========================================
- Coverage 92.28% 90.86% -1.43%
==========================================
Files 38 46 +8
Lines 3474 3918 +444
==========================================
+ Hits 3206 3560 +354
- Misses 175 235 +60
- Partials 93 123 +30 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds opt-in anonymous CLI telemetry for raid, including consent handling, PostHog event delivery, CLI controls, release-time API key injection, and public documentation.
Changes:
- Introduces
internal/telemetryfor consent state, anonymous ID management, event builders, sampling, preview payloads, and async capture. - Adds
raid telemetry on/off/status/purge/previewand hooks telemetry into command/task execution and root startup. - Updates release config, version, README/site docs, and tests for telemetry behavior.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 21 comments.
Show a summary per file
| File | Description |
|---|---|
src/resources/app.properties |
Bumps app version to 0.16.0-beta. |
src/internal/telemetry/version.go |
Reads raid version for telemetry payloads. |
src/internal/telemetry/telemetry.go |
Implements capture, send, flush, enrichment, and preview payloads. |
src/internal/telemetry/telemetry_test.go |
Adds telemetry unit/integration tests. |
src/internal/telemetry/sampling.go |
Adds task telemetry sampling. |
src/internal/telemetry/prompt.go |
Adds first-run consent prompt flow. |
src/internal/telemetry/id.go |
Adds anonymous UUID persistence and purge. |
src/internal/telemetry/events.go |
Adds event property builders. |
src/internal/telemetry/consent.go |
Adds consent state and DO_NOT_TRACK handling. |
src/internal/lib/task_runner.go |
Emits sampled task telemetry. |
src/internal/lib/command.go |
Emits custom command telemetry. |
src/cmd/telemetry/telemetry.go |
Adds telemetry Cobra subcommands. |
src/cmd/telemetry/telemetry_test.go |
Tests telemetry subcommands. |
src/cmd/raid.go |
Registers telemetry command and first-run prompt/flush hooks. |
site/docs/whats-new.mdx |
Adds 0.16 telemetry release note. |
site/docs/usage/raid.mdx |
Documents telemetry command and reserved name. |
site/docs/telemetry.mdx |
Adds telemetry disclosure page. |
README.md |
Adds telemetry summary. |
llms.txt |
Adds telemetry docs entry. |
.goreleaser.yaml |
Injects telemetry API key for releases. |
.goreleaser.preview.yaml |
Injects telemetry API key for preview releases. |
| // is set, so this is safe in CI / pipes / agent hosts. See | ||
| // telemetry.MaybePromptForConsent for the full skip matrix. | ||
| if !info && !isTelemetrySubcommand(args) { | ||
| _ = telemetry.MaybePromptForConsent(headlessFromArgs(args) || jsonModeFromArgs(args)) |
Comment on lines
107
to
+110
| startedAt := RecordRecentStart(found.Name) | ||
| err := runCommand(found) | ||
| RecordRecentEnd(found.Name, err, startedAt) | ||
| captureCommandTelemetry(found, err, time.Since(startedAt)) |
| // is set, so this is safe in CI / pipes / agent hosts. See | ||
| // telemetry.MaybePromptForConsent for the full skip matrix. | ||
| if !info && !isTelemetrySubcommand(args) { | ||
| _ = telemetry.MaybePromptForConsent(headlessFromArgs(args) || jsonModeFromArgs(args)) |
Comment on lines
+245
to
+252
| for _, a := range args[1:] { | ||
| if a == "--" { | ||
| return false | ||
| } | ||
| if strings.HasPrefix(a, "-") { | ||
| continue | ||
| } | ||
| return a == "telemetry" |
Comment on lines
+12
to
+14
| ## 0.16.0 — upcoming | ||
|
|
||
| **Opt-in anonymous CLI telemetry.** raid can now send anonymous usage events to PostHog so the project can prioritize features against real signal (which task types matter, which commands fail, which features go unused). **Off by default** — a fresh install never sends anything until you opt in. The first interactive run prompts (capital-N default); non-interactive contexts (TTY-less stdin, `--yes`/`--headless`, `--json`, `DO_NOT_TRACK=1`, or a `raid telemetry ...` invocation) skip the prompt and leave telemetry off. New `raid telemetry on / off / status / purge / preview` subcommand manages the state; `preview` renders the exact payload raid would post (with the API key redacted) so you can audit before opting in. Captured events carry sanitized properties only — command names and task types, never `cmd:` bodies, paths, env values, or stdout/stderr. See the full disclosure at [/docs/telemetry](/docs/telemetry). Closes [#80](https://github.com/8bitAlex/raid/issues/80). |
Comment on lines
+22
to
+25
| // idMu guards the in-process cache around the ID file so concurrent | ||
| // Capture calls don't race on the read/write. The file itself is | ||
| // effectively single-writer (the user, on this machine), so we don't | ||
| // need OS-level locking. |
| - [Custom commands](https://raidcli.dev/docs/usage/custom): Define and invoke `raid <cmd>` team workflows | ||
| - [raid doctor](https://raidcli.dev/docs/usage/doctor): Diagnose profile and repo configuration issues | ||
| - [raid root command](https://raidcli.dev/docs/usage/raid): Global flags and top-level invocation, including [headless mode](https://raidcli.dev/docs/usage/raid#headless-mode) (`-y` / `--yes` / `--headless` / `RAID_HEADLESS=1`) that auto-resolves Confirm and Prompt tasks for CI / agent runs | ||
| - [raid telemetry](https://raidcli.dev/docs/telemetry): Opt-in anonymous CLI telemetry. Off by default, first-run consent prompt, `on` / `off` / `status` / `purge` / `preview` subcommands, `DO_NOT_TRACK=1` honored, no PII or task content ever collected |
Comment on lines
+264
to
+275
| for _, a := range args[1:] { | ||
| if a == "--" { | ||
| break | ||
| } | ||
| switch { | ||
| case a == "-y", a == "--yes", a == "--yes=true", a == "--headless", a == "--headless=true": | ||
| return true | ||
| case a == "--yes=false", a == "--headless=false": | ||
| return false | ||
| } | ||
| } | ||
| return false |
Comment on lines
108
to
+110
| The following names are reserved for built-in commands and cannot be used as custom command names: | ||
|
|
||
| `profile`, `install`, `env`, `doctor`, `context`, `help`, `version`, `completion` | ||
| `profile`, `install`, `env`, `doctor`, `context`, `telemetry`, `help`, `version`, `completion` |
Comment on lines
+208
to
+211
| // Sampling and Capture both fast-path when telemetry is off, so the | ||
| // per-task overhead when opted out is effectively zero. | ||
| func captureTaskTelemetry(task Task, err error, dur time.Duration) { | ||
| if !telemetry.Sampled() { |
- Fire raid_first_run after a prompt-based opt-in (previously only the explicit `raid telemetry on` path produced the adoption event). - Honor RAID_HEADLESS=1 when deciding whether to skip the consent prompt, matching how the headless flag itself is treated. - isTelemetrySubcommand now accounts for value-taking root flags (--config / -c), so `raid --config telemetry install` resolves to the `install` subcommand rather than misreading the config path as the telemetry subcommand. - headlessFromArgs walks the full arg list and resolves to the final value, mirroring pflag's last-value-wins semantics. - loadOrCreateID writes the ID with O_CREATE|O_EXCL so two concurrent first-run raid invocations can't generate competing UUIDs; the loser re-reads the winner's value. Co-Authored-By: Copilot <copilot@github.com>
Sampled() now short-circuits when telemetry is inactive, so the existing rate=1 / intermediate tests must enable consent before exercising the rate logic. Adds an explicit inactive-fast-path test to pin the new behavior.
Adds tests for paths the initial PR didn't reach: - loadOrCreateID race-loser path (existing ID file is adopted, not overwritten) and the home-dir-error fast fail. - loadIDIfExists empty-on-missing. - CaptureSync send + no-op-when-inactive. - send() swallows network errors silently (Flush still returns). - PreviewPayload renders a placeholder when no ID exists. - headlessFromArgs last-value-wins (mirrors pflag). - isTelemetrySubcommand flag-value handling (--config / -c).
Owner
Author
|
Auto-review by meeseeks Updates pushed: 3 commits
Copilot comments addressed: 17 of 21
Skipped: 4
Codecov patch: 79.32% (project 90.86%) — needs human. One honest attempt added ~70 lines of new tests for Needs human: decide whether to (a) merge as-is and accept the -1.43% project-coverage delta, (b) accept the architectural decision that built-in subcommands don't emit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #80. raid now ships an opt-in, anonymous CLI telemetry pipeline so the project can prioritize features against real usage signal without ever capturing what users actually run. Off by default.
First-run consent. First interactive run prompts
[y/N/?](capital-N default;?shows the long-form disclosure inline and re-asks). Non-interactive contexts skip the prompt entirely and persist consent=off so we never re-prompt:DO_NOT_TRACK=1env var--yes/-y/--headlessflag--jsonflag (machine-readable output mode)go runbuilds — telemetry is dead code)raid telemetry ...subcommandEvents. Five events per the issue spec —
raid_first_run,raid_command_executed,raid_command_failed,raid_task_executed(sampled ~10% to bound volume),raid_telemetry_opt_out. Every event carries an anonymous UUIDv4 from~/.config/raid/telemetry-id, the raid version, OS, and arch. Sanitization is enforced by construction: every event builder is a small pure function that only accepts the permitted slots (command name, task type, error code, duration, success). Nocmd:bodies, paths, env values, or task output ever reach the payload.New subcommand.
raid telemetry on / off / status / purge / preview.status --jsonis parseable.previewrenders the exact JSON payload raid would post — with the API key automatically redacted (phc_…1234) — so users can audit before opting in.purgedeletes the anonymous machine ID so future events can't be linked to past ones.Build-time API key.
var APIKey stringin the telemetry package, injected via-ldflags -X .../telemetry.APIKey={{ envOrDefault \"POSTHOG_API_KEY\" \"\" }}in both.goreleaser.yamland.goreleaser.preview.yaml. Dev builds get empty key → telemetry is a no-op at every call site. SetPOSTHOG_API_KEYin the release CI environment to populate it for tagged builds.Lifecycle / safety.
Capturereturns immediately; the HTTP POST runs on a fire-and-forget goroutine with a 2-second timeout.executeRootdefersFlush(1500ms)so in-flight events finish (or get dropped on timeout) before raid exits. Any network error is silently swallowed — telemetry never breaks raid.Version bumped 0.15.0-beta → 0.16.0-beta. Full disclosure at /docs/telemetry.
Test plan
go test ./... -race— all packages greenIsActivetable: off by default; off when API key empty; off whenDO_NOT_TRACK=1; on only when API key + decided + enabled + not DO_NOT_TRACKDO_NOT_TRACK=1(integration test)Captureround-trip: payload structure has expected fields, no leaksPurgeIDremoves the file and is idempotentPreviewPayloadredacts the API keyDO_NOT_TRACK, already-decided; accepts ony; declines on empty; explainer-then-accept via?\nySetEnabledpersists bothdecidedandenabledkeys to vipercd site && npm run build— no broken links🤖 Generated with Claude Code