CLI telemetry (opt-in, anonymous)

[8bitAlex]

Today PostHog gives us website-level signal: who visits raidcli.dev. It tells us nothing about who actually installs and uses raid. That's a huge blind spot for prioritization — we can't see which task types matter, which commands fail, which features go unused, or what install-base growth looks like over time.

Opt-in anonymous telemetry from the binary itself closes that gap. Industry-standard pattern (Vercel, Vite, Nx, Astro, Tailwind all do this with consent flows).

Out of scope for v1.0

This is not on the v1.0 milestone. Ship v1 first; revisit immediately after. Listed here so the design conversation can start in parallel and the decisions are recorded.

Goals

• Measure adoption: installs, active users, retention
• Measure usage: which task types run, which commands are used, p50/p95 task duration, failure rates by error code
• Stay opt-in, anonymous, and trustworthy enough that we'd run it on our own machine

Non-goals

• Identify users (no email, no machine info beyond OS+arch)
• Capture task content (no cmd: values, no env values, no paths beyond their kind)
• Capture stderr/stdout content
• Run by default — first-run must prompt with a clear opt-in

Proposed events

event	properties (sanitized)
raid_first_run	os, arch, raid_version, install_method (if detectable)
raid_command_executed	command_name, task_count, task_types[], duration_ms, success
raid_command_failed	command_name, error_code (enum from #47), duration_ms
raid_task_executed	task_type, duration_ms, success — sampled to avoid flooding
raid_telemetry_opt_out	reason (optional free-text from raid telemetry off --why)

All events carry: anonymous machine ID (UUIDv4 generated on first run, stored at ~/.config/raid/telemetry-id), raid_version, os, arch.

Consent flow

First run after install:

raid would like to send anonymous usage telemetry to help prioritize features.
We never collect: file paths, command contents, env values, or anything that could identify you.
See: https://raidcli.dev/docs/telemetry

  [y] yes, send telemetry      [N] no, leave it off       [?] what's collected

Default is off (capital N). User can change later:

• raid telemetry on / raid telemetry off
• raid telemetry status (show current state + machine ID)
• raid telemetry purge (delete local ID; PostHog can't link future events to past ones)

Honor DO_NOT_TRACK=1 env var as a hard off, regardless of stored state.

Destination

• PostHog US Cloud, project Raid (id 403603)
• Use the publishable project key (phc_…) baked into the binary at build time
• Capture endpoint: https://us.i.posthog.com/i/v0/e/
• Failure mode: any network error → silent drop, never block a raid command

Trust commitments documented at /docs/telemetry

• What's collected (table above, fully enumerated)
• What's not collected (explicit list)
• How to opt out (3 ways: command, env var, never opt in)
• How to inspect what's about to be sent (raid telemetry preview shows the next event payload before sending it)
• Open-source: the telemetry module is in the same repo, reviewable

Engineering scope

• ~1–2 days of focused work
• New package internal/telemetry/
• New subcommand raid telemetry
• First-run hook in the CLI entry point
• Docs page at /docs/telemetry
• Integration test that asserts opted-out runs send zero network requests

Acceptance criteria

• First run after install presents the opt-in prompt; default is off
• All five events fire correctly when opted in, including sanitized properties only
• Telemetry can be turned off via command, env var (DO_NOT_TRACK=1), and config file
• raid telemetry preview shows a sample event payload without sending it
• raid telemetry status and raid telemetry purge work
• /docs/telemetry page exists with the full disclosure
• Opted-out binary makes zero network requests during raid execution (verified by integration test)
• No PII, no command contents, no file paths in any event property — verified by a unit test that scans all event builders