Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .goreleaser.preview.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ builds:
goarch: arm64
ldflags:
- -s -w
# Preview builds also receive the PostHog API key when
# POSTHOG_API_KEY is set in CI; absent → empty → telemetry
# no-ops. Mirrors the stable goreleaser config.
- -X github.com/8bitalex/raid/src/internal/telemetry.APIKey={{ envOrDefault "POSTHOG_API_KEY" "" }}

archives:
- name_template: "{{ .ProjectName }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}"
Expand Down
6 changes: 6 additions & 0 deletions .goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@ builds:
goarch: arm64
ldflags:
- -s -w
# Inject the PostHog publishable project key at build time. Dev
# builds and `go run` leave this empty, which makes the
# telemetry package no-op (no events sent). Set
# POSTHOG_API_KEY in the release CI environment to populate it
# for tagged builds. See src/internal/telemetry/telemetry.go.
- -X github.com/8bitalex/raid/src/internal/telemetry.APIKey={{ envOrDefault "POSTHOG_API_KEY" "" }}

archives:
- name_template: "{{ .ProjectName }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}"
Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,12 @@ RAID_HEADLESS=1 raid deploy
- `Prompt` tasks **use their `default:`** value instead of reading stdin.
- A `Prompt` **without a `default:`** fails fast with `HEADLESS_PROMPT_NO_DEFAULT` (exit code 3, category `task`) so the variable is never silently set to empty. Add a default for every Prompt you expect CI / agent invocations to run.

### Telemetry (opt-in, anonymous)

raid ships an **opt-in** anonymous CLI telemetry pipeline. It is **off by default** and the first interactive run prompts (capital-N default); non-interactive contexts (no TTY, `--yes`/`--headless`, `--json`, `DO_NOT_TRACK=1`) skip the prompt entirely and stay off. Events carry sanitized properties only — command names, task types, structured error codes, duration — never `cmd:` bodies, paths, env values, or task output.

Manage state with `raid telemetry on / off / status / purge / preview`. `raid telemetry preview` shows the exact payload raid would post (with the API key redacted) so you can audit before opting in. Full disclosure at [raidcli.dev/docs/telemetry](https://raidcli.dev/docs/telemetry).

### `raid <command>`

Run a custom command defined in the active profile or any of its repositories.
Expand Down
1 change: 1 addition & 0 deletions llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Raid is written in Go, distributed as a single self-contained binary, and publis
- [Custom commands](https://raidcli.dev/docs/usage/custom): Define and invoke `raid <cmd>` team workflows
- [raid doctor](https://raidcli.dev/docs/usage/doctor): Diagnose profile and repo configuration issues
- [raid root command](https://raidcli.dev/docs/usage/raid): Global flags and top-level invocation, including [headless mode](https://raidcli.dev/docs/usage/raid#headless-mode) (`-y` / `--yes` / `--headless` / `RAID_HEADLESS=1`) that auto-resolves Confirm and Prompt tasks for CI / agent runs
- [raid telemetry](https://raidcli.dev/docs/telemetry): Opt-in anonymous CLI telemetry. Off by default, first-run consent prompt, `on` / `off` / `status` / `purge` / `preview` subcommands, `DO_NOT_TRACK=1` honored, no PII or task content collected implicitly (the one explicit exception is `raid telemetry off --why "..."`, which records the user-provided opt-out reason on the opt-out event)
- [raid context](https://raidcli.dev/docs/usage/context): Snapshot the active workspace, or run `raid context serve` as an MCP server (stdio) exposing profile, env, repos, commands, recent, and live vars as resources, plus the canonical raid agent toolkit as tools

## Reference
Expand Down
2 changes: 1 addition & 1 deletion site/docs/references/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ raid test-all

Custom commands are defined in `commands` sections of the profile or in individual repository `raid.yaml` files. Run `raid --help` to see all available commands.

Custom command names cannot shadow built-in or reserved names (`profile`, `install`, `env`, `doctor`, `context`, `help`, `version`, `completion`).
Custom command names cannot shadow built-in or reserved names (`profile`, `install`, `env`, `doctor`, `context`, `telemetry`, `help`, `version`, `completion`).

To run a command from a specific repository (e.g. when profiles shadow a repo command with the same name):

Expand Down
106 changes: 106 additions & 0 deletions site/docs/telemetry.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
---
title: Telemetry
sidebar_position: 60
slug: /telemetry
description: What raid's opt-in anonymous CLI telemetry collects, what it never collects, and how to inspect, enable, disable, or purge it.
---

# Telemetry

raid ships an **opt-in, anonymous** CLI telemetry pipeline so the team can prioritize features against real usage signal — which task types matter, which commands fail, which features go unused — without ever capturing what you actually run.

**Off by default.** A fresh install never sends anything to the telemetry endpoint until you explicitly run `raid telemetry on` (or accept the first-run prompt). When opted out, raid makes zero telemetry / PostHog requests, and there's an integration test that pins this contract. (raid still performs an unrelated GitHub release-version check on normal invocations regardless of telemetry state — that request is independent of this pipeline.)

## What raid collects

| Event | Properties (sanitized) |
|---|---|
| `raid_first_run` | `os`, `arch`, `raid_version`, `install_method` (best-effort) |
| `raid_command_executed` | `command_name`, `task_count`, `task_types[]`, `duration_ms`, `success` |
| `raid_command_failed` | `command_name`, `error_code` (from the [errors table](references/errors)), `duration_ms` |
| `raid_task_executed` | `task_type`, `duration_ms`, `success` — **sampled** at ~10% to bound volume |
| `raid_telemetry_opt_out` | `reason` (optional free-text from `raid telemetry off --why "..."`) |

**Every event also carries:** `distinct_id` (anonymous UUIDv4 from `~/.config/raid/telemetry-id`), `raid_version`, `os` (e.g. `darwin`/`linux`/`windows`), `arch` (e.g. `arm64`/`amd64`).

## What raid never collects

- **Command bodies.** `cmd:`, `path:`, `runner:`, `src:`, `dest:`, `url:` — never sent.
- **Variable values.** Anything set by a `Set` task or passed in env vars.
- **Argument values.** `RAID_ARG_*`, declared args, declared flag values. The one exception is the explicit opt-out reason: `raid telemetry off --why "..."` sends the free-text string you typed as the `reason` property on `raid_telemetry_opt_out` — and only on that event. raid never collects argument or flag values implicitly.
- **Stdout / stderr.** Task output is never captured by the telemetry layer.
- **Identifiers.** Username, hostname, IP, MAC, OS version beyond `darwin`/`linux`/`windows`, terminal emulator — none.
- **File paths beyond their kind.** `task_types: ["shell", "shell", "print"]` is fine; the actual paths are not.

The source of truth is the [`src/internal/telemetry`](https://github.com/8bitalex/raid/tree/main/src/internal/telemetry) package. Every event builder is a tiny pure function — read them yourself.

## First-run consent

On your first interactive `raid <command>` invocation after install, raid prints the consent prompt to stderr:

```
raid would like to send anonymous usage telemetry to help prioritize features.
We never collect: file paths, command contents, env values, or anything that could identify you.
See: https://raidcli.dev/docs/telemetry

[y] yes, send telemetry [N] no, leave it off [?] what's collected
>
```

Default is **no** (capital `N`). `[?]` shows the long-form disclosure inline and re-asks.

**Non-interactive contexts skip the prompt entirely and leave telemetry off.** Specifically, raid does not prompt when:

- `DO_NOT_TRACK=1` is set in the environment (cross-tool standard — [consoledonottrack.com](https://consoledonottrack.com)).
- `-y`, `--yes`, or `--headless` is on the command line (see [Headless mode](usage/raid#headless-mode)).
- `--json` is on the command line (machine-readable output mode).
- stdin isn't a TTY (CI runners, pipes, agent hosts).
- This raid build wasn't compiled with a PostHog API key (dev builds — telemetry is dead code).
- The invocation is a `raid telemetry ...` subcommand (so you can run `raid telemetry on` without being prompted to opt in first).

In each of these cases raid records the decision as off-by-default so it won't try to prompt later either. Switch back on explicitly with `raid telemetry on`.

## Managing telemetry

```bash
raid telemetry on # opt in
raid telemetry off # opt out
raid telemetry off --why "ci runner" # opt out + record an anonymous reason
raid telemetry status # show current state, anonymous ID, ID file path
raid telemetry status --json # same, as JSON
raid telemetry purge # delete the anonymous ID file (breaks linkage to past events)
raid telemetry preview # render a sample event payload — does not send
```

`raid telemetry preview` is the recommended way to see exactly what raid would post before opting in. The API key is automatically redacted in the preview output so you can paste it anywhere.

## Three ways to opt out

1. **`raid telemetry off`** — persists the choice; future runs stay off.
2. **`DO_NOT_TRACK=1`** — env var; overrides the persisted state for the current process and any child it spawns. Honored on every run regardless of `telemetry on`.
3. **Never opt in.** The first-run prompt defaults to off, and non-interactive contexts skip the prompt entirely.

You can purge the anonymous machine ID at any time with `raid telemetry purge` — that breaks linkage between future events and any sent before. Useful if you've been opted in for a while and want to reset.

## Destination

- **PostHog US Cloud**, project `Raid` (id `403603`).
- Endpoint: `https://us.i.posthog.com/i/v0/e/`.
- The PostHog publishable project key is baked into release builds at compile time via a ldflag. Dev / `go run` builds have no key and never send.

Network failures are **always silent**. A 2-second HTTP timeout, fire-and-forget goroutines, and a 1.5-second flush deadline at process exit mean a stuck network can't slow raid down or break a command.

## Source code

The entire telemetry implementation is in [`src/internal/telemetry`](https://github.com/8bitalex/raid/tree/main/src/internal/telemetry):

| File | Role |
|---|---|
| [`telemetry.go`](https://github.com/8bitalex/raid/blob/main/src/internal/telemetry/telemetry.go) | `Capture`, `Flush`, the HTTP send path, `PreviewPayload` |
| [`consent.go`](https://github.com/8bitalex/raid/blob/main/src/internal/telemetry/consent.go) | Consent state (read/write via viper) + `DO_NOT_TRACK` honor |
| [`id.go`](https://github.com/8bitalex/raid/blob/main/src/internal/telemetry/id.go) | Anonymous UUIDv4 generation + persistence + purge |
| [`events.go`](https://github.com/8bitalex/raid/blob/main/src/internal/telemetry/events.go) | Event builders. Every builder is a few lines — easy to audit. |
| [`sampling.go`](https://github.com/8bitalex/raid/blob/main/src/internal/telemetry/sampling.go) | Per-task sample rate |
| [`prompt.go`](https://github.com/8bitalex/raid/blob/main/src/internal/telemetry/prompt.go) | First-run consent prompt flow |

The test file [`telemetry_test.go`](https://github.com/8bitalex/raid/blob/main/src/internal/telemetry/telemetry_test.go) pins the contracts above: zero network calls when opted out, zero network calls under `DO_NOT_TRACK`, sanitization of every event builder, UUIDv4 format of the anonymous ID, and the redacted-key preview output.
2 changes: 1 addition & 1 deletion site/docs/usage/custom.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ Each repository that defines commands appears as a subcommand in `raid --help`.

## Constraints

Custom command names cannot shadow reserved built-in CLI names: `profile`, `install`, `env`, `doctor`, `context`, `help`, `version`, `completion`.
Custom command names cannot shadow reserved built-in CLI names: `profile`, `install`, `env`, `doctor`, `context`, `telemetry`, `help`, `version`, `completion`.

## Running tasks in parallel

Expand Down
3 changes: 2 additions & 1 deletion site/docs/usage/raid.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ The third row is deliberate: silently setting the variable to an empty string wo
| [`profile`](./profile) | Create, add, list, switch, or remove profiles |
| [`doctor`](./doctor) | Check the active configuration for issues |
| [`context`](./context) | Print workspace snapshot; `context serve` runs the MCP server |
| [`telemetry`](../telemetry) | Manage anonymous CLI telemetry (off by default; `on` / `off` / `status` / `purge` / `preview`) |
| [`completion`](#shell-completion) | Generate shell autocompletion scripts |

Custom commands defined in the active profile or its repositories are also available as `raid <name>`. See [Custom Commands](./custom) for details.
Expand Down Expand Up @@ -106,6 +107,6 @@ Run `raid completion <shell> --help` for detailed instructions for each shell.

The following names are reserved for built-in commands and cannot be used as custom command names:

`profile`, `install`, `env`, `doctor`, `context`, `help`, `version`, `completion`
`profile`, `install`, `env`, `doctor`, `context`, `telemetry`, `help`, `version`, `completion`
Comment on lines 108 to +110

If a custom command in your profile uses a reserved name, it is ignored and a warning is printed.
4 changes: 4 additions & 0 deletions site/docs/whats-new.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ description: Feature-by-feature release notes for Raid.

User-visible changes per release, latest first. For full commit history see the [GitHub releases page](https://github.com/8bitalex/raid/releases).

## 0.16.0 — upcoming

**Opt-in anonymous CLI telemetry.** raid can now send anonymous usage events to PostHog so the project can prioritize features against real signal (which task types matter, which commands fail, which features go unused). **Off by default** — a fresh install never sends anything until you opt in. The first interactive run prompts (capital-N default); non-interactive contexts (TTY-less stdin, `--yes`/`--headless`, `--json`, `DO_NOT_TRACK=1`, or a `raid telemetry ...` invocation) skip the prompt and leave telemetry off. New `raid telemetry on / off / status / purge / preview` subcommand manages the state; `preview` renders the exact payload raid would post (with the API key redacted) so you can audit before opting in. Captured events carry sanitized properties only — command names and task types, never `cmd:` bodies, paths, env values, or stdout/stderr. See the full disclosure at [/docs/telemetry](/docs/telemetry). Closes [#80](https://github.com/8bitAlex/raid/issues/80).
Comment on lines +12 to +14

## 0.15.0 — upcoming

**Headless mode for CI, scheduled runs, and agents.** A new top-level `-y` / `--yes` / `--headless` flag (and `RAID_HEADLESS=1` env-var equivalent) auto-resolves interactive prompts so non-interactive callers no longer deadlock on a `Confirm` or `Prompt`. `Confirm` auto-accepts; `Prompt` skips stdin and uses its `default:` value; a `Prompt` without a default fails fast with a structured `HEADLESS_PROMPT_NO_DEFAULT` error (exit code `3`) instead of silently setting the variable to empty. The flag and env var are interchangeable — the flag works by setting the env var, so a single read site in lib serves both the CLI and programmatic entry points. Headless auto-accepts every `Confirm`, so stronger destructive-action guardrails must be expressed via a [`verify:`](/docs/references/schema#verify) entry, a `condition:`, or an explicit env-var check. See [raid → Headless mode](/docs/usage/raid#headless-mode). Closes [#67](https://github.com/8bitAlex/raid/issues/67).
Expand Down
93 changes: 93 additions & 0 deletions src/cmd/raid.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@ import (
"github.com/8bitalex/raid/src/cmd/env"
"github.com/8bitalex/raid/src/cmd/install"
"github.com/8bitalex/raid/src/cmd/profile"
telemetrycmd "github.com/8bitalex/raid/src/cmd/telemetry"
"github.com/8bitalex/raid/src/internal/lib"
"github.com/8bitalex/raid/src/internal/sys"
"github.com/8bitalex/raid/src/internal/telemetry"
"github.com/8bitalex/raid/src/raid"
"github.com/8bitalex/raid/src/raid/errs"
"github.com/spf13/cobra"
Expand All @@ -28,6 +30,7 @@ var reservedNames = map[string]bool{
"env": true,
"doctor": true,
"context": true,
"telemetry": true,
"help": true,
"version": true,
"completion": true,
Expand Down Expand Up @@ -59,6 +62,7 @@ func init() {
rootCmd.AddCommand(env.Command)
rootCmd.AddCommand(doctor.Command)
rootCmd.AddCommand(contextcmd.Command)
rootCmd.AddCommand(telemetrycmd.Command)
}

// isInfoCommand reports whether the invocation is for a built-in informational
Expand Down Expand Up @@ -162,6 +166,32 @@ func executeRoot(args []string) int {
// caller is providing a different args list (e.g. during tests).
rootCmd.SetArgs(args[1:])

// First-run consent prompt for telemetry. Runs only for non-info,
// non-telemetry-subcommand invocations to avoid prompting on
// `raid --help`, `raid telemetry on`, and similar. The prompt
// itself no-ops when stdin isn't a TTY, when --yes/--headless is
// set, when --json is set, or when RAID_HEADLESS=1 in the env
// (CI / agent-host opt-in path), so it's safe in non-interactive
// contexts. See telemetry.MaybePromptForConsent for the full skip
// matrix.
if !info && !isTelemetrySubcommand(args) {
skip := headlessFromArgs(args) || jsonModeFromArgs(args) || lib.IsHeadless()
switch telemetry.MaybePromptForConsent(skip) {
case telemetry.PromptAccepted:
// User opted in via the first-run prompt — fire the
// adoption event so `raid telemetry on` and the prompt
// path both produce raid_first_run. Synchronous so the
// event lands even if the user's command crashes before
// Flush runs.
telemetry.CaptureSync(telemetry.EventFirstRun, telemetry.FirstRunProps(""))
}
}

// Flush any pending telemetry events before exit so async sends
// don't get dropped when raid returns. The deadline is short so a
// stuck network can't drag out shutdown.
defer telemetry.Flush(1500 * time.Millisecond)

if err := rootCmd.Execute(); err != nil {
rErr, isStructured := errs.AsError(err)
var exitErr *exec.ExitError
Expand Down Expand Up @@ -217,6 +247,69 @@ func applyHeadlessFlag(cmd *cobra.Command, _ []string) error {
return nil
}

// isTelemetrySubcommand reports whether the user invoked one of the
// `raid telemetry ...` subcommands. The first-run consent prompt must
// skip these — prompting "do you want telemetry?" right before
// running `raid telemetry on` is hostile UX, and the off/status/
// purge/preview commands need to work for users who haven't opted in.
//
// Flag-aware: persistent flags that take a value (`--config <path>`
// / `-c <path>`) consume the following token, so an invocation like
// `raid --config telemetry install` should resolve to the `install`
// subcommand and not be misread as `telemetry`. The bool persistent
// flags (`--json`, `--yes`/`-y`, `--headless`) do not consume a
// following token.
func isTelemetrySubcommand(args []string) bool {
skipNext := false
for _, a := range args[1:] {
if a == "--" {
return false
}
if skipNext {
skipNext = false
continue
}
if strings.HasPrefix(a, "-") {
// Only the value-taking root flags consume the next
// token (and only in their bare form — `--config=path`
// keeps the value attached).
if a == "--config" || a == "-c" {
skipNext = true
}
continue
}
return a == "telemetry"
Comment on lines +264 to +281
}
return false
}

// headlessFromArgs is the early-scan counterpart to jsonModeFromArgs
// for the headless persistent flag. The first-run prompt needs to
// know the user's headless intent before cobra has parsed flags so
// it can skip prompting in non-interactive contexts. Matches every
// flag form: `-y`, `--yes`, `--yes=true`, `--headless`,
// `--headless=true`, plus their explicit `=false` opt-outs.
//
// Mirrors pflag's "last value wins" behavior: if the user passes
// `--yes=true --yes=false`, the parsed value is false, so this scan
// must also resolve to false. We walk the full arg list and only
// commit the final occurrence.
func headlessFromArgs(args []string) bool {
out := false
for _, a := range args[1:] {
if a == "--" {
break
}
switch {
case a == "-y", a == "--yes", a == "--yes=true", a == "--headless", a == "--headless=true":
out = true
case a == "--yes=false", a == "--headless=false":
out = false
}
}
return out
}

// jsonModeFromArgs reports whether the user passed `--json` (or
// `--json=true`) anywhere in args. Cobra resolves persistent flags
// during Execute, but on the error path we need to know before falling
Expand Down
Loading
Loading