Skip to content

Add minimal, anonymous usage telemetry#292

Open
sdairs wants to merge 9 commits into
mainfrom
issue-283-usage-telemetry
Open

Add minimal, anonymous usage telemetry#292
sdairs wants to merge 9 commits into
mainfrom
issue-283-usage-telemetry

Conversation

@sdairs

@sdairs sdairs commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Closes #283

Default-on anonymous usage telemetry with a Homebrew-style consent model: nothing is ever sent before the first-run notice has been shown.

Consent & state

  • ~/.clickhouse/telemetry.json ({"disabled": false}) doubles as the first-run marker. First run: write file → print notice to stderr (regardless of TTY) → send nothing. The notice is gated on the write succeeding, so an unwritable config dir fails open to disabled — no send, no error, no repeated notice.
  • DO_NOT_TRACK (donottrack.sh spec: set and not ""/"0"/"false") is checked first and overrides everything.
  • clickhousectl telemetry enable|disable|status manages the file. Consent is evaluated after the command runs, so telemetry disable silences its own event and telemetry enable sends one.

Payload

{"command":"local list","flags":["json"],"exit_code":0,"is_agent":true,"agent":"claude-code","ci":false,"version":"0.3.1","os":"macos","arch":"aarch64"}

command (path), flags (long names only), exit_code (gh-style: 0 success, 1 error, 2 cancelled, 4 auth required), is_agent + agent (canonical id from is-ai-agent, null for humans), ci (CI env), version, os, arch. No install ID, no fingerprinting. Agent, version, and exit code are set client-side from structured values rather than extracted worker-side from headers.

Values are structurally unreachable: main now parses via ArgMatches, and telemetry::capture walks only arg ids and Arg metadata (never get_one/get_raw), skipping positionals and non-command-line sources. A unit fixture asserts a payload built from cloud service get SECRET-ID --org-id SECRET-ORG contains no SECRET, and an integration test asserts the same over the wire.

Transport

Detached child process (telemetry send, hidden subcommand): payload via child env var, all stdio nulled, parent never waits; child fires one POST with a 2s timeout and dies silently on any failure. Measured ~0.3s command wall-clock with the endpoint blackholed. The child is short-circuited in main before dispatch, so a send can never trigger another send.

Every POST carries the canonical User-Agent (clickhousectl/<version>, optionally (agent=<id>)) via the shared http::client_builder(), pinned by a wire-level test — the ingest worker can reject anything without the clickhousectl/ prefix.

Transparency & opt-outs

  • CHCTL_TELEMETRY_DEBUG=1 prints the exact payload to stderr and sends nothing.
  • CHCTL_TELEMETRY_URL overrides the endpoint (tests / local worker dev).
  • Cargo feature telemetry (default-on): --no-default-features compiles the whole thing out including the subcommand — enforced by new CI steps.
  • README section documenting all of the above.

Tests

  • Unit: consent state machine (DNT precedence, first-run, unwritable dir, corrupt file), donottrack truthiness, payload key set + agent-field consistency, capture fixtures (no-leak, global-flag dedupe, default-value exclusion).
  • Integration (tests/telemetry_test.rs, subprocess + wiremock + sandboxed HOME): first-run notice/marker/no-send, payload shape + User-Agent prefix, failure exit code + positional non-leak, DNT silence, disable/enable persistence, debug mode, unwritable-HOME fail-open, parent-never-waits latency.
  • Existing subprocess tests now set DO_NOT_TRACK=1 so test runs never write real state or POST to production.

Verified end-to-end against the telemetry worker running locally (wrangler dev): first run silent, subsequent runs delivered events the worker accepted and inserted (200 OK).

Worker-side follow-ups (separate repo, in dev)

  • exit_code (number) replaces success (bool) — needs a numeric coercer + column (success is derivable as exit_code = 0).
  • is_agent is now a payload bool (was agent), and agent is the agent id string (null for humans) — needs a column + coercion update in toRow.
  • Optionally reject requests whose User-Agent doesn't start with clickhousectl/.

Pre-release TODO (not this PR)

🤖 Generated with Claude Code

Default-on telemetry with a Homebrew-style consent model (issue #283):
nothing is ever sent before the first-run notice has been shown.

- State/marker: ~/.clickhouse/telemetry.json ({"disabled": false}); first
  run writes the file, prints a notice to stderr, and sends nothing. The
  notice is gated on the write succeeding, so an unwritable config dir
  fails open to disabled with no repeated notice.
- DO_NOT_TRACK (donottrack.sh) overrides everything; CHCTL_TELEMETRY_DEBUG=1
  prints the exact payload to stderr and sends nothing.
- Payload is command path + flag NAMES only, built by walking the clap
  Command/ArgMatches metadata (never get_one/get_raw), so leaking values
  or positionals is structurally impossible. Plus success bool, agent/CI
  detection, version, os/arch.
- Transport is a detached `telemetry send` child (hidden subcommand, all
  stdio nulled, 2s timeout, parent never waits): zero latency impact even
  with the endpoint blackholed.
- `clickhousectl telemetry enable|disable|status` manages consent; the
  whole feature (including the subcommand) compiles out with
  --no-default-features for distro packagers, enforced in CI.
- Existing subprocess tests now set DO_NOT_TRACK=1 so test runs never
  write real state or post to the production endpoint.

Verified end-to-end against the telemetry worker running locally via
wrangler dev: first run silent, subsequent runs deliver events that the
worker accepts and inserts (200 OK).

Closes #283

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sdairs
sdairs requested a review from iskakaushik as a code owner July 7, 2026 16:33
Comment thread crates/clickhousectl/src/telemetry.rs
Comment thread crates/clickhousectl/src/telemetry.rs
Comment thread crates/clickhousectl/src/telemetry.rs
sdairs and others added 2 commits July 7, 2026 18:35
… prefix

The send child already goes through http::client_builder(), so every
telemetry POST carries the same User-Agent as all other outbound HTTP:
`clickhousectl/<version>`, optionally followed by an ` (agent=<id>)`
comment. Assert it wire-level so the ingest worker can safely reject
traffic without that prefix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The CLI already holds the detected agent as a structured value via
is-ai-agent, so set both facts client-side — is_agent (bool) and agent
(canonical id, e.g. "claude-code"; null for humans) — rather than having
the ingest worker string-parse them out of the User-Agent header. Same
for version: the payload value from CARGO_PKG_VERSION stays canonical.
The User-Agent still carries the same facts for prefix-based request
filtering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread README.md
The finalize hook already receives the exact exit code the process is
about to exit with (Error::exit_code: 0 success, 1 error, 2 cancelled,
4 auth required), so send that: success is derivable (exit_code = 0)
but the code also distinguishes cancellations and auth failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread crates/clickhousectl/src/telemetry.rs
Comment thread crates/clickhousectl/src/telemetry.rs
sdairs and others added 4 commits July 15, 2026 19:32
The telemetry send child used the shared http::client_builder(), which
attaches agent-session-id and traceparent default headers when running
under an AI coding agent — a stable session identifier on every
anonymous telemetry event. Build the client directly with only the
canonical User-Agent and the send timeout; the agent facts telemetry
needs already travel in the payload (is_agent/agent) by design.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
std::env::var errors on set-but-non-UTF-8 values, so real_env_lookup
returned None and the opt-out failed open. Use var_os with a lossy
conversion instead: the resulting non-empty string reads as truthy and
telemetry stays fully silent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`telemetry enable` records the preference even when DO_NOT_TRACK is set
(intentional, so it takes effect once DNT is lifted), but DNT overrides
everything at send time. Print a stderr note alongside the stdout
confirmation so the user isn't told "Telemetry enabled." while nothing
will actually be sent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sdairs
sdairs requested a review from rndD as a code owner July 15, 2026 18:39

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

There are 5 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 428bf07. Configure here.

Comment thread crates/clickhousectl/src/telemetry.rs
Comment thread crates/clickhousectl/src/telemetry.rs
Comment thread crates/clickhousectl/src/telemetry.rs
The Missing-state status said 'first-run notice pending', which reads as a
promise that never completes when ~/.clickhouse is unwritable (finalize fails
open to silent) and is stale within the same invocation once finalize prints
the notice. State the durable fact instead: nothing has been sent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add minimal, anonymous usage telemetry

1 participant