Skip to content

feat(cli): add opt-out anonymous telemetry via PostHog#52

Merged
jrusso1020 merged 6 commits intomainfrom
feat/telemetry-posthog
Mar 25, 2026
Merged

feat(cli): add opt-out anonymous telemetry via PostHog#52
jrusso1020 merged 6 commits intomainfrom
feat/telemetry-posthog

Conversation

@jrusso1020
Copy link
Copy Markdown
Collaborator

@jrusso1020 jrusso1020 commented Mar 25, 2026

Summary

  • Add anonymous usage telemetry to the CLI via PostHog's HTTP batch API (zero new dependencies)
  • Track command invocations, render performance, template choices, and environment info
  • Add hyperframes telemetry [enable|disable|status] command for user control
  • Config stored at ~/.hyperframes/config.json — future-proofed for more settings

Design Decisions

Decision Rationale
Raw fetch instead of posthog-node Zero new dependencies. Node 22 has built-in fetch. Can swap to SDK later if needed.
Opt-out with first-run disclosure Industry standard (Next.js, Homebrew, .NET CLI). Opt-in gets <3% participation.
Disabled in dev mode Uses .ts extension detection (shared utils/env.ts). Running via tsx = dev.
phc_ prefix check Safety net — if the API key is ever reverted to a placeholder, telemetry silently disables.
5-second timeout, fail-silent Telemetry must never slow down or break the CLI.
Detached spawn for exit flush flushSync spawns a detached child process so process.exit() paths don't block.

What's Collected

  • Command names (init, render, dev, etc.)
  • Render metrics (duration, fps, quality, workers, docker/gpu)
  • Template choices during init
  • OS, architecture, Node.js version, CLI version

What's NOT Collected

  • File paths, project names, or video content
  • IP addresses — $ip: null on every event payload (client-side) + "Discard client IP data" enabled in PostHog project settings (server-side)
  • Any personally identifiable information

Opt-Out Mechanisms

  • hyperframes telemetry disable
  • HYPERFRAMES_NO_TELEMETRY=1
  • DO_NOT_TRACK=1
  • Automatically disabled in CI (CI=true)

Files Changed

New files:

  • packages/cli/src/telemetry/config.ts — Config read/write at ~/.hyperframes/config.json (dir 0700, file 0600)
  • packages/cli/src/telemetry/client.ts — PostHog HTTP client (queue, batch, flush, detached flushSync)
  • packages/cli/src/telemetry/events.ts — Typed event helpers
  • packages/cli/src/telemetry/index.ts — Barrel exports
  • packages/cli/src/commands/telemetry.tshyperframes telemetry command
  • packages/cli/src/utils/env.ts — Shared isDevMode() (extracted from dev.ts)

Modified files:

  • packages/cli/src/cli.ts — Wire telemetry at entry point + add telemetry subcommand
  • packages/cli/src/commands/render.ts — Track render success/failure metrics
  • packages/cli/src/commands/init.ts — Track template selection
  • packages/cli/src/commands/browser.ts — Track browser download events
  • packages/cli/src/commands/dev.ts — Use shared isDevMode() from utils/env.ts

Testing

  • Verified typecheck passes (tsc --noEmit)
  • Verified lint passes (oxlint)
  • Verified format passes (oxfmt --check)
  • Tested hyperframes telemetry status/enable/disable commands
  • Verified first-run notice is suppressed in dev mode
  • Verified --help/--version don't trigger telemetry
  • Verified config file creation with correct permissions
  • Verified telemetry is no-op when API key lacks phc_ prefix

🤖 Generated with Claude Code

jrusso1020 and others added 3 commits March 25, 2026 22:55
Add anonymous usage telemetry to help improve the CLI. Uses PostHog's
HTTP batch API directly (zero new dependencies) with a 5-second timeout
and fail-silent behavior — telemetry never breaks the CLI.

What's collected: command names, render performance (duration, fps,
quality), template choices, OS/arch/Node version/CLI version.

What's NOT collected: file paths, project names, video content, or
any personally identifiable information.

Telemetry is:
- Disabled in dev mode (running via tsx)
- Disabled in CI (CI=true) or via HYPERFRAMES_NO_TELEMETRY=1
- Disabled when API key is placeholder (safe to merge before key is set)
- Controllable via `hyperframes telemetry [enable|disable|status]`
- Disclosed on first run with clear opt-out instructions

Config stored at ~/.hyperframes/config.json (0600 permissions).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Extract shared isDevMode() to utils/env.ts (was duplicated in dev.ts and client.ts)
- Use ui/colors.ts instead of raw ANSI escapes in telemetry notice (respects NO_COLOR)
- Derive known commands from subCommands object instead of maintaining duplicate set
- Skip telemetry on --help/--version and unknown commands
- Gate incrementCommandCount() behind shouldTrack() (no disk writes in CI)
- Add flushSync() for process.exit() paths (beforeExit doesn't fire on explicit exit)
- Remove dead trackBrowserInstall(success) param (failure path never called it)
- Remove redundant isEnabled/anonymousId caching in client.ts (config.ts cache suffices)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PostHog's $ip: null property tells the server to not associate the
request IP with the event. Combined with the "Discard client IP data"
project setting for server-side enforcement.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
// This is a public project API key — safe to embed in client-side code.
// It only allows writing events, not reading data.
const POSTHOG_API_KEY = "phc_zjjbX0PnWxERXrMHhkEJWj9A9BhGVLRReICgsfTMmpx";
const POSTHOG_HOST = "https://us.i.posthog.com";
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: The API key is real (phc_zjjbX0...), but the PR description says it's a placeholder (__POSTHOG_API_KEY__). The shouldTrack() check on line 57 for the placeholder string is dead code. Either keep it as a placeholder and replace later, or remove the dead check.

Comment thread packages/cli/src/telemetry/client.ts Outdated
const payload = JSON.stringify({ api_key: POSTHOG_API_KEY, batch });

try {
// Spawn a detached process to send the request so we don't block exit.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: flushSync uses require("node:child_process") inside an ESM module ("type": "module" in package.json). This works at runtime because Node allows it, but it's inconsistent with the rest of the codebase which uses ESM imports. More importantly, execFileSync blocks the process — on slow networks this could add up to 5 seconds to every process.exit(). Consider spawning a detached child with { detached: true, stdio: 'ignore' } and calling unref() so the parent exits immediately.

}));
eventQueue = [];

const controller = new AbortController();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: : null tells PostHog to not record the IP. Good. But the PR description says "Will configure PostHog to discard" — this property already does it. Update the description to match, or note that server-side IP logging should also be disabled in PostHog project settings for belt-and-suspenders.

mkdirSync(CONFIG_DIR, { recursive: true });
writeFileSync(CONFIG_FILE, JSON.stringify(config, null, 2) + "\n", { mode: 0o600 });
cachedConfig = { ...config };
} catch {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: mode: 0o600 is good for the config file (user-only read/write). But mkdirSync on line 74 doesn't set directory permissions — it inherits the umask default (typically 0o755). For consistency, consider mkdirSync(CONFIG_DIR, { recursive: true, mode: 0o700 }) so the directory is also user-only.

telemetryNoticeShown: false,
commandCount: 0,
};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: telemetryEnabled: true as default means new users are opted-in before seeing the notice. The notice is shown on first command run (not on install), so there's a window where telemetry fires before the user knows about it. The first-run notice in showTelemetryNotice() and the tracking in trackCommand() are called in the same block in cli.ts, but trackCommand runs before the notice is persisted. If the process crashes between track and write, the user gets tracked without seeing the notice. Consider: show and persist the notice before tracking.

Comment thread packages/cli/src/cli.ts

const commandArg = process.argv[2];
const isHelpOrVersion = process.argv.includes("--help") || process.argv.includes("--version") || process.argv.includes("-h");
const command = commandArg && commandArg in subCommands ? commandArg : "unknown";
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: trackCommand fires before the command actually runs. If the command fails to import (e.g., missing dependency), we still track it as invoked. This inflates usage numbers. Consider moving the track call to after the command completes, or track it as "command_started" and add a separate "command_completed" event.

}

scaffoldProject(destDir, basename(destDir), templateId, localVideoName);
trackInitTemplate(templateId);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: trackInitTemplate is called in two places (line 384 and line 503) for the two code paths (direct template arg vs interactive prompt). If both paths somehow execute (bug), you'd double-track. Consider moving the track call to a single place after scaffoldProject — or dedup by tracking only once per invocation.

});
await producer.executeRenderJob(job, projectDir, outputPath);
} catch (error: unknown) {
trackRenderError({ fps: options.fps, quality: options.quality, docker: true });
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: The error tracking fires before process.exit(1). Since process.exit triggers the synchronous flushSync in cli.ts, the error event will be flushed via execFileSync — a blocking child process spawn during an error path. If the network is down (which might be why the render failed), this adds a 5-second delay before the user sees the error exit. Consider making the error track fully fire-and-forget.

* In dev: files are .ts (running via tsx). In production: bundled into .js by tsup.
*/
export function isDevMode(): boolean {
try {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude nit: The try/catch here is defensive but import.meta.url is always available in ESM. The only way this throws is if new URL() fails, which shouldn't happen. The catch returning false means "assume production" on failure — that's the right default for telemetry (fail-safe), but consider adding a comment explaining this is intentional.

jrusso1020 and others added 2 commits March 25, 2026 23:13
- Run oxfmt on cli.ts and client.ts
- Replace literal placeholder comparison with phc_ prefix check
  (TS2367: comparing two different string literals has no overlap)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- flushSync: use detached spawn + unref() instead of execFileSync,
  so process.exit() paths don't block up to 5s on slow networks
- showTelemetryNotice: persist notice flag BEFORE printing/tracking,
  so users are never tracked without having seen the disclosure
- Config dir: set mode 0o700 on ~/.hyperframes/ directory (was umask default)
- $ip: null comment: clarify this is belt-and-suspenders with server-side discard
- shouldTrack: update comment — phc_ prefix check is a safety net, not dead code
- env.ts: add comment explaining try/catch fail-safe defaults to production
- init.ts: consistently call trackInitTemplate after scaffoldProject in both paths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jrusso1020
Copy link
Copy Markdown
Collaborator Author

All review feedback from @miguel-heygen addressed in e52a19b:

  1. API key / dead code — Changed to startsWith("phc_") prefix check (safety net for key format validation)
  2. flushSync blocking — Switched from execFileSync to detached spawn + unref(). Parent exits immediately, no more 5s blocking.
  3. $ip: null description — Added inline comment clarifying belt-and-suspenders with server-side "Discard client IP data" setting. PR description updated.
  4. Config dir permissions — Added mode: 0o700 to mkdirSync
  5. Notice before trackingshowTelemetryNotice now persists the flag BEFORE printing, so tracking only happens after disclosure is durably written
  6. trackCommand before run — Intentional for v1: tracking invocation intent is the primary signal. Can add command_completed later if needed.
  7. Double trackInitTemplate — Both paths are mutually exclusive (non-interactive returns early). Fixed ordering: both now consistently call after scaffoldProject.
  8. Error path blocking — Fixed by the detached spawn change (initial code #2)
  9. env.ts try/catch — Added comment explaining fail-safe defaults to production

@jrusso1020 jrusso1020 merged commit d845068 into main Mar 25, 2026
21 checks passed
Copy link
Copy Markdown
Collaborator Author

Merge activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants