Skip to content

fix(cli): stop dropping CI/agent telemetry, suppress HeyGen CI at workflow level#980

Merged
jrusso1020 merged 1 commit into
mainfrom
05-20-fix_cli_stop_dropping_ci_agent_telemetry_suppress_heygen_ci_at_workflow_level
May 20, 2026
Merged

fix(cli): stop dropping CI/agent telemetry, suppress HeyGen CI at workflow level#980
jrusso1020 merged 1 commit into
mainfrom
05-20-fix_cli_stop_dropping_ci_agent_telemetry_suppress_heygen_ci_at_workflow_level

Conversation

@jrusso1020
Copy link
Copy Markdown
Collaborator

@jrusso1020 jrusso1020 commented May 20, 2026

What

Two changes:

  1. Remove the CI=true early-exit in shouldTrack() in packages/cli/src/telemetry/client.ts. The previous block silently dropped every event from any environment that set CI=true or CI=1 — which is most of where modern usage now lives: coding agents in Codespaces, GitHub Actions, hosted dev sandboxes.
  2. Add HYPERFRAMES_NO_TELEMETRY=1 to every HeyGen workflow that exercises the CLI (ci, regression, preview-regression, catalog-previews, player-perf, windows-render, docs). HeyGen-internal CI stays out of telemetry; external users' CI now flows in.

Why

Investigating two recent traffic spikes on PostHog (Apr 23-30 and May 19) it became clear that most of the headless/agentic traffic we can see is from environments that don't set CI=true (Codespaces, agent sandboxes). The traffic we can't see is from environments that do — i.e. proper CI pipelines. That's the wrong split: we want both populations measured, with the ability to slice between them.

Every event still carries is_ci (plus ci_name, is_docker, is_tty) from system.ts, so CI vs interactive can be separated in PostHog post-hoc instead of being filtered at ingest.

How

  • Deleted the CI=true / CI=1 branch in shouldTrack(). The remaining gates are unchanged: HYPERFRAMES_NO_TELEMETRY=1, DO_NOT_TRACK=1, dev mode, missing PostHog key, user opt-out via the config file.
  • Added env: HYPERFRAMES_NO_TELEMETRY: "1" at the top of each HeyGen workflow that runs the CLI directly or through producer tests. Skipped publish.yml and codeql.yml (don't exercise the CLI).
  • detectCI() and getCIName() in system.ts continue to read CI, GITHUB_ACTIONS, GITLAB_CI, etc. to label events with is_ci and ci_name. Unchanged.
  • autoUpdate.ts and updateCheck.ts still skip on CI=true — that's about update prompts, unrelated to telemetry. Left alone.

Test plan

  • bunx oxlint / bunx oxfmt --check clean on changed files
  • bunx tsc --noEmit passes in packages/core and packages/studio
  • bunx fallow audit clean
  • CI green on this PR
  • After merge, monitor PostHog: cli_command events with is_ci=true should start appearing (previously near-zero). Events from *.github/workflows/* runs of this repo should NOT appear.

Notes

  • Behavior change for any downstream user whose CI was relying on silent telemetry suppression. They can keep the previous behavior by setting HYPERFRAMES_NO_TELEMETRY=1 (or DO_NOT_TRACK=1) themselves.
  • The first-run telemetry notice (showTelemetryNotice in client.ts) still gates on shouldTrack(), so CI environments will technically "show" the notice (to nobody) once per ephemeral machine. Standard OSS telemetry pattern.

…kflow level

The CI=true early-exit in shouldTrack() was hiding most modern usage
(coding agents in Codespaces, CI pipelines, agent sandboxes). Remove it.
Each event still carries is_ci/is_docker/is_tty from system.ts, so CI vs
laptop traffic can be separated in PostHog without being dropped at
ingestion.

HeyGen's own CI is suppressed via HYPERFRAMES_NO_TELEMETRY=1 added to
each workflow that exercises the CLI.
Copy link
Copy Markdown
Collaborator Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@jrusso1020 jrusso1020 merged commit 07bcb4f into main May 20, 2026
45 of 57 checks passed
Copy link
Copy Markdown
Collaborator Author

Merge activity

@jrusso1020 jrusso1020 deleted the 05-20-fix_cli_stop_dropping_ci_agent_telemetry_suppress_heygen_ci_at_workflow_level branch May 20, 2026 05:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants