Telemetry: test runs ship to the production sink, inflating machine counts by ~33%

[anandgupta42]

Description

Our own test runs ship telemetry to the production Application Insights resource, inflating every install and active-machine metric.

In the 2026-07-22 → 2026-08-05 window, 1,020 of 3,135 distinct machine ids emitted provider_id="test" and cli_version="local" — i.e. test processes, which regenerate their machine id on every run. That is roughly a third of the headline machine count.

Root cause

doInit() in packages/opencode/src/altimate/telemetry/index.ts gates only on:

if (process.env.ALTIMATE_TELEMETRY_DISABLED === "true") { ... }

There is no test/CI guard, so the baked-in DEFAULT_CONNECTION_STRING is used by the test suite. bun test sets NODE_ENV=test, so this affects both CI and developer machines running tests locally.

Impact

Any dashboard counting machines, installs, or new users is inflated by ~33%. It also means adoption trends can move purely because CI volume changed.

Constraint on the fix

The gate must key on test runners, not CI. altimate-code-actions wraps this CLI, so every run of that shipped product sets CI/GITHUB_ACTIONS — gating on those would blind a real product surface.

Telemetry suites that point at their own sink via an explicit APPLICATIONINSIGHTS_CONNECTION_STRING must keep working.