Skip to content

Onboarding funnel events cannot be correlated into a single run #1048

Description

@sahrizvi

Problem

Onboarding telemetry events cannot be correlated into a single user's funnel, because the events that make up one first run are emitted from two different threads at times when no common identifier exists.

altimate-code tui runs the TUI on the process main thread and the HTTP server in a worker. Each loads its own instance of the telemetry module. The first-run UI events (onboarding_started, model_picker_shown, provider_selected, the Big Pickle pair, onboarding_completed, the scan-gate pair) and the gateway auth events all fire before any chat session exists, so they stamp an empty session_id and land under the synthetic startup session. The activation events (activation_menu_shown, activation_job_selected, first_job_completed, first_prompt_sent) fire later from a real session and carry its real id.

The result: the top of the funnel and the bottom of the funnel share no key.

There is a fallback identifier — machine_id, a random UUID at ~/.altimate/machine-id — but it is unreliable for exactly this case. On a genuinely new install both threads can observe the file missing and independently create it, so a single first run can emit two different machine_id values. (That race is worth fixing regardless, and is cheap: create the file exclusively and re-read on collision.)

Impact

Countable today: impressions, choices, gateway auth outcomes, scan/skip splits, sample setup invocations — anything that is a simple tally of one event type.

Not answerable today:

  • End-to-end conversion. "How many people who saw the picker ended up connected?" requires linking a model_picker_shown to an onboarding_completed.
  • Per-attempt drop-off. onboarding_abandoned carries a last_stage, but it cannot be tied to the events that led there.
  • Retry behaviour. A user who fails gateway auth and succeeds on the second attempt is indistinguishable from two users.
  • time_to_connect_ms attribution when a user retries.

In other words: the events are individually correct, but the funnel is not reconstructable, which is the main thing the taxonomy exists for.

Proposal

Generate one random id per process launch and stamp it on every onboarding event, from both threads.

  • Generated once in the CLI entry path, before either thread emits.
  • Passed to the worker at startup so both telemetry module instances use the same value.
  • Not persisted, not derived from anything about the machine or user, and not reused across launches. It carries no identifying information — it exists only to group events that belong to the same run.

A launch is the right grain here: onboarding state already resets on every process start by design, so "one launch" and "one onboarding attempt" are the same thing in practice.

Alternatives considered

  • Reuse machine_id. Groups a machine, not an attempt, so retries and repeat launches collapse together. Also affected by the creation race above.
  • Reuse session_id. Does not exist for most of the funnel, which is the original problem.
  • Fix attribution after the fact by joining on timestamp proximity. Fragile, and impossible to do correctly for concurrent instances on the same machine.

Notes

This does not require a new sink, endpoint, or storage. It is one value threaded through the existing event envelope.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestonboardingFirst-run and setup experience

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions