feat(onboarding): first-run funnel telemetry - #1049
Draft
sahrizvi wants to merge 51 commits into
Draft
Conversation
- Add an `oauth` `method:"auto"` to the Altimate auth plugin: bind a loopback server on `localhost:7317`, open the browser to the web authorize page, verify `state`, and save the gateway credential to `~/.altimate/altimate.json` - Surface `altimate-backend` first in provider selection (TUI + clack) with the "Recommended · best tool-calling · 10M free tokens" hint Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…uth success
- Add `WelcomePanel` boot box (readiness-aware tips + "What is Altimate Code")
on the home screen; first run is a welcoming panel, not an auto-opened modal
- Restructure the model picker into READY / NEEDS-SETUP; add the curated
`DialogModelWelcome` (top 5 providers + "Search all providers"), Big Pickle
fallback, and `useReady` / `markSetupComplete`
- `/connect` opens the curated picker
- Altimate LLM Gateway sign-in confirms inline ("Authentication successful") and
auto-closes (auto-selecting a model) instead of dropping into the model picker
- Don't force Google: land on the sign-up page and let the user choose
(drop `google_start`); reword the sign-in instruction
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- /auth: sign in to the Altimate LLM Gateway directly via the OAuth loopback, skipping the provider picker (new `DialogAltimateAuth`) - /logout: clear the stored gateway credential (`AltimateApi.clearCredentials`) and disconnect (dispose + bootstrap; the provider loader drops the now-stale auth-store entry on reload) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DialogModelWelcome now tags each row with its providerID (and modelID for Big Pickle) and compares against local.model.current(), rendering a bright-green ✓ plus a "· selected" note on the active provider — so the user can see at a glance that e.g. Altimate LLM Gateway is already selected. Bright green (diffHighlightAdded) is used because plain ANSI green renders dim in some terminals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ebase safety) Shrink our footprint on the upstream opencode `dialog-model.tsx` so future upstream merges are easier to rebase: - Move `useReady`/`markSetupComplete`, `DialogModelWelcome`, and `DialogBigPickleConfirm` into a new altimate-owned `component/altimate-onboarding.tsx` (zero rebase surface — our file) - `dialog-model.tsx` (424 → 160 lines) now holds only pristine `useConnected` plus the `DialogModel` READY/NEEDS-SETUP restructure, fully wrapped in `altimate_change` markers so an upstream conflict is confined and human-resolvable - Repoint imports in `app.tsx`, `home.tsx`, `dialog-provider.tsx`, `welcome-panel.tsx` The onboarding file imports `DialogModel`/`useConnected` back from `dialog-model`, but only inside callbacks/JSX — the circular reference is runtime-only and safe. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address sahrizvi's review on #1001: - Register the pending flow in a state-keyed Map synchronously in authorize() BEFORE opening the browser, and have callback() await that promise — an already-signed-in user's instant redirect is no longer dropped as CSRF, and concurrent /auth flows no longer clobber each other - Bind the callback server to 127.0.0.1 (was all-interfaces / LAN-reachable) - Validate `state` BEFORE honoring the `?error=` branch, so an unauthenticated request can't cancel an in-progress sign-in - HTML-escape reflected values in the callback error page (localhost XSS) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- altimate.ts: reset the callback server on a failed listen (EADDRINUSE no longer wedges the singleton) + surface the reason; validate the callback instance name before persisting; log the failure reason on the auth catch path - dialog-provider: DialogAltimateAuth onMount wrapped in try/catch (no more stuck "Starting sign-in…"); AutoMethod guards post-await updates + clears its auto-close timer on unmount (onCleanup); on connect-with-no-model, open the picker instead of faking a green ✓ - dialog-model: guard the favorite keybind against string (NEEDS-SETUP) rows; treat undefined model cost as not-paid in providerReady() and useConnected() - app.tsx /logout: try/catch with error toast + reset the setupComplete flag - altimate-onboarding: add resetSetupComplete() Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…oopback URL) The loopback now receives a short-lived `token` (login_token) instead of the raw key; callback() exchanges it via POST /auth/social/exchange (AltimateApi. exchangeSocialToken) for the auth_token and saves that. Single code path for both the Google and email/password connect flows. State validation, loopback bind, HTML escaping, and the pending-Map/one-time-server logic are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- NEW-2: callback redirect uses http://127.0.0.1 to match the loopback bind (a plain `localhost` redirect can resolve to ::1 and hit a closed port) - NEW-6: neutral browser copy ("Authorization received") — the loopback replies before the CLI has exchanged/persisted, so it must not claim "Signed in" - NEW-5: guard the /connect OAuth authorize path with try/catch → toast + clear (parity with DialogAltimateAuth; port-collision no longer fails silently) - NEW-4: AutoMethod surfaces a toast on callback failure instead of clearing silently (the precise reason is logged server-side) - NEW-3: swallow a never-awaited pending rejection (void result.catch) to avoid an unhandled rejection when the dialog is dismissed before callback() runs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The welcome/top-5 picker already leads with the Altimate LLM Gateway. Order the full DialogModel list the same way: export `PROVIDER_PRIORITY` and sort the READY section's providers by it (the NEEDS-SETUP section already used it), so the gateway leads whether or not it is connected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve conflicts from upstream's TUI extraction (packages/opencode/src/cli/cmd/tui → packages/tui) and fork-features-as-plugins re-architecture, re-applying the AI-7520 CLI onboarding onto the new structure: - dialog-provider: export/reorder PROVIDER_PRIORITY (Altimate LLM Gateway first), brand the gateway option, add WARNLIST + DialogAltimateAuth, harden the OAuth authorize/AutoMethod UX (try/catch → toast, inline green success + auto-close) - dialog-model: keep the READY / NEEDS-SETUP restructure + Big Pickle, wired through upstream's new DialogVariant flow; gateway leads the full list - use-connected + home tips: fix undefined-cost mislabeling OpenCode as connected - app.tsx: /connect (welcome picker), /auth (gateway), /logout (plugin command + resetSetupComplete); gate the update-available dialog on onboarding readiness - home.tsx: render the WelcomePanel in the home_logo slot - provider-credentials plugin: add altimate.provider.logout - providers.ts: Altimate LLM Gateway first in the login picker - relocate altimate-onboarding + welcome-panel into packages/tui with relative imports Verified: `bun run typecheck` green in packages/tui and packages/opencode; the onboarding/provider/model/app-lifecycle tests pass. Pre-existing sync*.test.tsx failures come from origin/main's own merge (files untouched here). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cherry-pick two chunks from plg-onboarding-prototype-v2, re-applied onto the relocated packages/tui (excludes the prototype stub-server, device-flow, demo flags, and Part 3 sample-env per scope): Part 2 — scan gate: - dialog-scan-gate.tsx: one-time "Scan your environment?" Yes/No gate - register hidden `onboard-connect` command + Part-2-only template (hands off to the existing /discover; discover.txt unchanged) - app.tsx: fire the gate exactly once on a genuine not-ready→ready transition, wired to the existing onboardingReady/useReady signal TUI polish: - prompt/index.tsx: Claude-style input bar (thin rules, `›`, meta row below); first-run submit opens the welcome picker instead of erroring; ⚠ unreliable model chip driven by WARNLIST - autocomplete.tsx + command-palette.tsx: first-run filtering to local/safe commands until a model is ready; hide onboard-connect from the slash menu - sidebar footer: JTBD "here's what you can do" panel + community/docs links, dotted dividers; sidebar left border Verified: bun run typecheck green in packages/tui and packages/opencode; onboarding/provider/model/app-lifecycle tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- welcome-panel: drop the "Tips for getting started" section; the header now shows only "What is Altimate Code" with updated positioning copy. - app: auto-open the curated provider picker (DialogModelWelcome) on a fresh launch with no usable model — the first-run onboarding entry point. Fires exactly once and only after startup settles (`ready()`), so a returning user with valid credentials never sees it; chat input stays visible with submit gated in the prompt until setup completes. Replaces the removed "/connect" tip as the way first-run users are guided to connect. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Previously "No" fired `/onboard-connect skip`, which had the agent post an intent question. Instead, declining now simply closes the gate and drops the user into the empty chat input — the gate already clears itself, so the app-level handler treats `skip` as a no-op and only "Yes" starts the scan flow. - `app.tsx`: onChoose ignores `skip`; only `scan` submits the command - `onboard-connect.txt`: drop the now-unreachable skip branch and the `$ARGUMENTS` dispatch — the template is scan-only - add interaction coverage for `DialogScanGate` (copy renders; `y`→scan, `n`→skip, Enter on default Yes→scan) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Marker Guard flags altimate additions to upstream-shared files that sit outside `altimate_change start … end` blocks (single-line `// altimate_change —` comments do not cover the following line). Wrap each flagged region so the guard passes and the code survives upstream merges: - app.tsx: `connected` + `onboardingReady` readiness signals - dialog-provider.tsx: move `end` past the PROVIDER_PRIORITY close brace - prompt/autocomplete.tsx, prompt/index.tsx: `useReady` gate + WARNLIST chip - use-connected.tsx: undefined-cost "not paid" fix - home/tips.tsx: connected() undefined-cost fix - sidebar/footer.tsx: rewritten View signature + Dotted helper - routes/session/sidebar.tsx: left-edge border attributes Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Second Marker Guard pass — wrap the next uncovered altimate regions: - app.tsx: /connect command entry (curated welcome picker) - dialog-provider.tsx: authorize try/catch guard - prompt/autocomplete.tsx: first-run `if (ready())` server-command gate - sidebar/footer.tsx: sidebar_footer slot registration Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Third Marker Guard pass: - app.tsx: onboarding-aware update-available dialog gate - dialog-provider.tsx: `const local = useLocal()` in AutoMethod (sets the connected model as the post-connect default) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wrap the entire AutoMethod success/auto-close flow (inline green confirm + model auto-select instead of opening the picker) in one outer altimate_change block; the existing inner markers nest. Clears the last Marker Guard finding. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New shipping asset at `packages/opencode/sample-projects/jaffle-shop-duckdb/`.
Ships alongside the wrapper npm package (publish.ts wiring lands in the
next commit); a runtime resolver in `sample-source-resolver.ts` finds it
across dev / test / dist install layouts. A `/starter` slash command (Phase 4)
materializes a copy into a user-owned directory so a fresh user can walk a
real dbt project without connecting a datasource.
**Shape:**
- `dbt_project.yml` / `profiles.yml` — DuckDB profile with project-relative
path, so no host paths bake into the shipped source.
- 4 models (2 staging, 2 marts) + 2 seed CSVs — jaffle-shop layout,
forked from the dbt-tools test fixture and given its own life so a
test-fixture edit can't accidentally change the shipped sample.
- `models/staging/schema.yml` + `models/marts/schema.yml` — per-column
descriptions + `unique` / `not_null` / `relationships` tests, so the
static reviewer surfaces (/discover, /review) have real material to
work with.
- `sample-manifest.json` — version-stamped project metadata used by
the marker-based conflict-detection when the sample is materialized to
the user's filesystem (Phase 4).
- `target/manifest.json` — pre-compiled dbt manifest committed alongside
the source. Ships so that static workflows work with ZERO external
tools installed (no dbt-core, no dbt-duckdb needed for /discover or
/review). Sanitized to strip host paths (replaced with
`{{SAMPLE_ROOT}}` sentinels) and to zero timestamps + invocation_id so
regenerations are deterministic.
- `regenerate.sh` — maintainer script that re-runs `dbt compile` +
sanitizes + stages the manifest. Run after editing sample source; the
freshness test (Phase 5) will fail if source changes without a matching
manifest refresh.
**Not shipped:** `target/graph.gpickle` (Python pickle, no JS consumer),
`target/catalog.json` (warehouse introspection with env-specific
metadata), `target/run_results.json` (run-specific, no static value).
Wire the shipping side of the starter sample so the wrapper npm package
carries it and runtime code can find it across install layouts.
**publish.ts::copyAssets** — copies `packages/opencode/sample-projects/`
into the wrapper package alongside the existing `bin/` and `skills/`
copies. Only the shippable subset of `target/` (manifest.json) is copied
— the rest is excluded so a stale `dbt build` on a materialized user
copy can't contaminate the shipped source.
**sample-source-resolver.ts** — runtime lookup for the shipped sample.
Hunts across four candidate layouts so a single code path works in dev
(bun run src/index.ts), test (bun test), production (compiled bun exe
under `<wrapper>/bin/altimate-code`), and less-common install layouts
(pnpm content-addressable, npx cache) where the exe sits two hops from
the wrapper root. `ALTIMATE_STARTER_SAMPLE_DIR` env override is honored
first — used by tests to point at a fixture and by users pointing at a
hand-curated fork of the sample.
Also exports `loadShippedManifest()` — reads the pre-compiled
`target/manifest.json` and rehydrates the `{{SAMPLE_ROOT}}` sentinels
with the user's materialized target path. This is what the static
review-pipeline consumers (/discover, /review) call to walk the sample
DAG without dbt on PATH.
…eview fixes
Applied four gaps from the Phase 3 codex adversarial review:
1. **Resolver honors symlinked `process.execPath`.** npm global installs
symlink the binary from `/usr/local/bin/altimate-code` to the actual
location under `lib/node_modules/`, Homebrew uses `libexec`, pnpm uses
`.bin` shims. The previous resolver walked from the shim's dirname and
would miss `../sample-projects/`. Now `fs.realpathSync(process.execPath)`
first, then hunt from the real location.
2. **`loadShippedManifest` JSON-safe rehydration.** The prior text-level
substitution of `{{SAMPLE_ROOT}}` sentinels corrupted JSON if the
materialized target path contained JSON-significant characters. A path
like `/tmp/a"b` broke the JSON with an unescaped double-quote; a
Windows path like `C:\Users\name\sample` produced invalid `\U` escape
sequences. Now: parse the manifest first, walk the tree with a new
exported `rehydrateSentinels()`, substitute ONLY inside string leaf
values, preserve object keys / numbers / booleans / nulls. Same fix
applied in `regenerate.sh` so a maintainer's home directory can't
accidentally sentinelize legitimate content (a model description or
compiled SQL literal that happens to contain the maintainer's absolute
path).
3. **`generated_at` pinned to a plausible past date instead of the epoch.**
Zero-epoch (`1970-01-01`) is a trap for downstream freshness-check /
staleness-detection tooling that may treat it as pathological. Fixed
to `2026-07-24T00:00:00Z` — bumped only when the maintainer wants to
signal a manifest-shape refresh, not on every regenerate.
4. **Longest-sentinel-first replace order.** In `rehydrateSentinels` the
`{{SAMPLE_ROOT_PARENT}}` sentinel is substituted before
`{{SAMPLE_ROOT}}` so the shorter one can't match inside the longer
one's expansion window.
Windows `cp`/`mkdir` portability in `publish.ts::copyAssets` (codex point
#3) was DEFERRED: matches the existing shell-based pattern used elsewhere
in the file, and release CI runs on Ubuntu only. Track as a follow-up
when the release moves off Ubuntu.
…r, materialize, tools, KV) Phase 4a — the non-UI logic behind the activation prompt and /starter slash command. Adds five modules under `packages/opencode/src/altimate/onboarding/`: - **kv-keys.ts** — four KV keys (`onboarding.activation.dismissed_at`, `.completed_choice`, `onboarding.sample_project.path`, `.version`) plus the `ActivationChoice` enum. Keys deliberately split so a support engineer can inspect each state independently and so KV / on-disk marker divergence stays reasonable to debug. - **tool-detection.ts** — `detectDbtRuntime()` probes `dbt --version` once per process and parses its plugin list for the "duckdb" adapter line. Cached for the process lifetime; force-refresh available for tests. Post-materialize UX filters "run" options (dbt build, live query) when `hasDbtDuckdb` is false, so shipped commands can't silently fail on users without the Python side installed. - **marker.ts** — `.altimate-sample.json` marker semantics. `classifyTarget(dir, version)` returns one of `empty` / `our-sample-current` / `our-sample-different-version` / `unknown-dir`. `findSafeTarget()` walks the `<name>`, `<name>-2`, `<name>-3` sequence until a non-unknown-dir slot appears — never overwrites an unknown directory. Marker is the AUTHORITATIVE source of truth for filesystem safety; the KV path is merely a convenience index (per codex feedback: "marker wins on divergence"). - **materialize.ts** — copies the shipped sample source into the user's chosen target. Whitelist-driven (no wholesale recursive copy) so future contributor scratch files don't accidentally leak into user installs. Enforces the marker-based conflict policy from marker.ts. `rejectUnsafeHome()` refuses to materialize into `/root` (sudo mistake), `/tmp/*` (ephemeral runner), or `/` (misconfigured container) with an actionable error the caller surfaces verbatim to the user. - **detection.ts** — `detectUsableSetup(cwd)` returns `"usable" | "detected-not-usable" | "nothing"` for ordering the three options in the activation dialog. Wraps the existing `detectDbtProject()` primitive with a targeted `profiles.yml` regex scan that respects dbt's precedence (project-local → `$DBT_PROFILES_DIR` → `~/.dbt/`). Doesn't validate credentials — that would require a warehouse handshake we're not spending on activation. Phase 4b (next commit) wires the TUI dialog, the slash commands, and the app.tsx / onboard-connect.txt integration points that call into this logic.
Four adjustments from the Phase 4a codex adversarial pass:
1. **detection.ts — broadened profile-key regex.** The old regex
`^<name>:\s*$` only matched an unquoted, non-commented top-level key
with nothing after the colon — false-negatived quoted keys
(`"jaffle-shop":`), inline mappings (`jaffle_shop: {target: dev}`),
and trailing comments (`jaffle_shop: # local`). New pattern accepts
optional matching single/double quotes and any trailing content.
Known Jinja-wrapped / anchor-referenced false-negatives are
documented as accepted for v1 — verdict impact is only option
ordering (dialog still shows every choice), so erring on the
sample-side is the safer bias.
2. **marker.ts — attemptLimit 10 → 100 + randomized fallback.** A user
with `altimate-sample-dbt-{1..10}` (retries, benchmark runs, support
copies) would previously get a hard `Error: No safe target found`
at activation time. Now the loop tries 100 numeric slots; if all
are held by unrelated content, one final randomized `-<6hex>` slot
is attempted. Only after both fall through does it throw — the
collision odds on the randomized slot alone are ~1-in-16.7M, so the
only realistic path to failure is genuine environmental hostility.
3. **marker.ts + materialize.ts — parent-writable pre-check.** New
`checkParentWritable()` in marker.ts is called in materialize.ts
BEFORE candidate hunting. Turns a raw `EACCES` from `mkdirSync`
deep in the copy step into a clear "Target parent directory X is
not writable: <reason>" error the caller can surface. Handles
read-only enterprise homes, NFS glitches, container mounts.
4. **materialize.ts — README.md added to MATERIALIZE_ENTRIES.** New
`sample-projects/jaffle-shop-duckdb/README.md` documents what's
inside the sample and what to try (works-with-zero-tools vs
needs-dbt-duckdb). Codex flagged that shipping a sample directory
with no accompanying "what next" reading material was a
context-loss risk. README is also copied into the wrapper package
via publish.ts.
5. **tool-detection.ts — call-site guidance for cache staleness.**
Docstring on `detectDbtRuntime` now explicitly documents WHEN
callers must pass `{ force: true }` (after materialization, before
any run-workflow invocation). This is Phase 4b's problem to obey,
but calling it out here makes the intent visible in the module.
Refactor: `MaterializeResult.suffixIndex` → `suffix` (renamed for the
new `number | string` shape now that randomized fallback is a possible
value).
Phase 4b — the customer-facing surfaces that consume Phase 4a's core logic. Adds three touch-points: **1. `DialogActivation` (packages/tui/src/component/dialog-activation.tsx)** The 3-option picker that fires when the scan-gate "No" path resolves and also from the `/activation` slash command. Custom `<box>` layout matching the visual style of `DialogScanGate` and `DialogModelWelcome` (part 1 + part 2a of onboarding). Keyboard: 1/2/3 shortcuts, up/down + Enter, Escape → dismissed. On any selection: persists both `onboarding.activation.completed_choice` and `onboarding.activation.dismissed_at` in KV so the dialog does not auto-fire on future launches, then calls the injected `onChoose` callback. Options ordered by an async `detectUsableSetup(cwd)` probe: a `dbt_project.yml` + resolvable `profiles.yml` verdict leads with "Connect data"; otherwise "Open sample project" is first. `packages/tui/src/altimate/onboarding/` — TUI-side `kv-keys.ts` + `detection.ts` colocated here because the TUI package cannot import from `packages/opencode` (workspace dep is `@opencode-ai/core` only). Detection is a small self-contained fs walk mirroring `detectDbtProject()` from opencode-side project-scan. **2. `app.tsx` scan-gate "No" → open DialogActivation** The scan-gate at app.tsx:585 previously hardcoded `if (arg === "skip") return` — dropped the user into an empty chat with no continuation. Now the "skip" branch calls `dialog.replace(() => <DialogActivation ... />)` with a shared `dispatchActivationChoice` handler that routes each of the four possible choices to the right follow-up: `connect_data` runs `/onboard-connect scan`, `sample_project` runs `/starter`, `describe_use_case` prefills the prompt buffer with a starter hint (does NOT auto-submit — user finishes the sentence), `dismissed` is a no-op with an already-persisted KV timestamp. **3. Slash commands: `/starter` + `/activation`** - `starter.txt` — LLM template that invokes the (Phase 4a) materialize logic, then reports the result with one of three branch outputs: reused, fresh, or fresh-with-suffix. Post-materialize UX suggestions are strictly static-workflow only (dbt-duckdb install caveat goes at the end for users who want live queries). - `activation.txt` — one-line placeholder. The real work happens in `appCommands` where a slash-name registration intercepts `/activation` BEFORE it reaches the LLM and directly reopens the dialog. Escape hatch for users who dismissed the dialog too early — addresses the design consult's #1 concern ("if 'skip all' hides every useful recovery affordance, users are stranded"). Both commands are registered in `packages/opencode/src/command/index.ts` under `Default.STARTER` and `Default.ACTIVATION`, joining the existing `/onboard-connect` follow-up family. **4. `onboard-connect.txt` branch 4 → mentions `/starter`** The "genuinely nothing yet" branch of the scan template previously advertised "scaffold a project" as an aspirational offer with nothing behind it. Now it explicitly points at `/starter` alongside "cd into your project and run /discover" and "paste SQL / describe your use case" — the same three routes the activation dialog offers, so both entry points (scan-gate "No" AND scan-found-nothing) converge on the same next-action set.
Four fixes from the Phase 4b codex adversarial pass: 1. **Rename TUI detection → `tui-detection.ts`** to make the ownership split explicit: this module is DISPLAY ORDER ONLY, not an authoritative "usable setup" verdict. Opencode-side consumers (agents, tools, /discover flows) must NOT import from here — they own their own detection surface. Rewrote the file header to spell this out so a future contributor doesn't wire this into a slash command by accident. 2. **Gate keyboard input on detection completion** in `DialogActivation`. `selected` starts at -1 while `detectUsableSetup()` is in flight; only Escape works during that window. This closes a race where a user with a valid dbt setup could Enter on the sample-first fallback ordering (because verdict was undefined) and end up on /starter when "Connect data" was the intended default. Detection is ~100ms so the "Checking local project…" label is one-frame territory. Also documented the `process.cwd()` assumption + wrapper-installer edge case for the Phase 5 e2e test to cover. 3. **Prefill wording** for the "Describe your own use case" path changed from the sentence fragment `"I'd like to "` to `"Describe what you're trying to do: "`. If a user accidentally hits Enter on the preamble, the LLM still receives a coherent question rather than a broken fragment. Comment now also flags the `parts: []` clear as a low-risk-but-non-zero draft-loss for future recovery-flow work. 4. **`/activation` template becomes a real non-TUI fallback.** Previous version claimed the dialog was already open even when invoked in headless / ACP / `--print` mode where the TUI intercept doesn't fire — the model would then advertise a picker that didn't exist. Now the template presents the three choices as a plain-text list and asks the user to pick by name. The TUI palette intercept still short-circuits this template entirely for interactive sessions, so the fallback only renders where it's needed.
…ple-source-resolver Four unit test files landing 47 tests total, plus an off-by-one fix in the resolver that the first test-run flushed out. **marker.test.ts** — 22 tests covering `readMarker`/`writeMarker` round-trip, the four `classifyTarget` decision-table branches (empty / our-current / our-different-version / unknown-dir), and `findSafeTarget`'s numeric-suffix loop + randomized-hex fallback when all numbered slots are held by unrelated content. Plus the codex-flagged `checkParentWritable` pre-check with writable and nonexistent parents. **materialize.test.ts** — 8 tests. Fresh copy verifies all whitelisted files land + profiles.yml is intact + marker is correct. Second-call reuse-detection asserts no re-copy + no marker rewrite (materialization timestamps preserved). Preferred-collision → `-2` suffix without touching user's original file. Version-bump paths cover both the "prompt before upgrading" (allowInPlaceUpgrade=false) and "upgrade-in-place" (allowInPlaceUpgrade=true) branches. Unsafe HOME scenarios exercised via `rejectUnsafeHome` (undefined, `/`, `/tmp/*`, `/root` under non-root uid). **sample-source-resolver.test.ts** — 12 tests. Env override path, default dev-source-tree path, and the JSON-safe `rehydrateSentinels` tree walk with adversarial inputs codex flagged in Phase 3 review: strings with double quotes (`/tmp/a"b`), Windows backslash paths (`C:\Users\...`), object keys that happen to match the sentinel literally (must be preserved), and the parent-then-root replace order (so the shorter sentinel can't shadow the longer one). Plus end-to-end `loadShippedManifest` against the real committed manifest — asserts no dangling sentinels after substitution AND the target path appears where expected. **tool-detection.test.ts** — 5 tests pinning the `dbt --version` parser regex against representative dbt 1.x outputs. Covers the codex- flagged "dbt present, no dbt-duckdb" scenario, the "'duckdb' as a substring in an upgrade hint (not a plugin line)" false-positive guard, and the strict-formatting requirement so bare-word "duckdb" without a colon never counts. **Resolver off-by-one fix**: the `dev-source-tree` candidate went 4 hops up from `packages/opencode/src/altimate/onboarding/` — which lands at `packages/` — but sample-projects lives at `packages/opencode/sample-projects/`. Fixed to 3 hops. The mistake was invisible to Phase 3's smoke path (it happened to fall through to a different candidate); Phase 5's tests forced a real assertion and exposed it. 47 pass, 0 fail. Full altimate suite: 3773 pass, 640 skip, 0 fail.
`starter-sample.tape` — VHS script that drives the demo. Renders `docs/media/starter-sample.gif` when run with `vhs docs/media/starter-sample.tape`. The rendered GIF is git-ignored (regenerable, ~700KB binary blob not worth committing to a public repo); reviewers render locally or view attachments on the PR. `starter-sample-demo.sh` — helper shell script that the tape shells out to. VHS's `Type` command can't reliably escape long nested-quote shell one-liners (bun -e with an inline JS string that imports materializeSample), so the tape delegates each demo step to a named action on this script. Also lets a reader `bash` this file directly for a non-recorded reproduction. **What the recorded demo shows** (in order): 1. `materialize` — fresh copy to `$HOME/altimate-sample-demo/` 2. `ls` — expected files (README, dbt_project.yml, profiles.yml, .altimate-sample.json marker, models/, seeds/, target/) 3. `find` — full tree layout 4. `cat .altimate-sample.json` — the conflict-detection marker 5. `wc -l target/manifest.json` — pre-compiled manifest present 6. README head — what-to-try guidance 7. Second `materialize` — reports `reused: true`, no re-copy **NOT in this recording** — the interactive TUI activation dialog itself. That path gates on `useReady()` which needs a live provider or the setupComplete signal from finishing OAuth; scripting it in VHS needs an auth-mock harness we don't have yet. Tracked as a follow-up. This recording proves the LEAF flow (sample materialization) works end-to-end against the shipped asset resolver + marker + copy logic committed in Phases 3 & 4a. Also gitignores `docs/media/*.gif` so future renders don't accidentally get staged.
…nd materializeSample
Fills the vaporware gap in the `/starter` slash-command flow: the
template at `packages/opencode/src/command/template/starter.txt`
already asks the LLM to *"Call the `starter_materialize` tool exactly
once, with no arguments"* — but I had never actually registered such a
tool. Users typing `/starter` would have hit an "unknown tool" error
and the whole materialization flow would silently fail.
**`packages/opencode/src/altimate/tools/starter-materialize.ts`** —
new `Tool.define("starter_materialize", ...)` modeled directly on
`feedback-submit.ts` (the canonical altimate-side pattern for a
slash-command tool that DOES something and returns structured metadata
for the template to branch on):
- Zod schema with 3 OPTIONAL parameters (preferred_target_name,
target_parent, allow_in_place_upgrade) — none required, so the
LLM's "no arguments" invocation works.
- Reads `sample-manifest.json`'s `version` field via the shipped
sample-source-resolver, so bumping the sample version auto-flows to
the marker without a code change here.
- Delegates to `materializeSample()` from Phase 4a; wraps its return
into the `{title, metadata: {targetPath, reused, suffix, note},
output}` shape the template's three branches consume.
- On failure (unsafe HOME, unwritable parent, missing source),
returns `{title, metadata: {error}, output: <actionable message>}`
— output text is passed through verbatim by the template.
**Registered** in `packages/opencode/src/tool/registry.ts` inside the
existing altimate_change block, right next to `FeedbackSubmitTool`.
Import + array entry.
**Test** at `packages/opencode/test/altimate/tools/starter-materialize.test.ts`
(5 tests, all pass). Covers each of the three success branches
(reused / fresh / suffixed), the version-mismatch prompt-hint branch,
and the failure-message passthrough. Uses the existing `initTool()`
fixture in `test/altimate/tool-fixture.ts` to unwrap the Effect-based
Tool.define into a plain `execute(args, ctx)` for assertion.
All 3778 altimate suite tests pass, typecheck clean.
This is the fix for the biggest gap I flagged in the last doubt list:
the /starter flow now actually works end-to-end. Precedent confirmed
via Sarav's `/onboard-connect` implementation (same pattern:
LLM-driven slash template + registered tool with structured return)
and via the `feedback-submit.ts` shape — both established before this
work.
The onboarding template we route to (packages/opencode/src/command/template/ onboard-connect.txt) refers to the sample bootstrap tool as sample_setup; align the tool's registered name so the LLM's tool call resolves. No behavior change — same materialization logic, same schema, same tests.
…dal dialog
The activation-menu design lands as agent-emitted text at the end of every
/onboard-connect branch, matching the ticket mockup: JTBD-worded options
(see downstream / review a SQL PR / try the sample project / describe
your own), rendered inline in the chat rather than a modal overlay.
Rationale: the modal DialogActivation approach we prototyped had two
practical blockers — auth-arrival timing meant the false→true transition
never fired for users with pre-loaded creds, and the fixed option-set
couldn't personalize per scan verdict. Emitting the menu from the
onboard-connect template dodges both, keeps warehouse-connected
personalization ("you've got 12 dbt models and a Snowflake connection…"),
and reuses existing skill routing (dbt-analyze, sql-review, cost-report,
sample_setup, dbt build, sql_execute) instead of standing up parallel
dispatch machinery.
Also handles the 'Build & query it' branch when dbt-duckdb isn't
installed: bash-probes for the adapter first and, if missing, surfaces
an actionable install instruction with two paths (paste an existing
dbt binary path, or run 'pip install dbt-duckdb'). Once available,
runs dbt build, then explicitly wires dbt-profiles → warehouse_add →
sql_execute so the DuckDB connection is registered before any query.
Scan-gate 'No' now dispatches /onboard-connect skip again (previously
close-only) so the template's skip branch runs and the menu emerges
naturally in the chat.
Removes: /starter and /activation slash commands + their templates,
the DialogActivation TUI component, KV keys + detection modules that
supported it, and the /starter VHS demo tape (no longer meaningful).
Also drops the stale KV_SAMPLE_PROJECT_PATH reference in marker.ts'
docstring.
…mplate shell/path safety, manifest asset-set guard Bot review (cubic-dev-ai + a class-scoped codex re-sweep) surfaced three real bugs on this branch and one test-coverage gap: - cubic P1-1: `classifyTarget` was returning `our-sample-current` on any marker whose VERSION matched, ignoring `sampleName`. A future second bundled sample writing a marker with the same version would be silently reused/upgraded through as this sample. Add `expectedSampleName` gate in `classifyTarget`; threading through `findSafeTarget` and the reverify-before-rmSync call. Fallback slot and materialize reverify now both consult sample identity. Tests added for wrong-sampleName → unknown-dir + symlink and unreadable-dir branches (which were previously untested at the marker.test.ts level even though materialize.test.ts covered symlink end-to-end). - cubic P1-2: template's "user pastes a dbt path" branch interpolated the pasted string directly into a bash pipeline. A paste like `; rm -rf ~` would execute. Rewrite the branch to (a) refuse paths containing shell metacharacters, (b) refuse non-executable files, (c) verify via a single-quoted `'<path>' --version` invocation, (d) require single-quoting on every subsequent use of the path. - cubic P1-3: DuckDB profile advertises `target/jaffle.duckdb` — a RELATIVE path. If passed as-is to `warehouse_add`, `sql_execute` resolves it against its own cwd and connects to (or creates) an empty database file wherever the CLI is running — not the one dbt just built. Template now instructs the model to join the sample path with the profile's `path:` and pass the ABSOLUTE result. - cubic P2 #4: `copySampleTree` silently skipped ANY missing entry ("`.gitignore` is optional; skip quietly"), even required ones. A broken package with missing `models/` would materialize an empty dir + write a marker, then reuse-forever on subsequent runs. Split into required vs optional entries; missing required entry throws with a reinstall pointer instead of writing a marker. - codex NEW-10: `verify-freshness.test.ts` iterated only over checksummed manifest nodes. A maintainer adding a new .sql model without re-running `regenerate.sh` would pass the per-node hash check (their new model simply had no manifest node to compare against). Added a set-membership test: walk `models/*.sql` and `seeds/*.csv` on disk and assert each has a manifest node.
…lowup) Kilo-code-bot flagged that the previous fix refused only UNMATCHED single- quotes in the pasted dbt path. A path with a matched pair like `a'b'c` survives the denylist, but wrapping it as `'a'b'c'` for the shell makes bash see three tokens (`a`, unquoted `b`, `c`) — the middle segment escapes single-quoting silently and we'd run against a different binary than the user pasted. Metacharacter denylist already blocks command injection; this is the remaining correctness gap. Refuse ANY single-quote instead of trying to count matched pairs. Single-quoting the resulting path is then provably path-preserving and the model doesn't have to reason about quote parity.
Adds the onboarding funnel event taxonomy and wires the first six events, all emitted from the TUI worker thread. - `Telemetry.Event`: 19 onboarding variants under the spec's own event and property names. Doc comments record the two things that would otherwise mislead whoever writes the queries — `gateway_device_code_issued` describes a browser loopback OAuth (this flow has no device code), and the activation events are inferred from proxies rather than observed, so their counts are lower bounds. - `altimate/telemetry/onboarding.ts`: typed emitter that awaits the idempotent `Telemetry.init()` before tracking. Onboarding events can fire before any prompt runs, and in the TUI worker `init()` only happens via `session/prompt.ts` — a user who quits during gateway auth never reaches a prompt, so those events would sit in the pre-init buffer and never ship. Also owns the monotonic funnel-stage state behind `onboarding_abandoned`. - `Telemetry.shutdown()`: serialize concurrent calls. Three paths can now overlap, and each would enter `flush()`, which splices a shared buffer — one caller could post a half-empty batch while the other dropped events. - Close two flush holes. `cmd/tui.ts` ends its handler with `process.exit(0)`, skipping the outer `finally` in `index.ts` that normally flushes; and the TUI worker's `rpc.shutdown()` drained traces but never its own telemetry buffer. The two threads hold separate `Telemetry` module instances, so both need one. Bounded at 2s each: `flush()` can block for `REQUEST_TIMEOUT_MS` (10s), which exceeds the 5s the shutdown RPC is given and would be a visible hang on exit. Events wired: the four gateway auth events, `environment_scan_completed`, and `sample_setup_completed`. `gateway_auth_failed.reason` is classified by tagging the error at each rejection site rather than by matching message text. The `error` query param is attacker-influenced text we must neither parse nor forward, and message matching would drift the moment a string is reworded. Note that an unknown `state` never rejects a pending flow — the handler returns 400 without touching the map — so a CSRF mismatch correctly surfaces as `timeout`, not `denied`. `has_warehouse` and `connections_found` derive from `totalConnections`, not `connections.alreadyConfigured`: warehouses discovered from dbt profiles, docker, or env vars are counted separately, and a user whose only warehouse was auto-discovered would otherwise be recorded as having none. `sample_setup_completed` counts models and seeds by walking the shipped sample tree. A constant would drift the first time someone adds a model, and dbt's `target/manifest.json` is ~17k lines to extract two integers on a path the user is waiting on. No instance or tenant names, filesystem paths, raw error text, or authorize URLs (which carry the CSRF `state`) reach telemetry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
Follow-up to the first telemetry batch, from a review pass over it.
Shutdown correctness:
- `init()` now waits out an in-flight `shutdown()`. The previous commit only
serialized shutdown callers, which was half the problem: `session/prompt.ts`
init()s at the start of every session loop and shuts down at the end, so a new
session could begin while the previous shutdown was still awaiting `flush()`.
`init()` returned the stale `initPromise`, the new session's events landed in
`buffer`, and the in-flight `doShutdown()` then cleared that buffer — every
event tracked in the gap was lost.
- The 2s bounds on the exit paths did not bound anything. `withTimeout()` and
`Promise.race()` do not cancel the promise they lose to, so a timed-out flush
kept running and reset module state after the caller had already resumed, and
the worker's raw race additionally left a live timer behind. The deadline is
now applied inside — `shutdown({timeoutMs})` → `flush(timeoutMs)` → the
existing `AbortController` — so the request is genuinely aborted and no
external timer exists.
Gateway funnel gaps:
- `gateway_auth_failed` was never emitted when `startCallbackServer()` throws,
which is what happens when port 7317 is already in use. That is a real and
reasonably common failure, and it occurs before any callback object exists, so
it was invisible to the funnel.
- Repeated `callback()` invocations re-ran the whole body and could emit
completion/failure more than once per attempt, re-reporting a connect time
measured from the original attempt. One outcome per attempt now.
- `time_to_connect_ms` started after the browser open, excluding the callback
server startup and the browser launch — both part of the wait the user
actually experiences. It now runs from the top of `authorize()`.
Coverage:
- `sample_setup_completed(success: false)` now also fires when the shipped
sample assets cannot be resolved (a CLI installed without its wrapper package
assets). Previously only materialization failures were counted, so failure
numbers silently excluded the broken-install case.
Also brackets the whole moved `shutdown()` body in `altimate_change` markers;
the previous commit left most of it outside, which would conflict awkwardly on a
future upstream merge.
Known and unchanged: `environment_scan_completed` and `sample_setup_completed`
fire on every invocation of their tools, including `/discover` and
model-initiated calls, so they are not onboarding-scoped. Scoping them needs the
session-level onboarding state that lands with the activation events.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
Wires the four session-scoped funnel events, completing the worker-side half of the taxonomy. The three activation events are DERIVED, and the code says so. The activation menu is not UI — it is text the model writes from the `/onboard-connect` template — and the user picks a job by replying in free text, so neither moment is observable. What is deterministic is which command started the session and which tool ran next, so that is what they infer from: - `activation_menu_shown` — for the `skip` branch, at command dispatch, where the variant is known because there is no scan to wait for. For the `scan` branch, when `project_scan` returns, since the menu follows the scan and the variant depends on what the scan found. Emitting both at dispatch would have over-counted sessions that error out before reaching the menu. - `activation_job_selected` / `first_job_completed` — from the first job-shaped tool call in an onboarding session: `sample_setup`, or the `skill` tool carrying `dbt-analyze`, `sql-review`, or `cost-report`. A job that starts and then fails still counts as selected. `something_else` — the "just let me chat" branch — has no tool signature and therefore cannot be detected at all. Those counts are lower bounds, noted in both the event type and the plugin. `first_prompt_sent` excludes slash commands, which matters because the scan gate submits a hidden `/onboard-connect` as an ordinary user message: without the exclusion it would be recorded as the user's first prompt in every single fresh onboarding. Implemented as a plugin (`command.execute.before`, `tool.execute.after`) rather than as edits scattered through `session/prompt.ts` — the hooks already carry everything needed, and this keeps the inference logic in one fork-owned file. Only `first_prompt_sent` lives in the session loop, next to the existing `session_start`/`task_classified` telemetry it belongs with. Session-scoped state is capped and evicts in insertion order: a `serve` process is long-lived and sees unboundedly many sessions. The scan variant sums all four connection buckets from `project_scan` metadata (`existing`, `new_dbt`, `new_docker`, `new_env`) — there is no pre-summed total. A user whose only warehouse was discovered from a dbt profile, docker compose, or env vars still has a warehouse, and reporting `no_data` would send them down the sample-project branch of the menu. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
…n, claims Follow-up to the activation batch, from a review pass over it. `first_prompt_sent` was wrong in three ways at once: - It fired on every user turn, not once per session. The enclosing `step === 1` block reads like "first message of the session" but is not: `step` is declared inside `loop()`, and `loop()` runs once per user turn, so the block executes every turn. Now claimed once per session explicitly. - It was not scoped to onboarding at all, so an onboarding-taxonomy event fired for every session in the product — TUI, `run`, GitHub, API callers. - Combined with the above, a resumed session could emit it for a historical message. `first_job_completed` no longer fires on skill load. The `skill` tool loads an instruction bundle and returns; the agent then does the actual work with other tools. Treating that as completion reported "downstream impact analysis complete" the moment the instructions were read. Only `sample_setup`, which really does the work and reports `metadata.success`, is evidence of completion. Skill-driven jobs are now absent from this event rather than wrong in it — `activation_job_selected` still covers them, since selection IS observable. Session attribution: `emit()` now takes the caller's session id instead of always reading `Telemetry.getContext()`. That context is process-global and set by the session loop, so a plugin hook firing for session A while the context pointed at session B misattributed the event — and stamped an empty session for the gateway events, which run before any session exists. Session state is now one record per session instead of five parallel sets. Separate sets could evict a session from some and not others, leaving the worst possible state: still considered "onboarding" but with its once-per-session claims forgotten, so it re-emitted `first` events. One map evicts a session's whole state atomically. Cap raised 64 → 256, since a long-lived `serve` process can hold many concurrent sessions. Note for whoever reads the existing telemetry: the `step === 1` misreading above is not limited to this change. `session_start` and `task_classified` are emitted from the same block and therefore also fire once per user turn rather than once per session, which inflates session counts by the number of turns. That is pre-existing and untouched here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
Wires the eight TUI-side funnel events, completing the taxonomy. `packages/tui` cannot import `packages/opencode`, where the Telemetry module lives, so the host injects a callback through `TuiInput` and a context carries it to the components — the same seam `context/exit.tsx` already uses for the host's exit function. No HTTP route, no SDK surface, no worker round-trip: the TUI renders on the main thread, so this reaches the process's own Telemetry instance, already initialized by the CLI middleware. The provider is mounted above `DialogProvider`, and that placement is load-bearing: `ui/dialog.tsx` renders dialog contents as a *sibling* of its children, so a provider placed around `<App>` is invisible to every dialog. `dialog-scan-gate.tsx` already documents the same trap for the prompt ref. For the scan gate specifically the event is emitted from the `onChoose` closure in `app.tsx`, reusing the prop bridge that exists for exactly this reason. Events and their anchors: - `onboarding_started` — only on the branch that actually opens the first-run gate, so returning users never enter the funnel. - `model_picker_shown` — carries `trigger`. The picker also opens from `/connect`, from declining Big Pickle, and from the prompt gate; without the discriminator every impression would read as a fresh first run. - `provider_selected` — through a single activation choke point so keyboard and mouse cannot diverge, plus the `/` shortcut, which is the same intent as the "Search all providers…" row. Fires on selection, before auth resolves: a cancelled sign-in still counts as a provider having been chosen. - `big_pickle_confirm_shown` / `big_pickle_choice`, `scan_gate_shown` / `scan_gate_choice` — both choice handlers are now guarded against double-submit. Keyboard and mouse both call them directly and nothing stopped two firing before unmount, which would double-count the choice and, for the scan gate, submit `/onboard-connect` twice. - `onboarding_completed` — the readiness false→true transition, which is exactly the spec's "a model is ready and chat is live". Selecting the gateway provider also marks the auth funnel stage on this thread, because the browser flow runs in the worker and cannot reach the main thread's abandonment state. The `name` → `type` remap in `cli/cmd/tui.ts` is the one untyped point in the chain: the package boundary forces `packages/tui` to declare its own mirror of the event union. Both sides are independently type-checked; a test will pin the two lists together. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
… safety Follow-up to the TUI batch, from a review pass over it. Abandonment was going to be mostly noise. Nearly every piece of UI that emits a funnel event is also reachable outside onboarding: `/connect` opens the same welcome picker, `/model` opens the same Big Pickle interstitial. A returning user doing either advanced the funnel stage to `model_picker`, never emitted `onboarding_completed`, and was therefore reported as ABANDONED on exit. The headline drop-off metric would have been dominated by established users opening `/connect`. Stage tracking is now gated on the funnel actually having started — `onboarding_started`, which is emitted only from the branch that opens the first-run gate. Everything downstream is a no-op outside a real first run, which also fixes the same class of problem for the host marking `gateway_auth` on any gateway selection. Other fixes: - `prompt_gate` was never emitted. The prompt gate opened the picker without passing a trigger, so every "typed a prompt before connecting" impression was misattributed to `/connect`. - `provider_selected` was not fire-once. Keyboard return and mouse-up both reach the activation path with nothing stopping two firings before unmount — that double-counted the choice *and* started the provider flow twice. The `/` shortcut now routes through the same guarded path instead of emitting its own event. - The telemetry context no longer throws when absent. It was built on `createSimpleContext`, which throws for consumers outside its provider; these dialogs are rendered directly by tests and can be reused by other hosts, and analytics is not a reason to crash a UI. Missing provider now means no tracking. The host callback is also isolated, so a throwing tracker cannot propagate out of a mount handler or a keypress and take the dialog transition with it. Known and unchanged: under `OPENCODE_FAST_BOOT`, sync reports ready before credentials are populated, so a returning user can transiently look un-onboarded and see the first-run gate. That is a pre-existing product behaviour — the gate itself already had it — and these events faithfully report what the UI did. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
A whole-feature review found the events individually correct but the funnel not reconstructable. These are the fixes. Correlation (see #1048): - Every event now carries `launch_id`, a random value generated once per process and shared across threads through the environment. The funnel spans the TUI main thread and the server worker, and most of it runs before any chat session exists — so the first half of the funnel stamped an empty session and the second half a real one, leaving no key to join a single run on. It is not persisted, not derived from the machine or the user, and not reused across launches. Placed in the event envelope alongside `machine_id`, so it applies to all telemetry rather than only the onboarding events. - `~/.altimate/machine-id` is now created exclusively. Both threads initialise their own copy of the telemetry module, and on a genuinely new install both can find the file missing at the same moment; with a plain write the loser overwrote the winner while each kept its own value in memory, so one first run reported two machine ids. The fallback identity broke on exactly the run it mattered most for. `wx` makes one writer fail and re-read. Abandonment: - `last_stage` no longer advertises values it can never report. `scan_gate` and `activation` both occur after `onboarding_completed`, which sets `completed` and suppresses abandonment entirely — abandonment is by definition quitting before connecting. They are removed rather than documented as live. - Added `provider_setup`, advanced by any provider choice. A user who picked Anthropic and quit during key entry was previously reported as abandoning at `model_picker`, as though they had never chosen anything. Shutdown: - `shutdown({timeoutMs})` no longer ignores the budget when a shutdown is already in flight. It returned the existing promise, which may have been running on the default 10s, so the exit path waited past its 2s budget and the worker was terminated mid-flush anyway. The in-flight flush cannot be shortened, but the caller no longer waits on it beyond its own deadline. Known and unchanged: choosing "Search all providers…" and then picking from the full catalogue records only `provider=search_all`. Fixing that means emitting from the full model picker, which is used outside first-run too, so it needs a product call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
…eanup block `test/cli/tui/command.test.ts` guards the ordering of the TUI worker cleanup by slicing `cli/cmd/tui.ts` between the "clean up TUI worker after failed --session validation" start marker and the next `altimate_change end`. The onboarding telemetry seam added a nested start/end pair inside that region, so the slice ended early and `await stop()` fell outside it — the guard failed with the cleanup index at -1. Uses a single-line marker instead, and says why in the comment so the next person adding something there does not reintroduce it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
full receipts (7 sessions)
orchestrator ·
|
| subagent | cost |
|---|---|
| You are exploring the altimate-code monorepo to inform how a **preloaded sample… | 3,624,157 tokens |
| Task: survey the altimate-code repo for conventions I need to follow while fixi… | 2,039,125 tokens |
| You are exploring the altimate-code repo to inform the design of two new featur… | 2,460,388 tokens |
orchestrator · 3559fc02
- - - - - - - - - - - - - - - - - - - - - - - - -
AIRECEIPTS
“Investigate telemetry hooks implementation pl…”
Claude Code · Jul 29 2026 05:50 UTC · 4h 15m
claude-opus-5 100%
cache served 99% of input tokens
pre-edit: 5% of tokens (45/317 turns)
(share before the first named edit tool)
Bash...................53,016,584 tok (200 calls)
Edit....................27,693,201 tok (68 calls)
(thinking/reply).........9,889,096 tok (31 turns)
Read.....................6,149,396 tok (28 calls)
Write....................5,357,175 tok (14 calls)
ToolSearch..................567,411 tok (5 calls)
mcp__atlassian__addCommen…...511,535 tok (1 call)
TaskUpdate..................489,339 tok (3 calls)
TaskCreate..................288,417 tok (2 calls)
AskUserQuestion..............150,188 tok (1 call)
mcp__atlassian__getJiraIss…...52,719 tok (1 call)
--------------------------------------------------
TOTAL..............................104,165,061 tok
no price table matched
- - - - - - - - - - - - - - - - - - - - - - - - -
npx aireceipts-cli
github.com/anandgupta42/receipts
- - - - - - - - - - - - - - - - - - - - - - - - -
codex · 71121181
- - - - - - - - - - - - - - - - - - - - - - - - -
AIRECEIPTS
“commit b47757370796f98f13f79e1d0e6db60c3fb14f…”
Codex · Jul 29 2026 08:58:50 UTC · 3m 48s
gpt-5.5 100%
cache served 88% of input tokens
pre-edit: no named edit tool observed
(share before the first named edit tool)
exec_command...............689,866 tok (22 calls)
--------------------------------------------------
TOTAL..................................689,866 tok
no price table matched
- - - - - - - - - - - - - - - - - - - - - - - - -
npx aireceipts-cli
github.com/anandgupta42/receipts
- - - - - - - - - - - - - - - - - - - - - - - - -
codex · ee70612d
- - - - - - - - - - - - - - - - - - - - - - - - -
AIRECEIPTS
“commit ebd2e782a099956fb1865e819f8b686697fb2c…”
Codex · Jul 29 2026 09:15:40 UTC · 3m 17s
gpt-5.5 100%
cache served 83% of input tokens
pre-edit: no named edit tool observed
(share before the first named edit tool)
exec_command...............752,551 tok (30 calls)
--------------------------------------------------
TOTAL..................................752,551 tok
no price table matched
- - - - - - - - - - - - - - - - - - - - - - - - -
npx aireceipts-cli
github.com/anandgupta42/receipts
- - - - - - - - - - - - - - - - - - - - - - - - -
codex · 9f29ce8c
- - - - - - - - - - - - - - - - - - - - - - - - -
AIRECEIPTS
“commit b4252c5f699b70365b9b709d2ec468abca9456…”
Codex · Jul 29 2026 09:37:58 UTC · 3m 56s
gpt-5.5 100%
cache served 93% of input tokens
pre-edit: no named edit tool observed
(share before the first named edit tool)
exec_command.............1,368,318 tok (39 calls)
--------------------------------------------------
TOTAL................................1,368,318 tok
no price table matched
- - - - - - - - - - - - - - - - - - - - - - - - -
npx aireceipts-cli
github.com/anandgupta42/receipts
- - - - - - - - - - - - - - - - - - - - - - - - -
codex · 55530872
- - - - - - - - - - - - - - - - - - - - - - - - -
AIRECEIPTS
“eba46bfba0 fix(onboarding): TUI funnel review…”
Codex · Jul 29 2026 09:56:26 UTC · 2m 44s
gpt-5.5 100%
cache served 78% of input tokens
pre-edit: no named edit tool observed
(share before the first named edit tool)
exec_command...............352,832 tok (19 calls)
--------------------------------------------------
TOTAL..................................352,832 tok
no price table matched
- - - - - - - - - - - - - - - - - - - - - - - - -
npx aireceipts-cli
github.com/anandgupta42/receipts
- - - - - - - - - - - - - - - - - - - - - - - - -
codex · 2594c394
- - - - - - - - - - - - - - - - - - - - - - - - -
AIRECEIPTS
“# Test strategy — onboarding funnel telemetry…”
Codex · Jul 29 2026 10:34:29 UTC · 3m 24s
gpt-5.5 100%
cache served 83% of input tokens
pre-edit: no named edit tool observed
(share before the first named edit tool)
exec_command...............536,427 tok (34 calls)
--------------------------------------------------
TOTAL..................................536,427 tok
no price table matched
- - - - - - - - - - - - - - - - - - - - - - - - -
npx aireceipts-cli
github.com/anandgupta42/receipts
- - - - - - - - - - - - - - - - - - - - - - - - -
Generated by aireceipts
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
Answers "does this user action emit that event, with those properties" at each emission point, rather than trying to reconstruct a whole cross-thread stream. Expected values are written out from the product spec rather than computed the way the implementation computes them, so a test cannot agree with the code while both are wrong. Every assertion here was checked by breaking the implementation and confirming the test fails: - removing the funnel-start gate → the returning-user test fails - treating a skill load as job completion → the skill test fails - dropping `launch_id` from the envelope → the correlation test fails - removing the scan-gate guard → both double-submit tests fail Coverage of note: - A returning user opening the picker is not enrolled in the funnel and is never reported as abandoned. - Quitting after choosing a provider reports `provider_setup`, not `model_picker`. - A warehouse discovered from a dbt profile counts as having a warehouse — the adversarial fixture for the connection-total bug, where only `existing` was being read. - Loading a skill selects an activation job but does not complete one. - Tools run outside an onboarding session emit nothing. - `launch_id` is asserted at the envelope level, because it is attached during App Insights conversion and is invisible to a `Telemetry.track` spy — the transport-level test is the only place the correlation fix is observable. Not covered yet: mounting the provider picker and the Big Pickle interstitial needs the full sync/SDK stack (`LocalProvider` depends on both), so those belong with the app-lifecycle harness. The scan gate is covered because its choice is handed out through a prop and needs no telemetry context. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
1 similar comment
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
The picker was the last uncovered emission point. I had written it off as needing the app-lifecycle harness because `LocalProvider` depends on sync and the SDK — that was wrong. `test/fixture/tui-sdk.ts` already provides a fake fetch and event source, so the real provider stack mounts fine. Covered: the impression carries the trigger that opened the picker; an unset trigger is attributed to `/connect` rather than to a first run; choosing the first row records the gateway; a rapid second Enter records one selection, not two; and the `/` shortcut records the same choice as the search row. Verified by mutation, as with the other funnel tests — removing the activation guard fails the double-Enter test, and changing the default trigger fails the attribution test. The trigger parameter is typed by extraction from the event union, so renaming a trigger breaks this test at compile time rather than silently drifting. The fake SDK serves an empty provider list. That is fine here: the curated rows are hardcoded and the funnel event is recorded at the moment of choice, before the provider lookup that would act on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
1 similar comment
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
…threads Found by driving a real CLI process through a PTY against a local telemetry sink: the eight TUI-thread events arrived with one launch_id and the worker-thread events with a different one. The correlation id could not join the two halves of the funnel — which is the only thing it exists to do. Two approaches that look correct are not, both confirmed end to end: - Mutating `process.env` at runtime. A Bun Worker does not observe it, so the worker minted its own id. This is what the original implementation did, and it was lazy as well, so the value did not exist until the first flush — long after the worker had started. - Deriving it from the process (pid + start time). Both threads really are one process and share a pid, but `process.uptime()` is per-THREAD in Bun, so each thread computed a different start time. What works is handing the id to the worker explicitly at construction, which Bun's Worker supports through its `env` option. Verified: all eleven events from a full first run — across both threads and two different session ids — now carry a single launch_id. Also loosens the launch_id unit test to select envelopes by event name instead of by position. The telemetry buffer is module-global, so a sibling test file can leave events in it that flush alongside these, which made the exact-length assertion fail only when the files ran together. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
1 similar comment
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
…ssion `environment_scan_completed` and `sample_setup_completed` were emitted without a session, so they fell back to the process-global telemetry context — which is set per prompt loop. With two concurrent sessions in `serve`, or in the TUI worker, one session overwrites the other's context and a scan or sample setup gets attributed to whichever session most recently started a turn. Both tools have their own `ctx.sessionID`. The activation plugin was already passing it explicitly; these two were not. Same root cause as the launch-id bug: state that looks process-wide is not describing what you assume it describes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
1 similar comment
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the onboarding funnel event taxonomy to the CLI: 19 events covering first run, provider choice, gateway auth, the scan gate, activation, and drop-off.
Approach
Events are emitted from three places, each chosen because it is where the thing being measured actually happens:
packages/tuicannot import the Telemetry module, so the host injects a callback throughTuiInputand a context carries it down — the same seamcontext/exit.tsxalready uses. No HTTP route and no new API surface: the TUI renders on the process main thread, so this reaches the existing Telemetry instance directly.Telemetry.track()calls.command.execute.beforeandtool.execute.afterhooks.Things worth knowing before reviewing
Three activation events are derived, not observed. The activation menu is not UI — it is text the model writes from a prompt template, and the user replies in free text.
activation_menu_shown,activation_job_selectedandfirst_job_completedare inferred from the closest deterministic signals. Their counts are lower bounds. Two gaps are documented rather than papered over: the "something else" branch has no tool signature and is never counted, and skill-driven jobs cannot be confirmed complete so they are absent fromfirst_job_completedrather than wrongly counted in it.gateway_device_code_issuedis a spec name, not a description. The flow is a browser loopback OAuth; there is no device code. Kept for taxonomy fidelity, with the mismatch noted in the type and the docs.launch_id. The funnel spans two threads and most of it runs before any chat session exists, so the first half stamped an empty session and the second half a real one — nothing to join a run on. Every event now carries a per-launch random id. Not persisted, not derived from the machine or user, not reused across launches. See #1048.Also fixed along the way
Telemetry.shutdown()was not reentrancy-safe, and neither exit path flushed — the TUI handler exits viaprocess.exit(0), skipping the outer flush, and the worker never flushed its own buffer at all. Both threads hold separate buffers.~/.altimate/machine-idcould be created twice on a new install, giving one first run two machine ids.Known gaps
provider=search_all. Fixing it means emitting from the full model picker, which is used well outside first run — wanted a product call before touching it.OPENCODE_FAST_BOOT, sync reports ready before credentials load, so a returning user can transiently look un-onboarded and see the first-run gate. Pre-existing behaviour; these events faithfully report what the UI did.Status
Draft because automated tests for the new code are not written yet — that is the remaining work.
Typecheck is clean on both packages. Full
packages/opencodesuite: 10,780 pass / 7 fail. All 7 failures are pre-existing and unrelated (6 inacp permissions, which reproduce with this branch's changes reverted to the base commit; 1 flakyrunsubprocess test that passes in isolation).Unrelated bug found while working on this and filed separately: #1047.
🤖 Generated with Claude Code
https://claude.ai/code/session_018SLUQF3xgZHsGZHSjxe7vb
Summary by cubic
Adds end-to-end first-run onboarding funnel telemetry across the CLI/TUI and ships a safe, self-contained dbt DuckDB starter sample. Also improves first-run UX (welcome picker, scan gate) and hardens telemetry, auth, and sample setup flows.
New Features
packages/opencode/src/altimate/telemetrywith per-launchlaunch_idfor cross-thread correlation.packages/tui(picker, Big Pickle, scan gate, completion) with double-submit guards.activation_menu_shown,activation_job_selected,first_job_completed, andfirst_prompt_sent.sample-projects/jaffle-shop-duckdb) with precompiled manifest and README; resolver works across dev/test/prod.sample_setuptool: marker-based reuse/upgrade, atomic staging + rename, strict path validation, and orphan sweep.environment_scan_completedin an onboarding-shaped form;/authand/logoutcommands added; provider picker and welcome UX refreshed (Altimate LLM Gateway prioritized).Bug Fixes
machine-id; sharedlaunch_idacross TUI and worker threads; abandonment gated to real first runs.environment_scan_completedandsample_setup_completedby passing each tool’s ownctx.sessionID(no more cross-session misattribution in long-lived workers).cmd.exe.Written for commit ecd94c8. Summary will update on new commits.