Skip to content

Add first-run onboarding and diagnostics - #13

Merged
luhe19001 merged 6 commits into
mainfrom
hlu/eng-1315
Aug 5, 2026
Merged

Add first-run onboarding and diagnostics#13
luhe19001 merged 6 commits into
mainfrom
hlu/eng-1315

Conversation

@luhe19001

Copy link
Copy Markdown
Collaborator

Context

coSlash currently gives little guidance when no sessions are available or one agent source cannot be scanned. This adds actionable first-run states and support diagnostics while keeping healthy sources usable.

Changes

  • Show first-run guidance, source coverage warnings, and diagnostics for empty, missing, and unreadable Claude Code and Codex sources.
  • Add coslash doctor text and JSON output with safe paths, counts, versions, storage checks, and failure-aware exit codes.
  • Preserve healthy sources when another fails, and refresh sessions alongside diagnostics after setup.

Test

  • cd collector && gofmt -l ./cmd ./internal && go vet ./... && go test ./... — passed
  • cd frontend && npm run lint && npm test && npm run format:check && npm run build — passed
  • Manual: verified empty, partial-source, unreadable-path, and orphaned-subagent diagnostics through the CLI and local APIs.

Screenshots

  • First-run onboarding — no agent sessions detected
  • Source coverage and diagnostics — one healthy source with another missing or unreadable

Sessions are fetched per time window, so an empty list no longer means
an empty machine. Decide first run from the diagnostics transcript count
instead, and keep the window check as the fallback while diagnostics are
still loading.

@calvintvu calvintvu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Over-engineering review (complexity only — no correctness/security pass)

+1300 / −35 for a diagnostics panel. Roughly a third of it is machinery the feature does not need yet. Priorities below: P1 = cut before merge, P2 = cut, P3 = nit.

P1 — structural, cut before merge

  • collector/cmd/coslash/diagnostics.go L1-40: delete the whole file. A TTL cache + mutex + ?refresh=1 protocol for two directory walks on localhost, single user. Replace with mux.HandleFunc("GET /api/diagnostics", func(w, r) { writeJSON(w, diagnostics.Collect(r.Context(), diagnostics.Default(version))) }) in main.go. This also removes the refresh=1 branch in use-diagnostics.ts L11.
  • collector/cmd/coslash/diagnostics.go L17-21: yagni newDiagnosticsHandlerWithCollect is an injection seam with one caller and no test that uses it. Falls out with the cache.
  • collector/internal/diagnostics/snapshot.go L105-139: yagni the 13-field Deps struct. LookPath, CLIVersion, Home, UserHome, GOOS, GOARCH, Now are all injectable with exactly one injector (Default) and zero tests injecting anything. Call exec.LookPath / runtime.GOOS / time.Now directly and make the signature Collect(ctx, version). ~45 lines.
  • frontend/.../components/SourceCoverageBanner.tsx L1-30: delete. Third surface for the same facts — the Diagnostics button's status dot already signals warn/fail and the dialog already lists them. Removes the banner, uncoveredSources, sourceCoverageMessage, coverageGaps, the flex-wrapper rework in CoslashPage, and 2 of the 4 diagnostics.test.ts cases. ~75 lines.
  • frontend/.../components/FirstRunOnboarding.tsx L1-48: shrink. Duplicates DiagnosticsDialog's loadFailed / loading / checklist / re-run block verbatim, only the chrome differs. One <DiagnosticsPanel diagnostics isLoading loadFailed onRefresh /> used by both.
  • frontend/.../lib/page-copy.ts L7-16: shrink. The {kind:'first-run'} | {kind:'copy',…} union rewrites three return sites so one branch can return a sentinel. Keep the function returning copy and branch on noTranscripts in CoslashContent, which already renders FirstRunOnboarding. Drops ~30 of the 76 new test lines with it.

P2 — cut

  • snapshot.go L45, L153: delete Settings any, always nil; frontend types it settings: null. Nothing reads it.
  • snapshot.go L172-178: shrink synthesis-CLI presence is found by scanning snapshot.Sources for a matching CLI.Name. _, err := exec.LookPath(synthCLI) is one line and stays correct when the synthesis CLI is not an agent source.
  • snapshot.go L273: native filepath.Join(home, "summaries")synthesis.SummariesDir() already exists at collector/internal/synthesis/paths.go:14.
  • snapshot.go L274-282: delete the summaries count. It feeds no check; it is a decorative number in two UIs.
  • snapshot.go L20 + L229: delete maxSkippedPaths = 10. It re-slices a list vendors.Scan already capped at 10 via maxRecordedSkippedPaths.
  • snapshot.go L85-88, L199-234: shrink diagnostics.SkippedPath is a field-for-field copy of vendors.SkippedPath plus a re-map loop. Reuse the vendor type and map only the display paths.
  • diagnostics/paths.go L1-29: shrink two path-redaction functions where displayError is already a blunt strings.ReplaceAll(s, home, "~"). Keep that one, drop the 14-line displayPath. ~20 lines.
  • collector/internal/collector/collector.go L3, L27, L33-40: shrink the root func() field and its error branch exist only to report the root string. Put Root on vendors.SourceScan; SourceHealth collapses to {Agent, Scan, Err}.
  • vendors/claude/discovery.go L10-21: shrink Files() and Scan() both do root → scan → filter. func Files() ([]string, error) { s, err := Scan(); if err != nil { return nil, err }; return s.Files, nil }.
  • frontend/.../components/DiagnosticsButton.tsx L1-27: yagni a whole file for <Button> + icon + status dot, one caller. Inline it in DiagnosticsDialog's DialogTrigger.
  • CoslashPage.tsx L161, L175, L292: delete the noTranscripts prop threaded through two components while diagnostics is passed right alongside it. Derive at the use site.

P3 — nits

  • diagnostics/versions.go L9, L19-22: delete the 64-char truncation of a --version line. Nothing needs the cap.
  • doctor.go L53-60: shrink doctorExitCodeslices.ContainsFunc(snapshot.Checks, func(c diagnostics.Check) bool { return c.Status == diagnostics.StatusFail }), inline at the return.
  • doctor.go L28-31: shrink marker := string(check.Status); if fail { marker = "FAIL" } exists to upcase one of three statuses. strings.ToUpper(string(check.Status)).
  • doctor.go L12-15: delete the log.SetOutput(io.Discard) / restore dance. The noise it hides is the JSONLFilesUnder skip log, which Scan now returns as data anyway.
  • diagnostics/checks.go L4-30: shrink two separate range snapshot.Sources loops (source check, then CLI check) into one.
  • vendors/claude/discovery.go L27: shrink the filepath.Separator + filepath.Join rebuild of "/subagents/workflows/". Release targets are darwin/arm64 and darwin/amd64, and terminalLaunchSupported is darwin-only — keep the literal.

net: −450 lines possible.

@luhe19001

Copy link
Copy Markdown
Collaborator Author

Thanks—applied the reductions with clear behavior and maintenance benefits in 1129419. Diagnostics now runs fresh per request, no longer carries cache/refresh or unused dependency machinery, probes the selected synthesis CLI directly, and omits the duplicate coverage banner and unused summary facts.

I kept the first-run and dialog renderers separate because their actions and facts differ; combining them would add mode-specific indirection. I also retained diagnostic-owned skipped-path/redaction handling and doctor log suppression to preserve API boundaries, safe path display, and clean JSON output.

Validation: npm run lint, npm test, npm run format:check, npm run build, gofmt -l ./cmd ./internal, go vet ./..., and go test ./....

@calvintvu calvintvu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness + security follow-up (reviewed at 1129419)

Two notes before the findings.

My earlier complexity comment was reviewed against 4f5acf0, and one item in it was wrong. I flagged the diagnostics TTL cache as unnecessary state before tracing what Collect costs. It was acting as a rate limiter on the most expensive operation in the app. 1129419 removed it on my advice, so GET /api/diagnostics now runs a full collect per request with no cache and no mutex. Retracting that recommendation — but the fix is not to restore the cache. It is to make the collect cheap enough that no rate limiter is needed. Plan in the last section.

Everything else in that comment that landed in 1129419 was a good call, and the result is a smaller, clearer diff.


Cost of one Collect at 1129419

Step Cost
probeStorage 3 syscalls
collector.Sources()vendors.Scan ×2 2 WalkDirs, stat-only
SessionCountsByAgent()List(0) 2 more WalkDirs over the same roots, then parse() on every transcript ever recorded (8 workers, reads every byte), then probeEnvironment git subprocesses per distinct (cwd, branch)
commandVersion ×2 2 process spawns, ≤2s each

Roughly all of it is one call that exists to produce a map[string]int. GET /api/diagnostics is also a CORS-simple request and nothing on /api checks Origin or Sec-Fetch-Site, so any page the user visits can trigger it cross-origin in a loop. The response is opaque to the attacker, but the full-machine scan and the git process fan-out still run, now unserialized.


High

1. derive() reports "No sessions found on this machine" when collection fails — with the wrong remediation. internal/diagnostics/checks.go

noSessions is computed from source.Sessions > 0, and counts is set to an empty map whenever SessionCounts() errors. So on a machine with thousands of transcripts where collection failed, or where transcripts exist but fail to parse, the user gets a correct per-source FAIL and sources.none FAIL telling them to "Run claude or codex in a repo for one turn". Two contradictory FAILs, and coslash doctor exits 1 with the wrong diagnosis in the exact scenario the tool exists for.

The frontend already has the right predicate — sources.every(s => s.transcripts === 0). Item A below removes this branch entirely.

2. Unauthenticated cross-origin GET triggers the full-machine scan. See the cost table. Fix is A and E below.

Medium

3. Path redaction fails open. CollectuserHome, _ := os.UserHomeDir()

The error is discarded. When os.UserHomeDir() fails, userHome is "" and both displayPath and displayError short-circuit to identity, so every path in the JSON and in the copy bundle becomes absolute. Narrow window, but a privacy control that silently becomes a no-op deserves the error checked.

4. The copy bundle leaks project paths that home redaction cannot reach. lib/diagnostics.ts formatDiagnosticsForCopy + collectSource

Claude transcript paths are ~/.claude/projects/<cwd-slug>/…, where the slug encodes the absolute working directory with dashes — this repo appears as -Users-<user>-centauri-coslash. displayPath/displayError replace /Users/<user>, not -Users-<user>. So skipped[].path carries the full path of every repo with an unreadable transcript into a bundle whose tooltip promises "never transcript content or session names", and which users paste into public issues.

Either redact the slug form too, or omit skipped[].path from the copy payload and keep it in the local UI. The existing "no session content" test won't catch this — it only asserts that fields the formatter never reads are absent.

5. Synthesis check can report a false OK, and hides the fault it should surface. Collect + checks.go

Enabled is derived from settings on disk, but in main.go synthesis is actually off when NewRunner returns an error (discarded via runner, _ =) or when EnsureDirs fails (mgr.SetRunner(nil)). In those cases doctor reports "Enabled with <model>" while no debrief will ever run.

Conversely, when settings are invalid the check says "Disabled; coSlash will show deterministic transcript details only" and never surfaces state.Error — yet per the README, invalid settings also block terminal launches. That is precisely what a user runs doctor to find.

(The exec.LookPath(synthesisCLI) change in 1129419 is the right fix for the CLI half of this.)

6. Empty-root advice. sourceCheck — when Root() fails, Root stays "", producing Could not fully scan : <err> and Fix: Run ls -la and check ownership.

Low

  • coslash doctor --help exits 2. flag.ContinueOnError returns ErrHelp and runDoctor maps every parse error to 2. main() in the same binary treats ErrHelp as success.
  • version[:maxVersionLength] slices bytes, so a 64-byte cut can split a UTF-8 rune and put U+FFFD in the JSON.
  • probeStorage mutates during a read-only checkMkdirAll creates ~/.coslash as a side effect of doctor, and a crash between CreateTemp and Remove leaves .diagnostics-* files behind.
  • "N sessions from M transcripts" reads as a fault on healthy machines — for Claude, Transcripts counts subagent files that collapse into parents, so a normal user sees e.g. "3 sessions from 47 transcripts". Fixed for free by item A below.
  • copyState never resets to idle — the button reads "Copied" until unmount.
  • navigator.clipboard undefined throws synchronously, not caught by the .catch. Low: 127.0.0.1 is a secure context.

Making the collect cheap instead of cached

A. Drop List(0); count from the walk already done. The Scan in Sources() already holds every path, and root vs child is path-derivable for both vendors using helpers that exist:

  • Claude — claude.ParentIDFromPath(file) == "" means root (pure string split, already used by FilesSince).
  • Codex — codex.SessionIDFromRollout(file) != "" means a rollout file (regex on the basename).

Zero file reads, zero subprocesses. Three things fall out: the countsError branch disappears (no second collection left to fail), finding #1 disappears with it, and the "3 sessions from 47 transcripts" wording is fixed because you report root transcripts rather than every subagent file.

The orphaned-subagent check survives in a better form: roots == 0 && files > 0 means every transcript is a child with no parent — exactly what the current Fix text describes, now path-only.

Worth naming the field sessionFiles rather than sessions: this counts transcript files, not sessions after subagent grouping and excludeSynthesisRuns, so coSlash's own synthesis runs are included.

B. One walk, not four. After A, Sources()'s Scan is the only walk. Today each root is walked twice per request — once by Sources(), once by Files() inside List(0).

C. Move --version off the HTTP path. LookPath is a PATH stat and already gives found + path; the dialog's Facts line falls back to cli.path on its own. commandVersion is two process spawns with 2s timeouts, and claude --version alone pays Node startup. doctor should run them — the user asked and latency is irrelevant there. One bool parameter on Collect splits it.

D. Don't fetch on mount. useDiagnostics fires an unconditional useEffect, so every page load pays for the endpoint even though the panel is behind a button. It is needed in exactly two cases, both already known from useSessions: the dialog is open, or sessions.length === 0 (first-run onboarding). Gate the effect on enabled and pass diagnosticsOpen || sessions.length === 0. For the common user the endpoint is never hit. That is a trigger condition, not stored state.

E. Reject cross-origin on /api. Stateless, and worth doing independent of cost: if Sec-Fetch-Site is present and is neither same-origin nor none, return 403. This closes the drive-by vector rather than relying on the handler staying cheap, and it also covers POST /api/launch on the same mux.

After A and C a collect is 2 WalkDirs + 2 LookPaths + 3 storage syscalls — directory-listing cost, nothing worth caching. With D it rarely runs at all.


Tests

3089455 ("chore: defer Go diagnostics tests") deletes diagnostics_test.go and doctor_test.go, so the Go side of this feature ships with no tests. The handler test is legitimately obsolete now that the cache is gone, but renderDoctor/doctorExitCode isn't, and derive()/sourceCheck() are pure functions over a Snapshot — a table test there is ~15 lines and catches finding #1 directly. displayPath/displayError are pure too, and they're a privacy control.

Frontend coverage is reasonable; page-copy.test.ts covers the new branches well.


Minimum before merge

  1. Replace SessionCountsByAgent/List(0) with path-derived counts from Scan (fixes #1, #2, and the transcript-count wording).
  2. Sec-Fetch-Site check on /api.
  3. Redact the -Users-<user>- slug in the copy bundle, or drop skipped[].path from it.
  4. A table test over derive().

@luhe19001

Copy link
Copy Markdown
Collaborator Author

Addressed in 59502b9. Diagnostics now uses path-only session counts, loads only when needed, avoids HTTP subprocesses, and rejects cross-origin API requests. Also fixed misleading empty-state checks, copied-path privacy, settings/storage reporting, and the smaller CLI/UI correctness issues. Validated with frontend lint/tests/build/format and collector gofmt/vet/test.

@luhe19001
luhe19001 merged commit 6754add into main Aug 5, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants