Show the user their own rows at the end of hyp init - #407
Conversation
Setup ended on a suggested query that could not run (`select count(*) from logs` against a dataset the install does not create) and on no evidence that capture worked at all. Both are now one thing: a rendered block of the user's own traffic, printed by the wizard and re-runnable as `hyp query overview`. The block is four sections - providers and models, sessions and tokens per day, repos, tools - counted in input/cached/output tokens rather than rows, because a "part" is an internal grain nobody outside the codebase should have to learn. Bars are two-tone over input+output with cache excluded; it would swamp both (cache is 98.9% of tokens). Both callers share one planner. A timed probe measures this machine, then picks the widest window that fits a 5s/150k-row budget, and the chosen window is always stated so a short one reads as a stated scope rather than a wrong answer. They diverge only on overrun: the wizard holds a deadline and renders whatever finished (naming the unfinished sections as unfinished), because a stall at the end of a successful install reads as a broken install; the command has no deadline, because there the user asked and is watching, and holds --days either way. Nothing in the wizard's half can fail setup: the whole step - queries, render, and write - sits in one try, and the caller discards its result. Withheld rows (LLP 0105) are disclosed on both surfaces, deduped across the five statements and worded by the query verb's own renderer so the two cannot drift. The freshness line is dropped in the wizard only: mid-install it names a condition the user cannot act on. Also fixes a null-propagation bug the block would have inherited: an unguarded `cache_read + cache_write` is NULL for every OpenAI row, silently zeroing their cache reads. Per-term coalesce, pinned by a test, corrected in LLP 0035's canonical SQL and in both report skills. Closes #389 Closes #390 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review round 1 -
|
philcunliffe
left a comment
There was a problem hiding this comment.
Reviewed at 0e29586 under neutral:review (review-only, so nothing was pushed). Full findings are in the review record comment above.
No blockers. The cache-read bug fix is real, complete across the tree, and pinned by a test that runs through the real query seam. Containment, the deadline machinery, disclosure wiring, and LLP discipline all check out.
Requesting changes on five findings that make the block report something other than what is true, which matters more than usual on a feature whose whole purpose is an honest picture of the user's own data:
overview.js:106- repos fold count saturates at 12 (SQLlimit 20vsMAX_REPO_ROWS = 8), and the "no repo recorded" line can be dropped entirely on a machine with more than 20 repos. Contradicts the comment at:100-103.overview.js:95- the daily table truncates at 14 rows with no fold line, while the header states the full window.overview.js:550- "Nothing recorded yet" prints when every row was withheld, while stderr says rows were withheld.first_look.js:135-140+first-look.test.js:169-191- the EPIPE claim does not match Node's behavior (EPIPE on a pipe is an async'error'event, not a synchronous throw), and the test's stub throws synchronously.hyp query overview | headstack-traces and exits 1.query.js:97-105---days=7is silently ignored, and unknown flags are not rejected, because argv is hand-parsed instead of going throughargvToParams.
This verdict is advisory. neutral does not ready or merge contributor PRs, so the call is yours: if you would rather land it as-is, all of these are follow-up sized and can ride a tracking issue. Reply here either way and neutral will pick it up next tick.
Five places where a block whose whole purpose is an honest picture of the user's own data quietly said something narrower than the truth. **Fold counts were computed against a LIMIT, not the data.** `repos` carried `limit 20` while the renderer shows 8, so "+ N more repos" saturated at 12: with 30 repos the user was told 12 were hidden when 22 were. Worse, the repo-less group sorts by token volume like any other row, so past 20 repos it fell off the end and took its own "+ N sessions with no repo recorded" line with it - exactly the disclosure #398 makes load- bearing. Both LIMITs dropped; the grouping was computed in full either way, the LIMIT only decided how much of it the renderer got to see. **The daily table truncated silently.** `limit 14` under a header that states a 30-day window, with no fold line, so anyone summing the column got half the period they had just been told they were reading. Now folds in the renderer with a stated count, like every other section. **"Nothing recorded yet" was false when every row was withheld.** The LLP 0105 filter taking everything left an empty probe and that sentence on stdout, contradicted by the withheld-row count on stderr. The runner now reports whether it withheld, and the renderer picks the sentence that is true. **`--days=7` was silently ignored.** Hand-rolled `indexOf('--days')` missed the `--flag=value` form and accepted unknown flags, so a user who pinned a window got the auto-planned one with no indication. Routed through `parseCommandArgv`, which the rest of this file already uses. **`NO_COLOR` was ignored on a TTY.** Both call sites gated on `isTty` alone; added `useColor` next to it in stdio.js and used it in both. Also: the wizard's notice sink now closes when the step returns, so a query abandoned at the deadline cannot print a disclosure after the privacy narration; `buildOverviewSql` asserts `since` is a plain date rather than resting on a projector invariant maintained in another package; the EPIPE claim in the first-look comment and test was corrected to state only what it pins (synchronous write failures - async stream errors bypass every try/catch and are filed as #409); an inline `import('...')` type and six em dashes were removed per CLAUDE.md. Deferred with issues rather than decided unilaterally: #409 (CLI-wide async EPIPE handling) and #410 (the block renders 132 columns; getting under 80 means compact counts like `1.2B`, which changes what the block is). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks - this was a genuinely useful review. The clustering observation was the right read: findings 1, 2, and 3 are the same mistake in three places, a block whose purpose is an honest picture quietly reporting something narrower. All five are fixed in 256dfc3, plus 7, 8, 9, and 10. Two are deferred with issues rather than decided unilaterally, and one claim I could not reproduce. Fixed1 - repos fold count. Confirmed exactly as described: 30 repos reported "+ 12 more". Dropped the 2 - daily truncation. Same fix, and I agree it is the worse of the two because the header states the window right above the table. 3 - false empty state. The runner records whether the LLP 0105 filter withheld anything ( 5 - 7 - NO_COLOR. Added 8 - style. Both fixed. On the em dashes: your read is the charitable one but it is still the rule, and matching local style is how a file stays non-compliant forever. 9 - late disclosure. Went with gating over threading a signal: the sink now closes when the step returns. An AbortSignal through 10 - Corrected rather than fixed4 - EPIPE. You are right that the comment overclaims, and I have corrected it and the test to state only what they pin (synchronous write failures). But I could not reproduce the crash, on either surface:
So the structural point stands and the fix belongs CLI-wide, but Deferred with an issue6 - terminal width. Confirmed and worse than reported: 132 columns here, not 101. The bar is not the problem - shrinking Verification2707 tests pass (was 2696; 11 added), typecheck clean, all 13 smoke flows from the release checklist green. LLP 0135 gains a On the pre-existing |
|
Correcting a measurement in my previous comment: I reported the block as 132 columns wide. That was wrong. This changes the conclusion on finding 6. I argued compact units ( #410 has been rewritten with the correct measurement, the per-column breakdown, and a concrete fix (thread |
Review round 2 -
|
| stdout writes | bytes | piped to head -1 |
|---|---|---|
| 1 | 4,000 | clean, exit 0 |
| 1 | 70,000 | EPIPE stack, exit 1 |
| 2 | 4,000 each | EPIPE stack, exit 1 |
The crash needs a single write over the buffer, or a second write after the reader is gone. runQueryOverview writes the block in exactly one stdout.write() and the renderer folds every section, so the text block stays ~1.7 to 3.6 KB regardless of cache size. It can hit neither condition. Confirmed through real dispatch at both heads, including 400 repos by 60 days at round 1's head: clean, exit 0. Your correction to the comment and the test is right, and #409 is the right home for the structural gap.
Findings
1. BLOCKER. The new withheld-rows empty state tells the user to run a flag that hyp query overview rejects. overview.js:592-596 ends with Re-run inside one of those directories, or with --include-local-only. That flag is a query sql verb flag (verb.js:36); QUERY_OVERVIEW_SCHEMA (commands/query.js:84-92) declares only json, sql, and days, and overviewRunnerFromCtx never passes includeLocalOnly to executeQuerySql. Round 2's parseCommandArgv migration (fix 5) turned what would have been a silent ignore into a hard refusal, so the two fixes collide:
$ node bin/hypaware.js query overview --include-local-only
hyp query overview: unknown flag --include-local-only
usage: hyp query overview [--json] [--sql] [--days <n>]
exit=2
Reproduced end to end through real dispatch with one fixture row under a .hypignore local-only directory and the caller outside it: stderr says "rerun with --include-local-only", the block itself repeats the advice, and following it exits 2.
Failure scenario: a user whose recorded sessions all live under a local-only directory runs hyp query overview, or simply finishes hyp init, since the same block closes the wizard. They are told twice to re-run with a flag that does not exist on this surface. The fix whose stated goal was "the renderer picks the sentence that is true" ships a sentence whose second half is not, and query-overview.test.js:734 (assert.match(stdout.text(), /--include-local-only/)) now pins the wrong advice, so it will not self-correct.
Fix, either: drop the clause from overview.js:594 and suppress or reword the renderLocalOnlyNotice override on this surface, or add 'include-local-only' to QUERY_OVERVIEW_SCHEMA and thread it through to executeQuerySql. Update the test either way. Minor, same line: it says "in this window" when a fully withheld probe means no window line was printed at all.
2. NON-BLOCKING, but it changes what #409 says. Dropping the SQL LIMITs made --json unbounded, and that does cross the pipe buffer. overview.js:106-108,125-127 plus commands/query.js:203. The text renderer folds, so fix 1 and 2 are safe there. But --json dumps repoRows and dailyRows whole, and those are exactly the two statements whose LIMIT was removed. Measured through real dispatch:
| input | round 1 head 0e29586 |
round 2 head 256dfc3 |
|---|---|---|
--json, 400 repos x 60 days |
8,609 B, | head -1 clean |
89,868 B, uncaught EPIPE, exit 1 |
--json, 30 repos x 730 days |
(capped by the limits) | 117,177 B, uncaught EPIPE, exit 1 |
| text, any size | ~3.6 KB, clean | ~3.6 KB, clean |
hyp query sql cannot do this: it is capped by DEFAULT_QUERY_MAX_BYTES = 32_768. overview --json has no such bound and was, until this round, incidentally bounded by the SQL limits. So on a cache spanning about two years, hyp query overview --json | head -20, or | less with an early quit, prints a stack trace and exits 1.
This does not undo fix 1 or 2. The #folds reasoning in LLP 0135 is sound and the scan cost is unchanged (at an absurd 20,000 distinct repos the whole run is 18.8s vs 15.2s, dominated by the scan; at realistic cardinality there is no measurable difference). The point is narrower: #409's "no live reproduction" note is now false, and the measurements above belong on that issue. A byte cap on the --json payload would close it in-PR; otherwise #409's stdout 'error' handler should land before users with long histories hit it.
3. NON-BLOCKING, latent. overview.js:309 declares let withheld = false (the flag sawWithholding() returns) and :338 declares const withheld = renderLocalOnlyNotice(...), a string, inside run. No defect today, since say() is a separate closure and nothing reads withheld inside run above line 338. But any future line inside run that touches it above 338 gets a TDZ ReferenceError, in the function that decides whether an LLP 0105 disclosure happens. Worth renaming one.
Round 1 fix verification
| # | Verdict |
|---|---|
| 1 repos fold | PASS. limit 20 gone (:127); hidden computed from the full named set, loose from all repo-less rows. Pinned at query-overview.test.js:797: 30 repos plus a repo-less group gives + 22 more repos and + 41 sessions with no repo recorded, and asserts + 12 more is absent. |
| 2 daily fold | PASS. limit 14 gone (:108); MAX_DAY_ROWS plus + N earlier day(s) in this window (:735-738). Pinned both ways (30 rows folds, 3 rows prints no line). Bonus catch: Math.max(...) now spreads shown rather than the unbounded array. |
| 3 false empty state | PASS on mechanism, defective on wording. sawWithholding() is a per-runner closure created fresh in overviewRunnerFromCtx, so no cross-run leak is possible, and both callers build a runner per invocation. But see finding 1. |
| 4 EPIPE | Round 1 was wrong, see above. Comment and test correctly narrowed; #409 open and accurate. |
5 --days |
PASS. Verified against the real CLI: --days 7 and --days=7 both work, 0/-1/abc/3.5 exit 2 with "expects a positive integer (got 0)", --bogus exits 2, a bare positional exits 2, --help exits 0. No previously-valid invocation broke; newly refused flags (--refresh, --format) were silently ignored before, which is an improvement. |
| 6 terminal width | Deferred legitimately. #410 open, and it corrects itself: 96 columns, explicitly retracting the 132 figure because awk counts bytes and the bar glyphs are 3 bytes each. |
7 NO_COLOR |
PASS. useColor (stdio.js:21-24) at both sites, isTty semantics unchanged, every other isTty call site untouched, and empty-string NO_COLOR correctly does not disable colour (matches no-color.org). Pinned for the query surface; the wizard call site has no test. |
| 8 style | PASS. @import { OverviewWindow } at the top of the test file; zero em dashes on any added line in the PR diff. |
| 9 late disclosure | PASS, and no suppression regression (the thing I most wanted to check). notices.close() (wizard/index.js:186) runs strictly after runWizardFirstLook resolves, and every notice for a rendered section is emitted synchronously inside runner.run() during the awaited collectOverview, so no in-window disclosure can be dropped. A section still in flight contributes no rows and is named as unfinished. |
10 since assert |
PASS. /^\d{4}-\d{2}-\d{2}$/ at :91-93, with '' allowed for the no-window render. Tested with an injection payload and a bare word; both callers catch, so a malformed cached date degrades to a skipped step rather than a stray query. |
Verification run
npm test at this head: 2707 tests, 2699 pass, 7 fail, 1 skipped. npm run typecheck exit 0. Targeted node --test across the four changed test files: 113 pass, 0 fail.
Baselines from clean git archive trees: merge-base 78447ae 2637 tests / 7 fail; round 1 head 0e29586 2696 / 7 fail; this head 2707 / 7 fail. The failure count did not grow at any point, all in test/core/leave-command.test.js, still the only failing file. Your "2696 to 2707, 11 added" is accurate.
One calibration note on those pre-existing failures: the "8 failures" figure quoted earlier is master's. ae63a02 has 11 tests in that file with 8 failing; this branch forks from 78447ae and has 10 with 7 failing. Master added one more failing leave test the branch has not picked up. Unrelated to this PR either way.
Deferrals ground-truthed with gh issue view: #409 open and matching, #410 open and matching.
Requesting changes on finding 1 alone. Findings 2 and 3 are yours to weigh, and finding 2 may be better handled as an update to #409 than as work in this PR.
This is round 2, which is neutral's cap, so it will not keep re-reviewing. The verdict stays advisory: neutral does not ready or merge contributor PRs, so landing this is your call and the maintainer's. For what it is worth, the response to round 1 was unusually thorough, and finding 1 is the kind of defect that only appears when two independently correct fixes meet.
philcunliffe
left a comment
There was a problem hiding this comment.
Round 2 reviewed at 256dfc3. Full findings in the review record above.
One blocker, and it is an interaction between two of your round 1 fixes rather than a mistake in either: the new withheld-rows empty state (overview.js:592-596) advises --include-local-only, but QUERY_OVERVIEW_SCHEMA (commands/query.js:84-92) does not declare it, and the parseCommandArgv migration turned that from a silent ignore into exit 2, unknown flag. A user whose sessions are all local-only is told twice to run a flag that refuses. query-overview.test.js:734 currently pins the wrong advice.
Nine of ten round 1 findings verified as genuinely fixed, including the one I most wanted to check: gating the notice sink does not suppress any in-window disclosure.
Round 1's EPIPE finding was wrong and I have corrected it in the record. It does not reproduce at either head; your buffer explanation is right, and the false positive came from a test stub that threw synchronously. One thing to add to #409 though: dropping the SQL limits made --json unbounded, and that payload does cross the 64 KB buffer (89,868 B at 400 repos x 60 days, uncaught EPIPE, exit 1). So #409's "no live reproduction" note is now out of date.
This is round 2, neutral's cap, so it will not keep re-reviewing. The verdict is advisory as always: neutral does not ready or merge contributor PRs, so whether this lands is your call.
… of #407) **Blocker: the block advised a flag it refused.** The withheld-rows notice names `--include-local-only` as the remedy, on stderr and again in the new empty state - and round 1's move to `parseCommandArgv` turned that flag from a silent no-op into `exit 2, unknown flag`. A user whose sessions are all local-only was told twice to run something that refuses. Fixed by supporting the flag rather than softening the advice: the disclosure is right that this is the way to see those rows. Declared on `QUERY_OVERVIEW_SCHEMA` with the same consent wording as the `query sql` verb, threaded through `overviewRunnerFromCtx` to `executeQuerySql`. The wizard never passes it - nothing in a setup step should quietly widen what a captured transcript can carry, and whoever reads the notice can run the command themselves. **The EPIPE finding was right after all, and this PR is what reaches it.** I reported no live reproduction; that was measured on payloads under the ~64 KiB pipe buffer, where the write completes before the reader's exit can matter. Dropping the SQL limits made `--json` unbounded (the counts behind the fold lines have to be exact), and past the buffer an async 'error' event is fatal: `exit 1` and a stack trace from a command that had already done its work. `bin/hypaware.js` now installs a stdout/stderr 'error' listener before anything writes (`cli/stream_errors.js`). EPIPE is swallowed - a reader closing the pipe is `| head` saying enough, not a failure to report - and the run finishes its normal path, observability shutdown included, exiting with the code the command chose. Pinned against a real pipe, both directions: 400 KB into a closed reader exits 7 with an empty stderr with the handler, exits 1 with a stack without it. That is the test round 1 correctly said a synchronously-throwing stub could not stand in for. The first-look comment no longer claims async EPIPE is unreachable; it says where the handling lives and why it cannot be a command's to catch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Round 2 addressed in 64adc0d. Both findings were right, and the second one was right about me being wrong, which I appreciate the follow-up on. The blockerFixed by supporting Declared on Tested both directions: without the flag the rows are withheld, disclosed, and the empty state names it; with it, the rows appear and stderr is silent. Plus a test that the usage line lists every flag the codec accepts, since the failure mode here was precisely a gap between what the output advises and what the parser takes. The EPIPE correctionYou are right and I was wrong, and the interesting part is that the finding became true because of round 1. My "no live reproduction" was measured on payloads under the ~64 KiB buffer, where the write completes before the reader's exit can matter. Dropping the SQL limits made Measured here, both directions against a real pipe:
So I fixed it rather than leaving it in #409, which is now closed with that correction.
The first-look comment no longer claims async EPIPE is unreachable. It says where the handling lives and why it can never be a command's to catch. State2714 tests pass (7 added), typecheck clean, 13/13 smokes green. LLP 0135 #disclosure records why the override has to exist wherever the disclosure names it. Understood on round 2 being the cap - no further review needed from your side. #410 (terminal width) stays open as the only deferred item; it is cosmetic, does not bite above 96 columns, and its body now carries the corrected measurement and a worked fix. |
Review round 3 -
|
|
Round 3 reviewed at Approving. The blocker is genuinely fixed, verified end to end against a real cache and the real CLI, in both directions and on the mixed path. You fixed it the harder and better way: supporting the flag rather than softening the advice, and you were right that the stderr notice named it on every withheld run, so the bug was broader than round 2 reported. The consent surface got the hardest look, since a flag that widens disclosure is where a review should be least willing to take a claim on trust. It holds: the schema description is byte-identical to the One correction owed on my side: round 2 measured a real EPIPE crash, but round 3 could not reproduce it through the One non-blocking item remains, documented in the record: Three rounds in, this has had more scrutiny than most PRs get. Merging is yours: neutral does not ready or merge contributor PRs. |
Setup ended on a suggested query that could not run (
select count(*) from logs, against a dataset the install does not create) and on no evidence that capture worked at all. Both are now one thing: a rendered block of the user's own traffic, printed by the wizard and re-runnable ashyp query overview.Closes #389
Closes #390
The block
Four sections - providers and models, sessions and tokens per day, repos, tools - counted in input/cached/output tokens rather than rows. A "part" is an internal grain (LLP 0026) that nobody outside the codebase should have to learn, and "3,412 parts" answers a question no one asked.
Bars are two-tone over
input + outputwith cache excluded: cache is 98.9% of tokens, so including it makes every bar the same bar. Each table captions what its own bar charts, since they differ by section.One planner, two overrun behaviors
Both callers share one plan: a timed probe measures this machine, then picks the widest window that fits a 5s / 150k-row budget. The chosen window is always stated, so a short one reads as a stated scope rather than a wrong answer. Measured against simulated slowness, the planner picks 31 / 27 / 6 / 2 days at 1x / 3x / 10x / 50x.
They diverge only when the plan turns out wrong:
budget + 3s) and renders whatever finished, naming the unfinished sections as unfinished rather than omitting them. A stall at the end of a successful install reads as a broken install.hyp query overviewhas no deadline. There the user asked and is watching; no answer is worse than a slow one, and they hold--dayseither way.Nothing in the wizard's half can fail setup: the whole step - queries, render, and write - sits inside one
try, and the caller discards the result. An EPIPE fromhyp init | headused to escape and exit non-zero from an install that had already fully succeeded.Disclosure
Withheld rows (LLP 0105) are reported on both surfaces, to stderr. A block that quietly drops rows and reads as a complete picture is the failure that LLP exists to prevent, and it lands harder here than on a hand-written query: the user typed no filter and has no reason to suspect one. The five statements dedup to one sentence, worded by the query verb's own
renderLocalOnlyNoticeso the two surfaces cannot drift apart.The freshness line is dropped in the wizard only - mid-install it names a sub-two-minute lag the user did not cause and cannot act on, on a block whose backfilled rows were force-flushed on the way in.
Bug fixed along the way
An unguarded
cast(cache_read) + cast(cache_write)goes NULL for every OpenAI row (they omitcache_write_tokens), silently zeroing their cache reads - 25.5M of them on my machine. Per-termcoalesce, pinned by a test, and corrected in LLP 0035's own canonical SQL and in both report skills, which carried the same shape.Not in scope
#398 (Codex rows carry no
repo_root) stays open. The repos section counts those in a footnote line rather than dropping them.Verification
2696 tests pass,
npm run typecheckclean, and all 13 smoke flows from the release checklist green.🤖 Generated with Claude Code