feat(cache): tree-sitter cache management + surrender telemetry (#933 phase 7)#959
Merged
Conversation
…phase 7) Closes the tree-sitter integration feature (#933). Lazy-loaded parsers gain a first-class CLI surface for cache management; verbose mode surfaces a discoverability hint when the fast path falls through to regex. ## New `coco cache` subcommands Extends the existing `coco cache` (diff-summary info / clear) with three tree-sitter subcommands: - \`coco cache parsers\` — show every manifest language with its current cache status (cached size or "not cached" + the fetched-size estimate), version pin, and source URL. Footer summarizes total disk usage + quick-reference commands. - \`coco cache prefetch [languages...]\` — download specific parsers (e.g. \`coco cache prefetch py rs go\` or \`coco cache prefetch all\`). When invoked with no args AND stdin is a TTY, opens an interactive checkbox picker. In non-interactive contexts (CI, pipes), no-arg invocations error out with usage hints instead of hanging on a prompt. - \`coco cache clear-parsers\` — wipe \`~/.cache/coco/tree-sitter/\`. Idempotent; reports a per-language ✓ for each removed file. Aliases mirror \`COCO_PREFETCH\` env grammar: \`py\` / \`python\`, \`rs\` / \`rust\`, \`go\` / \`golang\`, \`all\`. ## Surrender telemetry In verbose mode, when the language-aware fast path is enabled and the parser chain falls through to LLM, emit a discoverability hint: \`Tree-sitter parser surrendered for 'python'; using regex fallback. Hint: \`coco cache parsers\` to inspect, \`coco cache prefetch python\` to enable.\` Quiet on the default path; visible only when the user is debugging summary quality. Hint copy adapts: bundled-language surrenders (\`ts\` / \`js\`) point at \`coco cache prefetch all\` because TS / TSX wasms are always shipped (the surrender is from a parser-init failure, not a missing download); lazy-loaded languages get a per-language prefetch hint. ## Implementation ### \`cache.ts\` (lazy-load cache module) - New \`getCachedParserStatus(language)\` returns \`{ language, cached, path, bytes?, mtime? }\` for the table renderer + interactive picker. - New \`clearCachedParser(language)\` unlinks the cached .wasm. Idempotent; returns \`true\` when a file was actually removed. ### \`structuralParserRegistry.ts\` - New \`hasTreeSitterParser(language)\` lets the LLM fallthrough path know whether a tree-sitter parser is registered for the language — used by the surrender-telemetry hint. Doesn't expose internals; the caller just needs the boolean. ### \`summarizeLargeFiles.ts\` - Surrender-telemetry block fires after the registry returns undefined and BEFORE the cache lookup. Only emits when the chain includes a tree-sitter parser, so regex-only languages don't get a misleading hint. ### \`commands/cache/\` - \`config.ts\` gains the \`CACHE_SUBCOMMANDS\` enum and a positional \`[languages..]\` for prefetch. Yargs validates the subcommand set; unknown tokens get caught by the language resolver. - \`handler.ts\` adds three new branches: - \`parsers\` calls \`renderParsersTable\` - \`prefetch\` resolves tokens via \`parsePrefetchEnv\` (reusing the env-var grammar), prompts when interactive, and delegates to \`prefetchTreeSitterParsers\`. Failed downloads → \`process.exitCode = 1\`. - \`clear-parsers\` walks every manifest entry, calls \`clearCachedParser\`, reports per-language status. ### \`inquirerPrompts.ts\` - New \`checkboxPrompt\` helper. Same dynamic-import shim as the other prompts; reuses the codebase's standard pattern for ESM inquirer modules under ts-jest. ## Tests 4 new test cases in \`handler.test.ts\` cover the new subcommands: \`parsers\` lists every manifest language, \`prefetch\` warns on unknown tokens, \`clear-parsers\` reports no-op when empty AND removes cached files when present. Test isolation: each test sets \`COCO_CACHE_DIR\` to the same tmp dir the existing tests use for \`XDG_CACHE_HOME\`, so the tree-sitter cache lives inside the per-test sandbox. ## Manual validation \`\`\` $ COCO_CACHE_DIR=/tmp/coco-phase7-smoke coco cache parsers Tree-sitter parser cache Python not cached (448.0 KB when fetched) Rust not cached (1.05 MB when fetched) Go not cached (212.1 KB when fetched) cached: 0/3 total on disk: 0 B $ coco cache prefetch py · Python: downloading https://cdn.jsdelivr.net/.../tree-sitter-python.wasm… ✓ Python parser cached (447 KB) Summary: 1 downloaded · 0 already cached · 0 failed $ coco cache clear-parsers ✓ cleared Python Cleared 1 parser(s) from ~/.cache/coco/tree-sitter/ \`\`\` ## Validation - \`npx tsc --noEmit\` → 0 errors - \`npm run test:jest\` → 1674/1674 pass (3 of 4 consecutive runs clean, 1 flake on the pre-existing scenarioInputs timeout pattern) - \`npx eslint\` on touched files → clean - Manual: all four subcommands round-trip cleanly ## Out of scope (genuine future work) - **Eval-harness side-by-side regex-vs-tree-sitter comparison in the report output**. Today the eval reports per-fixture outcomes but doesn't discriminate WHICH parser produced each summary. Surfacing the regex vs. tree-sitter delta requires registry injection at eval time (the harness builds its own parser chain instead of using the global). Reasonable follow-up; not gating on #933 closure. ## #933 status: feature complete | Phase | Status | |---|---| | 1.0 — Registry abstraction | ✓ #950 | | 1.1 — TS/TSX bundled | ✓ #955 | | 2 — Polish + ESM jest + arrow-fn fixture | ✓ #956 | | 3 — Lazy-load infra + Python | ✓ #957 | | 5 — Rust | ✓ #958 | | 6 — Go | ✓ #958 | | **7 — Cache CLI + telemetry** | **this PR** | Closes #933.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #933 — the final phase for the tree-sitter integration. Lazy-loaded parsers gain a first-class CLI surface for cache management; verbose mode surfaces a discoverability hint when the fast path falls through to regex.
New `coco cache` subcommands
Aliases: `py` / `python`, `rs` / `rust`, `go` / `golang`, `all` — same grammar as `COCO_PREFETCH`.
Surrender telemetry
In verbose mode, when the language-aware fast path is enabled and the parser chain falls through to LLM, emit a discoverability hint:
Quiet on the default path. Hint copy adapts: bundled-language surrenders point at `coco cache prefetch all`; lazy-loaded languages get a per-language prefetch hint.
End-to-end smoke
```
$ COCO_CACHE_DIR=/tmp/smoke coco cache parsers
Tree-sitter parser cache
Python not cached (448.0 KB when fetched)
Rust not cached (1.05 MB when fetched)
Go not cached (212.1 KB when fetched)
cached: 0/3 total on disk: 0 B
$ coco cache prefetch py
· Python: downloading https://cdn.jsdelivr.net/.../tree-sitter-python.wasm…
✓ Python parser cached (447 KB)
Summary: 1 downloaded · 0 already cached · 0 failed
$ coco cache clear-parsers
✓ cleared Python
Cleared 1 parser(s) from ~/.cache/coco/tree-sitter/
```
Test plan
Out of scope
Eval-harness side-by-side regex-vs-tree-sitter comparison in the report output requires registry injection at eval time (the harness builds its own parser chain instead of using the global). Genuine follow-up; not gating on #933 closure.
#933 — feature complete
Closes #933.