Skip to content

Commit 937b60f

Browse files
github-actions[bot]github-actions[bot]
committed
fix: wire --no-tests flag to all query commands (CLI + MCP)
The -T/--no-tests flag was defined on fn, fn-impact, context, explain, where, diff-impact, and search but missing from query, map, stats, deps, impact, and hotspots. The underlying data functions already supported noTests filtering but the CLI never passed it through. - Add -T/--no-tests to query, map, stats, deps, impact, hotspots in CLI - Add noTests support to queryNameData and statsData data functions - Add no_tests schema + handler passthrough to MCP tools: query_function, file_deps, impact_analysis, module_map - Standardize all --no-tests help text to 'Exclude test/spec files from results' - Add DOGFOOD-REPORT-2.1.0.md with full testing results and suggestions
1 parent 0e15f12 commit 937b60f

5 files changed

Lines changed: 216 additions & 25 deletions

File tree

‎DOGFOOD-REPORT-2.1.0.md‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# Dogfood Report — codegraph v2.1.0
2+
3+
**Date:** 2026-02-23
4+
**Platform:** Windows 11 Pro (win32-x64), Node v22.18.0
5+
**Native binary:** `@optave/codegraph-win32-x64-msvc` 2.1.0
6+
**Active engine:** native v0.1.0 (auto-detected)
7+
**Target repo:** codegraph itself (92 files, JS + Rust)
8+
9+
---
10+
11+
## 1. Test Summary
12+
13+
| Area | Result |
14+
|------|--------|
15+
| `npm install` | OK — native binary + WASM grammars built successfully |
16+
| `npm test` | **494 passed**, 5 skipped, 0 failures |
17+
| `npm run lint` | Clean — no issues |
18+
| Native engine build | 500 nodes, 724 edges |
19+
| WASM engine build | 527 nodes, 699 edges |
20+
| Incremental rebuild (no changes) | Correctly detected "Graph is up to date" |
21+
22+
---
23+
24+
## 2. Commands Tested
25+
26+
All 22 CLI commands were exercised against the codegraph codebase:
27+
28+
| Command | Status | Notes |
29+
|---------|--------|-------|
30+
| `build .` | OK | Both `--engine native` and `--engine wasm` |
31+
| `build .` (incremental) | OK | Correctly skips unchanged files |
32+
| `map` | OK | |
33+
| `stats` | OK | |
34+
| `cycles` | OK | 0 file-level, 2 function-level |
35+
| `deps <file>` | OK | |
36+
| `impact <file>` | OK | |
37+
| `fn <name>` | OK | |
38+
| `fn-impact <name>` | OK | |
39+
| `context <name>` | OK | Full source + deps + callers + tests |
40+
| `explain <file>` | OK | Data flow analysis is very useful |
41+
| `explain <function>` | OK | |
42+
| `where <name>` | OK | |
43+
| `diff-impact main` | OK | 56 functions changed, 31 callers affected |
44+
| `export --format dot` | OK | |
45+
| `export --format mermaid` | OK | |
46+
| `export --format json` | OK | |
47+
| `structure` | OK | 18 directories, cohesion scores |
48+
| `hotspots` | OK | |
49+
| `models` | OK | 7 models listed |
50+
| `info` | OK | Correctly reports native engine |
51+
| `--version` | OK | `2.1.0` |
52+
53+
### Edge cases tested
54+
55+
| Scenario | Result |
56+
|----------|--------|
57+
| Non-existent symbol (`query nonexistent`) | Graceful message: "No results for..." |
58+
| Non-existent file (`deps nonexistent.js`) | Graceful message: "No file matching..." |
59+
| Non-existent symbol (`fn nonexistent`) | Graceful message: "No function/method/class..." |
60+
| `--json` flag on all supporting commands | Correct JSON output |
61+
| `--no-tests` on fn, fn-impact, context, explain, where, diff-impact | Correctly filters test files |
62+
| `--file` filter on fn | Correctly scopes results |
63+
64+
---
65+
66+
## 3. Bugs Found & Fixed
67+
68+
### BUG: `--no-tests` flag missing on `map`, `deps`, `impact`, and `hotspots` CLI commands
69+
70+
**Severity:** Medium
71+
**Commit reference:** `ec158c3` claims to add `--no-tests` to these commands, but the CLI option was never wired up.
72+
73+
**Symptoms:**
74+
- `codegraph map --no-tests` → `error: unknown option '--no-tests'`
75+
- `codegraph deps <file> --no-tests` → `error: unknown option '--no-tests'`
76+
- `codegraph impact <file> --no-tests` → `error: unknown option '--no-tests'`
77+
- `codegraph hotspots --no-tests` → `error: unknown option '--no-tests'`
78+
79+
**Root cause:** The underlying data functions (`moduleMapData`, `fileDepsData`, `impactAnalysisData`, `hotspotsData`) all accept a `noTests` option and implement filtering, but the Commander CLI option definitions in `cli.js` were never updated to add `-T, --no-tests` and pass it through.
80+
81+
**Fix:** Added `-T, --no-tests` option and `noTests: !opts.tests` passthrough to all four commands in `cli.js`.
82+
83+
**Verification:**
84+
- `deps src/builder.js --no-tests` → "Imported by" drops from 5 to 1 (filters 4 test files)
85+
- `impact src/parser.js --no-tests` → Total drops from 30 to 8 files
86+
- All 494 tests still pass after fix
87+
88+
---
89+
90+
## 4. Observations
91+
92+
### 4.1 Engine Parity Gap (Native vs WASM)
93+
94+
| Metric | Native | WASM | Delta |
95+
|--------|--------|------|-------|
96+
| Nodes | 500 | 527 | +27 (+5.4%) |
97+
| Edges | 724 | 699 | -25 (-3.5%) |
98+
| Functions | 315 | 342 | +27 |
99+
| Call edges | 591 | 566 | -25 |
100+
| Call confidence | 96.8% | 99.3% | +2.5pp |
101+
| Graph quality | 83/100 | 82/100 | -1 |
102+
103+
The native engine extracts 27 fewer function symbols but resolves 25 more call edges. This suggests the native engine may be merging/deduplicating some symbols while being better at call-site resolution. The WASM engine has higher confidence (99.3% vs 96.8%) but lower caller coverage (55.5% vs 60.4%).
104+
105+
**Recommendation:** The parity test (`build-parity.test.js`) exists but only checks a small fixture. Consider adding a snapshot test on a larger fixture (or the codegraph repo itself) to track parity drift between engines.
106+
107+
### 4.2 `statsData` Does Not Support `noTests`
108+
109+
The `stats` command's underlying `statsData()` function accepts no options — it always reports counts including test files. Unlike `map`/`deps`/`impact`/`hotspots`, there's no `noTests` filtering path. This is an inconsistency: if a user wants a production-code-only view of their graph, `stats` always includes tests.
110+
111+
### 4.3 `query` Command Lacks `--no-tests`
112+
113+
The `query` command is the only remaining query command without `--no-tests`. It shows callers and callees, which often include test files. Adding `--no-tests` here would complete the consistency story.
114+
115+
---
116+
117+
## 5. Suggestions for Improvement
118+
119+
### 5.1 UX: Consistent Flag Coverage
120+
121+
Add `--no-tests` to all remaining query commands (`stats`, `query`, `cycles`, `export`). Users who use it on one command expect it on all. Alternatively, add a config option `noTests: true` in `.codegraphrc.json` so users don't have to repeat the flag every time.
122+
123+
### 5.2 UX: Default `--no-tests` in Config
124+
125+
Many codebases have large test directories. A `.codegraphrc.json` option like `"excludeTests": true` would let users default to production-only views:
126+
```json
127+
{
128+
"excludeTests": true
129+
}
130+
```
131+
This would save typing `-T` on every command while still allowing `--include-tests` to override.
132+
133+
### 5.3 UX: `map` Could Show Coupling Score
134+
135+
The `map` command shows fan-in/fan-out bars, but doesn't show the actual coupling score (in+out combined). The `stats` command shows "Top 5 coupling hotspots" — `map` could integrate this as a column since it already has the data.
136+
137+
### 5.4 UX: `explain` Is the Most Useful Command for AI Workflows
138+
139+
The `explain` command produces the most AI-agent-friendly output — structured sections (exports, internals, data flow) that give an LLM exactly the context it needs. Consider:
140+
- Making it the default recommendation in the README for AI workflows
141+
- Adding a `--depth` option to recursively explain dependencies (e.g., `explain src/parser.js --depth 1` also explains its imports)
142+
143+
### 5.5 Performance: Build Speed
144+
145+
Building 92 files takes under 2 seconds with the native engine. This is excellent. However, the native engine still prints "Using native engine" to stdout (not stderr), which pollutes piped output. Consider using `console.error` or `process.stderr.write` for status messages, keeping stdout clean for actual data output.
146+
147+
### 5.6 UX: `structure` Cohesion of 0.00 for Test Directories
148+
149+
All test directories show `cohesion=0.00`, which is technically correct (tests import source, not each other) but may alarm users who don't understand the metric. Consider:
150+
- Hiding cohesion for test directories
151+
- Or adding a note like `(test directory — low cohesion expected)`
152+
153+
### 5.7 UX: `diff-impact` Relative to Working Tree
154+
155+
`diff-impact main` is great for PR reviews, but it would be useful to also support `diff-impact HEAD` or `diff-impact` (no args) showing unstaged changes. The CLI help says it supports unstaged, but the default behavior when no ref is given could be better documented.
156+
157+
### 5.8 Missing: `--no-tests` Help Text Inconsistency
158+
159+
Some commands say "Exclude test/spec files from results", others say "Exclude test files from callers", others say "Exclude test/spec files". A consistent description across all commands would be cleaner.
160+
161+
---
162+
163+
## 6. Overall Assessment
164+
165+
Codegraph v2.1.0 on Windows x64 with the native engine is **solid**. All 22 commands work correctly, edge cases are handled gracefully, the test suite is comprehensive (494 tests), and the native binary installs cleanly as an optional dependency.
166+
167+
The one real bug found (missing `--no-tests` wiring on 4 commands) is fixed in this session. The engine parity gap is the most significant technical observation — worth tracking but not blocking since both engines produce usable graphs.
168+
169+
**Rating: 9/10** — Production-ready with minor consistency issues.

‎src/cli.js‎

Lines changed: 16 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -62,46 +62,51 @@ program
6262
.command('query <name>')
6363
.description('Find a function/class, show callers and callees')
6464
.option('-d, --db <path>', 'Path to graph.db')
65+
.option('-T, --no-tests', 'Exclude test/spec files from results')
6566
.option('-j, --json', 'Output as JSON')
6667
.action((name, opts) => {
67-
queryName(name, opts.db, { json: opts.json });
68+
queryName(name, opts.db, { noTests: !opts.tests, json: opts.json });
6869
});
6970

7071
program
7172
.command('impact <file>')
7273
.description('Show what depends on this file (transitive)')
7374
.option('-d, --db <path>', 'Path to graph.db')
75+
.option('-T, --no-tests', 'Exclude test/spec files from results')
7476
.option('-j, --json', 'Output as JSON')
7577
.action((file, opts) => {
76-
impactAnalysis(file, opts.db, { json: opts.json });
78+
impactAnalysis(file, opts.db, { noTests: !opts.tests, json: opts.json });
7779
});
7880

7981
program
8082
.command('map')
8183
.description('High-level module overview with most-connected nodes')
8284
.option('-d, --db <path>', 'Path to graph.db')
8385
.option('-n, --limit <number>', 'Number of top nodes', '20')
86+
.option('-T, --no-tests', 'Exclude test/spec files from results')
8487
.option('-j, --json', 'Output as JSON')
8588
.action((opts) => {
86-
moduleMap(opts.db, parseInt(opts.limit, 10), { json: opts.json });
89+
moduleMap(opts.db, parseInt(opts.limit, 10), { noTests: !opts.tests, json: opts.json });
8790
});
8891

8992
program
9093
.command('stats')
9194
.description('Show graph health overview: nodes, edges, languages, cycles, hotspots, embeddings')
9295
.option('-d, --db <path>', 'Path to graph.db')
96+
.option('-T, --no-tests', 'Exclude test/spec files from results')
9397
.option('-j, --json', 'Output as JSON')
9498
.action((opts) => {
95-
stats(opts.db, { json: opts.json });
99+
stats(opts.db, { noTests: !opts.tests, json: opts.json });
96100
});
97101

98102
program
99103
.command('deps <file>')
100104
.description('Show what this file imports and what imports it')
101105
.option('-d, --db <path>', 'Path to graph.db')
106+
.option('-T, --no-tests', 'Exclude test/spec files from results')
102107
.option('-j, --json', 'Output as JSON')
103108
.action((file, opts) => {
104-
fileDeps(file, opts.db, { json: opts.json });
109+
fileDeps(file, opts.db, { noTests: !opts.tests, json: opts.json });
105110
});
106111

107112
program
@@ -159,7 +164,7 @@ program
159164
.option('-k, --kind <kind>', 'Filter to a specific symbol kind')
160165
.option('--no-source', 'Metadata only (skip source extraction)')
161166
.option('--include-tests', 'Include test source code')
162-
.option('-T, --no-tests', 'Exclude test files from callers')
167+
.option('-T, --no-tests', 'Exclude test/spec files from results')
163168
.option('-j, --json', 'Output as JSON')
164169
.action((name, opts) => {
165170
if (opts.kind && !ALL_SYMBOL_KINDS.includes(opts.kind)) {
@@ -181,7 +186,7 @@ program
181186
.command('explain <target>')
182187
.description('Structural summary of a file or function (no LLM needed)')
183188
.option('-d, --db <path>', 'Path to graph.db')
184-
.option('-T, --no-tests', 'Exclude test/spec files')
189+
.option('-T, --no-tests', 'Exclude test/spec files from results')
185190
.option('-j, --json', 'Output as JSON')
186191
.action((target, opts) => {
187192
explain(target, opts.db, { noTests: !opts.tests, json: opts.json });
@@ -192,7 +197,7 @@ program
192197
.description('Find where a symbol is defined and used (minimal, fast lookup)')
193198
.option('-d, --db <path>', 'Path to graph.db')
194199
.option('-f, --file <path>', 'File overview: list symbols, imports, exports')
195-
.option('-T, --no-tests', 'Exclude test/spec files')
200+
.option('-T, --no-tests', 'Exclude test/spec files from results')
196201
.option('-j, --json', 'Output as JSON')
197202
.action((name, opts) => {
198203
if (!name && !opts.file) {
@@ -395,7 +400,7 @@ program
395400
.option('-d, --db <path>', 'Path to graph.db')
396401
.option('-m, --model <name>', 'Override embedding model (auto-detects from DB)')
397402
.option('-n, --limit <number>', 'Max results', '15')
398-
.option('-T, --no-tests', 'Exclude test/spec files')
403+
.option('-T, --no-tests', 'Exclude test/spec files from results')
399404
.option('--min-score <score>', 'Minimum similarity threshold', '0.2')
400405
.option('-k, --kind <kind>', 'Filter by kind: function, method, class')
401406
.option('--file <pattern>', 'Filter by file path pattern')
@@ -444,13 +449,15 @@ program
444449
.option('-n, --limit <number>', 'Number of results', '10')
445450
.option('--metric <metric>', 'fan-in | fan-out | density | coupling', 'fan-in')
446451
.option('--level <level>', 'file | directory', 'file')
452+
.option('-T, --no-tests', 'Exclude test/spec files from results')
447453
.option('-j, --json', 'Output as JSON')
448454
.action(async (opts) => {
449455
const { hotspotsData, formatHotspots } = await import('./structure.js');
450456
const data = hotspotsData(opts.db, {
451457
metric: opts.metric,
452458
level: opts.level,
453459
limit: parseInt(opts.limit, 10),
460+
noTests: !opts.tests,
454461
});
455462
if (opts.json) {
456463
console.log(JSON.stringify(data, null, 2));

‎src/mcp.js‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ const BASE_TOOLS = [
3030
description: 'Traversal depth for transitive callers',
3131
default: 2,
3232
},
33+
no_tests: { type: 'boolean', description: 'Exclude test files', default: false },
3334
},
3435
required: ['name'],
3536
},
@@ -41,6 +42,7 @@ const BASE_TOOLS = [
4142
type: 'object',
4243
properties: {
4344
file: { type: 'string', description: 'File path (partial match supported)' },
45+
no_tests: { type: 'boolean', description: 'Exclude test files', default: false },
4446
},
4547
required: ['file'],
4648
},
@@ -52,6 +54,7 @@ const BASE_TOOLS = [
5254
type: 'object',
5355
properties: {
5456
file: { type: 'string', description: 'File path to analyze' },
57+
no_tests: { type: 'boolean', description: 'Exclude test files', default: false },
5558
},
5659
required: ['file'],
5760
},
@@ -71,6 +74,7 @@ const BASE_TOOLS = [
7174
type: 'object',
7275
properties: {
7376
limit: { type: 'number', description: 'Number of top files to show', default: 20 },
77+
no_tests: { type: 'boolean', description: 'Exclude test files', default: false },
7478
},
7579
},
7680
},
@@ -408,13 +412,13 @@ export async function startMCPServer(customDbPath, options = {}) {
408412
let result;
409413
switch (name) {
410414
case 'query_function':
411-
result = queryNameData(args.name, dbPath);
415+
result = queryNameData(args.name, dbPath, { noTests: args.no_tests });
412416
break;
413417
case 'file_deps':
414-
result = fileDepsData(args.file, dbPath);
418+
result = fileDepsData(args.file, dbPath, { noTests: args.no_tests });
415419
break;
416420
case 'impact_analysis':
417-
result = impactAnalysisData(args.file, dbPath);
421+
result = impactAnalysisData(args.file, dbPath, { noTests: args.no_tests });
418422
break;
419423
case 'find_cycles': {
420424
const db = new Database(findDbPath(dbPath), { readonly: true });
@@ -424,7 +428,7 @@ export async function startMCPServer(customDbPath, options = {}) {
424428
break;
425429
}
426430
case 'module_map':
427-
result = moduleMapData(dbPath, args.limit || 20);
431+
result = moduleMapData(dbPath, args.limit || 20, { noTests: args.no_tests });
428432
break;
429433
case 'fn_deps':
430434
result = fnDepsData(args.name, dbPath, {

0 commit comments

Comments
 (0)