docs: update query benchmarks (3.9.2)#900
Conversation
Greptile SummaryThis automated PR records the 3.9.2 query benchmark results — updating the summary table, detailed results, notes section, and embedded JSON data block. The two concerns raised in the previous review cycle (diffImpact regression and mid-target shift from Confidence Score: 5/5Safe to merge — documentation-only update with no code changes, all numbers verified consistent, and prior review concerns fully addressed. All percentage deltas in the summary table, detail tables, and JSON data block are internally consistent and mathematically correct. The two concerns from the prior review cycle (diffImpact regression and ctx→node target shift) are explicitly documented in the new Note (3.9.2) with #904 tracking the regression investigation. No P0 or P1 findings remain. No files require special attention.
|
| Filename | Overview |
|---|---|
| generated/benchmarks/QUERY-BENCHMARKS.md | Automated 3.9.2 benchmark data appended; all percentage deltas verified correct, notes address prior review concerns about diffImpact regression (#904) and dynamic mid-target selection. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Workflow Run #689\n3.9.2 Release] --> B[Run codegraph queries\non its own graph]
B --> C[Collect medians\nover 5 runs per metric]
C --> D{Targets selected\ndynamically}
D --> E[hub = most-connected node\nbuildGraph]
D --> F[mid = next-tier node\nnode in 3.9.2]
D --> G[leaf = docs]
E & F & G --> H[Record fnDeps d1/d3/d5\nfnImpact d1/d3/d5\ndiffImpact]
H --> I[Append to QUERY-BENCHMARKS.md\nsummary table + detail + JSON]
I --> J[PR #900 merged\nto main]
Reviews (3): Last reviewed commit: "docs: add Note (3.9.2) to benchmark lege..." | Re-trigger Greptile
| | Version | Engine | fnDeps d1 | fnDeps d3 | fnDeps d5 | fnImpact d1 | fnImpact d3 | fnImpact d5 | diffImpact | | ||
| |---------|--------|----------:|----------:|----------:|------------:|------------:|------------:|-----------:| |
There was a problem hiding this comment.
WASM
diffImpact shows a 94% latency regression
diffImpact latency jumped from 6.5ms to 12.6ms for the WASM engine (nearly double), and native also regressed 28% (7.5ms → 9.6ms). Both regressions are flagged with ↑ arrows in the summary table. The 3.9.1 cycle had just brought both values down significantly (↓18–19%), so this reversal is a real step back. Worth investigating whether a recent change to domain/graph/resolve.ts, queries.ts, or the diff-impact query path is responsible before merging the release that triggered these numbers.
There was a problem hiding this comment.
Acknowledged — the diffImpact regression is real and significant. Since this PR only records benchmark data (no source code changes), investigating the root cause in the query/resolution path is out of scope here. Created #904 to track the investigation. Added a note to the 3.9.2 section in the benchmark legend flagging the regression and pointing to the likely source files.
|
|
||
| **Targets:** hub=`buildGraph`, mid=`ctx`, leaf=`docs` | ||
| **Targets:** hub=`buildGraph`, mid=`node`, leaf=`docs` | ||
|
|
There was a problem hiding this comment.
Mid-target changed from
ctx to node
The mid-connectivity benchmark target shifted from ctx (3.9.1) to node (3.9.2). If this reflects a genuine change in the graph's hub structure that's fine, but it reduces strict apples-to-apples comparability for the fnDeps/fnImpact mid-target rows across versions. Worth a brief note in the benchmark legend explaining that targets are selected dynamically as the most-connected nodes of their tier at run time.
There was a problem hiding this comment.
Fixed — added a Note (3.9.2) to the benchmark legend explaining that targets are selected dynamically at run time as the most-connected nodes of their tier, so the ctx→node shift reflects genuine changes in graph connectivity structure as the codebase evolves.
Automated query benchmark update for 3.9.2 from workflow run #689.