docs: update query benchmarks (3.9.4) - #956
Conversation
Greptile SummaryThis automated PR records query benchmark results for version 3.9.4, updating the summary table and latest-results section in
Confidence Score: 5/5Safe to merge — documentation-only, generated by automation, no runtime impact. All findings are P2 (readability/comparability concern about the mid target switch). The data itself is internally consistent and correctly formatted. No code paths are affected. generated/benchmarks/QUERY-BENCHMARKS.md — the percentage deltas in the 3.9.4 rows are compared against a different mid benchmark target (enumNode → ctx), which may mislead readers interpreting the WASM fnImpact regressions. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Release Workflow Trigger] --> B[Run codegraph on itself]
B --> C{For each engine}
C --> D[Native Rust Engine]
C --> E[WASM Engine]
D --> F[Query: fnDeps depth 1/3/5\nhub=buildGraph mid=ctx leaf=docs]
E --> F
F --> G[Query: fnImpact depth 1/3/5]
G --> H[Query: diffImpact]
H --> I[Median over 5 runs]
I --> J[Compute % delta vs prev version]
J --> K[Append to QUERY-BENCHMARKS.md]
K --> L[Open automated PR]
Reviews (1): Last reviewed commit: "docs: update query benchmarks (3.9.4)" | Re-trigger Greptile |
| | 3.9.4 | native | 26.9 ↑10% | 27.3 ↑13% | 27.3 ↑13% | 4.9 ↑11% | 5.1 ↑16% | 5.1 ↑16% | 7.3ms ↓15% | | ||
| | 3.9.4 | wasm | 37.8 ↑10% | 38.5 ↑12% | 39.2 ↑15% | 5.1 ↑19% | 6.2 ↑41% | 6.1 ↑42% | 14.3ms ↑35% | |
There was a problem hiding this comment.
Mid-target change makes percentage deltas not comparable
The mid benchmark target switched from enumNode (3.9.3) to ctx (3.9.4). All percentage arrows in the 3.9.4 rows are computed by dividing the new latency against 3.9.3 latency from a different function. A heavier or lighter ctx graph neighbourhood can explain part (or all) of the observed regression, so readers cannot tell from the table alone how much is engine performance vs. workload change. Consider adding a footnote like *mid target changed from enumNode; deltas not directly comparable* on the 3.9.4 rows, or re-running 3.9.3 with mid=ctx as a baseline.
Automated query benchmark update for 3.9.4 from workflow run #737.