docs: update query benchmarks (3.4.1)#646
Conversation
Greptile SummaryThis is an automated benchmark documentation update for version 3.4.1, generated from workflow run #484. It appends new rows to the summary tables, updates the "Latest results" and "Build Phase Breakdown" sections, and inserts a full JSON benchmark block for 3.4.1 in both Confidence Score: 5/5Safe to merge — documentation-only update with verified internal consistency and all previously requested changes in place. All data values have been cross-checked and are consistent between summary tables, detail sections, and embedded JSON. The Note (3.4.1) requested in the prior review thread is present. The engine parity note is the only new concern, and it is non-blocking: it explicitly links to issue #649 rather than framing the discrepancy as acceptable, so it doesn't fully violate CLAUDE.md's guidance. No source code is touched. No files require special attention; both files are auto-generated documentation. Important Files Changed
Sequence DiagramsequenceDiagram
participant WF as GitHub Actions Workflow #484
participant CG as codegraph CLI (native + WASM)
participant BUILD as BUILD-BENCHMARKS.md
participant QUERY as QUERY-BENCHMARKS.md
participant PR as Pull Request #646
WF->>CG: Run build benchmark (473 files)
CG-->>WF: Build metrics (time, nodes, edges, DB size)
WF->>CG: Run query benchmark (fnDeps, fnImpact, diffImpact)
CG-->>WF: Query latencies (hub=buildGraph, mid=noTests, leaf=docs)
WF->>BUILD: Prepend 3.4.1 rows + JSON block + parity note
WF->>QUERY: Prepend 3.4.1 rows + JSON block + Note(3.4.1)
BUILD-->>PR: Commit updated BUILD-BENCHMARKS.md
QUERY-->>PR: Commit updated QUERY-BENCHMARKS.md
Reviews (3): Last reviewed commit: "fix(docs): restore 3.4.1 query benchmark..." | Re-trigger Greptile |
| | 3.4.1 | native | 8.9 ↑5% | 8.9 ↑5% | 8.8 ↑5% | 3.2 ~ | 3.1 ↓3% | 3.1 ↓3% | 8.1ms ↑45% | | ||
| | 3.4.1 | wasm | 9.1 ↑7% | 9.2 ↑7% | 9.1 ↑7% | 3.3 ~ | 3.2 ↓3% | 3.2 ↓3% | 7.1ms ↑45% | |
There was a problem hiding this comment.
Unexplained 45%
diffImpact regression may benefit from a note
Both engines show a significant diffImpact latency jump from 3.4.0 → 3.4.1 (5.6ms→8.1ms native, 4.9ms→7.1ms wasm, both +45%). Precedent in this file — the <!-- NOTES_START --> block already contains explanatory notes for the large deltas in 3.3.1 and 3.4.0.
In absolute terms the new values (8.1ms / 7.1ms) fall within the historical range (e.g. 3.1.3 native: 8.3ms, 3.3.0 native: 8.8ms), so the +45% delta is inflated by 3.4.0 being an unusually low baseline. Still, without a note, a future reader comparing only consecutive rows may flag this as a regression worth investigating.
If there is a known explanation (e.g. 3.4.0's diffImpact was unusually fast due to a small staged diff, or the mid-query target changed from rule to noTests affecting scope), adding a Note (3.4.1): entry in the NOTES_START block — consistent with the notes for 3.3.1 and 3.4.0 — would keep the file self-explanatory.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Added a Note (3.4.1) entry in the NOTES_START block explaining the +45% diffImpact delta: the 3.4.0 baseline was unusually low (5.6ms/4.9ms), the new absolute values (8.1ms/7.1ms) fall within historical norms, and the mid-query target changed from rule to noTests. Consistent with the existing note format for 3.3.1 and 3.4.0.
There was a problem hiding this comment.
…034126' into benchmark/query-v3.4.1-20260327-034126
…ts (#646) The previous fix commit accidentally removed all 3.4.1 data instead of adding the explanatory note. This restores the summary table rows, "Latest results" section, and JSON block for 3.4.1.
|
All three P1 issues addressed in 2de77ca:
The previous fix commit (9dbca9f) accidentally removed all 3.4.1 data during merge conflict resolution. The net diff is no longer empty — the PR now correctly adds the full 3.4.1 benchmark record plus the explanatory note. |
Automated query benchmark update for 3.4.1 from workflow run #484.