Skip to content

release: v0.2.5816 — read CLI + Tier 5 fix + bench data + ACE spec + shootout codegraph #488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
justrach wants to merge 11 commits into from
Closed

release: v0.2.5816 — read CLI + Tier 5 fix + bench data + ACE spec + shootout codegraph #488

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
9f8cfeb
bench(eval): v0.2.5815 cross-corpus head-to-head — codedb vs codegrap…
justrach May 21, 2026
c9dd5b6
feat(cli): add `codedb read` subcommand
justrach May 21, 2026
fc72a92
fix(search): skip Tier 5 full-scan when trigram returned candidates
justrach May 21, 2026
2cdd343
docs(design): ACE integration spec — codedb as a tool ACE wraps
justrach May 21, 2026
e05a3b4
bench(shootout): add codegraph backend (codegraph_search via MCP stdio)
justrach May 21, 2026
143c0ed
Merge PR #484: codedb read CLI subcommand
justrach May 21, 2026
e9d8e34
Merge PR #485: skip Tier 5 when trigram returned candidates
justrach May 21, 2026
43ee79c
release: v0.2.5816 — codedb read CLI + Tier 5 short-circuit
justrach May 21, 2026
0f3574c
Merge PR #483: v0.2.5815 cross-corpus bench results
justrach May 21, 2026
52a8632
Merge PR #486: ACE × codedb integration spec
justrach May 21, 2026
b21c5e7
Merge PR #487: codegraph backend in shootout.py
justrach May 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions benchmarks/search-shootout/results/2026-05-21/flask-run1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# search-shootout — flask

**Date:** 2026-05-21 10:29
**Corpus:** `/Users/blackfloofie/codedb-bench/flask`
**Indexed files:** 127
**Corpus bytes:** 0.6 MB
**Iterations:** 20 warm

## Build phase

| Backend | Cold index time | On-disk size |
|---|---|---|
| fts5_trigram | 0.04s | 2.3 MB |
| fts5_unicode61 | 0.01s | 0.9 MB |
| codedb | 0.05s | 3.6 MB |
| lean-ctx | 8.29s | — |
| codegraph | 0.58s | 3.7 MB |

## Query latency (warm, ms)

> codedb: MCP stdio (one server, many calls).
> fts5_*: persistent SQLite connection.
> lean-ctx: per-call CLI spawn (includes ~700ms binary startup).
> Hit counts are NOT directly comparable across backends — use them as recall sanity (zero vs non-zero).

| query | kind | fts5_tri p50 | fts5_tri p99 | fts5_tri hits | fts5_uni p50 | fts5_uni p99 | fts5_uni hits | codedb p50 | codedb p99 | codedb hits | leanctx p50 | leanctx p99 | leanctx hits | codegraph p50 | codegraph p99 | codegraph hits |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| `useState` | common-identifier | 0.09 | 0.14 | 0 | 0.01 | 0.02 | 0 | 0.66 | 1.39 | 0 | 449.06 | 465.16 | 0 | 0.70 | 1.60 | 0 |
| `useEffect` | common-identifier | 0.01 | 0.01 | 0 | 0.00 | 0.01 | 0 | 0.05 | 0.08 | 0 | 451.48 | 472.32 | 0 | 0.62 | 0.80 | 0 |
| `forwardRef` | camelcase-identifier | 0.01 | 0.02 | 0 | 0.00 | 0.00 | 0 | 0.04 | 0.07 | 0 | 453.87 | 469.16 | 0 | 0.58 | 0.76 | 0 |
| `createElement` | camelcase-identifier | 0.02 | 0.03 | 0 | 0.00 | 0.01 | 0 | 0.05 | 0.08 | 0 | 453.36 | 466.15 | 0 | 0.68 | 0.83 | 0 |
| `Fiber` | substring-identifier | 0.01 | 0.01 | 0 | 0.00 | 0.00 | 0 | 0.04 | 0.09 | 0 | 451.23 | 467.81 | 0 | 0.65 | 0.78 | 0 |
| `Lane` | substring-identifier | 0.01 | 0.01 | 0 | 0.00 | 0.01 | 0 | 0.05 | 0.08 | 0 | 452.00 | 470.30 | 0 | 0.64 | 0.84 | 0 |
| `Suspense` | substring-identifier | 0.00 | 0.01 | 0 | 0.00 | 0.01 | 0 | 0.06 | 0.08 | 0 | 448.85 | 456.00 | 0 | 0.66 | 0.77 | 0 |
| `flushPassiveEffects` | rare-camelcase | 0.00 | 0.01 | 0 | 0.00 | 0.00 | 0 | 0.04 | 0.10 | 0 | 450.76 | 475.44 | 0 | 0.64 | 0.75 | 0 |
| `enableTransitionTracing` | rare-flag | 0.05 | 0.05 | 0 | 0.00 | 0.01 | 0 | 0.05 | 0.08 | 0 | 451.71 | 463.39 | 0 | 0.76 | 1.03 | 0 |
| `scheduleCallback` | camelcase-identifier | 0.04 | 0.13 | 0 | 0.00 | 0.01 | 0 | 0.05 | 0.10 | 0 | 448.54 | 467.48 | 0 | 0.64 | 0.79 | 0 |
| `concurrent` | lowercase-word | 0.02 | 0.03 | 2 | 0.00 | 0.00 | 1 | 0.07 | 0.14 | 10 | 457.91 | 474.26 | 11 | 0.15 | 0.30 | 2 |
| `function` | lang-keyword | 0.03 | 0.04 | 23 | 0.00 | 0.00 | 20 | 0.10 | 0.19 | 20 | 449.05 | 471.18 | 20 | 1.38 | 1.88 | 105 |
| `set` | short-trigram-exact | 0.00 | 0.01 | 46 | 0.00 | 0.01 | 29 | 0.10 | 0.22 | 20 | 437.50 | 457.94 | 20 | 0.35 | 0.49 | 70 |
| `ReactDOMRoot` | rare-camelcase | 0.02 | 0.03 | 0 | 0.00 | 0.00 | 0 | 0.05 | 0.17 | 0 | 440.40 | 455.97 | 0 | 0.64 | 0.82 | 0 |
| `xyzzy_react_does_not_exist` | negative | 0.01 | 0.01 | 0 | 0.00 | 0.00 | 0 | 0.06 | 0.09 | 0 | 444.86 | 453.83 | 0 | 0.57 | 0.69 | 0 |

43 changes: 43 additions & 0 deletions benchmarks/search-shootout/results/2026-05-21/react-run1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# search-shootout — react

**Date:** 2026-05-21 10:21
**Corpus:** `/Users/blackfloofie/codedb-bench/react`
**Indexed files:** 6,620
**Corpus bytes:** 26.5 MB
**Iterations:** 20 warm

## Build phase

| Backend | Cold index time | On-disk size |
|---|---|---|
| fts5_trigram | 2.06s | 93.5 MB |
| fts5_unicode61 | 0.44s | 36.3 MB |
| codedb | 1.18s | 69.5 MB |
| lean-ctx | 8.25s | — |
| codegraph | 15.12s | 195.5 MB |

## Query latency (warm, ms)

> codedb: MCP stdio (one server, many calls).
> fts5_*: persistent SQLite connection.
> lean-ctx: per-call CLI spawn (includes ~700ms binary startup).
> Hit counts are NOT directly comparable across backends — use them as recall sanity (zero vs non-zero).

| query | kind | fts5_tri p50 | fts5_tri p99 | fts5_tri hits | fts5_uni p50 | fts5_uni p99 | fts5_uni hits | codedb p50 | codedb p99 | codedb hits | leanctx p50 | leanctx p99 | leanctx hits | codegraph p50 | codegraph p99 | codegraph hits |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| `useState` | common-identifier | 0.88 | 1.03 | 674 | 0.02 | 0.02 | 674 | 1.85 | 2.02 | 20 | 470.90 | 491.82 | 20 | 2.42 | 3.34 | 65 |
| `useEffect` | common-identifier | 0.40 | 0.44 | 434 | 0.01 | 0.02 | 426 | 1.02 | 1.19 | 20 | 464.99 | 485.16 | 20 | 5.89 | 6.72 | 101 |
| `forwardRef` | camelcase-identifier | 0.18 | 0.21 | 129 | 0.01 | 0.01 | 127 | 0.25 | 0.32 | 20 | 468.88 | 486.10 | 20 | 2.16 | 3.11 | 55 |
| `createElement` | camelcase-identifier | 1.02 | 1.08 | 403 | 0.01 | 0.02 | 401 | 0.92 | 1.01 | 20 | 502.85 | 516.09 | 20 | 1.61 | 2.30 | 189 |
| `Fiber` | substring-identifier | 0.13 | 0.17 | 303 | 0.01 | 0.01 | 180 | 0.35 | 0.44 | 20 | 528.84 | 570.11 | 20 | 3.22 | 3.81 | 151 |
| `Lane` | substring-identifier | 0.06 | 0.06 | 72 | 0.01 | 0.01 | 40 | 0.12 | 0.20 | 20 | 561.95 | 569.48 | 20 | 2.37 | 3.87 | 305 |
| `Suspense` | substring-identifier | 0.40 | 0.43 | 314 | 0.01 | 0.02 | 259 | 0.54 | 0.68 | 20 | 473.47 | 492.31 | 20 | 2.59 | 3.13 | 52 |
| `flushPassiveEffects` | rare-camelcase | 0.20 | 0.26 | 8 | 0.01 | 0.01 | 7 | 0.15 | 0.29 | 11 | 613.06 | 633.25 | 20 | 1.76 | 2.99 | 4 |
| `enableTransitionTracing` | rare-flag | 0.78 | 0.86 | 25 | 0.01 | 0.01 | 25 | 0.19 | 0.33 | 20 | 605.41 | 627.06 | 20 | 1.48 | 2.04 | 162 |
| `scheduleCallback` | camelcase-identifier | 0.43 | 0.47 | 22 | 0.01 | 0.01 | 22 | 0.16 | 0.25 | 20 | 558.79 | 586.04 | 20 | 1.39 | 1.59 | 38 |
| `concurrent` | lowercase-word | 0.40 | 0.44 | 127 | 0.01 | 0.01 | 75 | 0.24 | 0.32 | 20 | 553.55 | 575.15 | 20 | 1.63 | 2.25 | 222 |
| `function` | lang-keyword | 1.77 | 1.89 | 5286 | 0.07 | 0.09 | 5245 | 16.07 | 16.36 | 20 | 491.60 | 516.44 | 20 | 8.22 | 9.47 | 69 |
| `set` | short-trigram-exact | 0.04 | 0.05 | 2039 | 0.02 | 0.02 | 851 | 3.71 | 4.00 | 20 | 486.10 | 506.05 | 20 | 3.45 | 4.23 | 103 |
| `ReactDOMRoot` | rare-camelcase | 0.13 | 0.16 | 8 | 0.01 | 0.01 | 6 | 0.11 | 0.21 | 8 | 626.39 | 648.94 | 12 | 1.45 | 1.97 | 26 |
| `xyzzy_react_does_not_exist` | negative | 0.20 | 0.22 | 0 | 0.03 | 0.04 | 0 | 0.07 | 0.11 | 0 | 630.53 | 657.50 | 0 | 7.35 | 8.38 | 0 |

41 changes: 41 additions & 0 deletions benchmarks/search-shootout/results/2026-05-21/react-run2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# search-shootout — react

**Date:** 2026-05-21 10:24
**Corpus:** `/Users/blackfloofie/codedb-bench/react`
**Indexed files:** 6,620
**Corpus bytes:** 26.5 MB
**Iterations:** 20 warm

## Build phase

| Backend | Cold index time | On-disk size |
|---|---|---|
| codedb | 1.13s | 69.5 MB |
| lean-ctx | 8.31s | — |
| codegraph | 15.05s | 195.5 MB |

## Query latency (warm, ms)

> codedb: MCP stdio (one server, many calls).
> fts5_*: persistent SQLite connection.
> lean-ctx: per-call CLI spawn (includes ~700ms binary startup).
> Hit counts are NOT directly comparable across backends — use them as recall sanity (zero vs non-zero).

| query | kind | codedb p50 | codedb p99 | codedb hits | leanctx p50 | leanctx p99 | leanctx hits | codegraph p50 | codegraph p99 | codegraph hits |
|---|---|---|---|---|---|---|---|---|---|---|
| `useState` | common-identifier | 2.87 | 4.91 | 20 | 498.69 | 523.87 | 20 | 2.75 | 4.31 | 65 |
| `useEffect` | common-identifier | 1.04 | 1.25 | 20 | 483.85 | 502.82 | 20 | 5.66 | 5.89 | 101 |
| `forwardRef` | camelcase-identifier | 0.24 | 0.34 | 20 | 483.99 | 497.38 | 20 | 2.26 | 2.76 | 55 |
| `createElement` | camelcase-identifier | 0.93 | 1.05 | 20 | 510.25 | 532.48 | 20 | 2.86 | 3.49 | 189 |
| `Fiber` | substring-identifier | 0.39 | 0.81 | 20 | 550.13 | 571.72 | 20 | 4.50 | 5.14 | 151 |
| `Lane` | substring-identifier | 0.11 | 0.24 | 20 | 563.37 | 594.56 | 20 | 2.13 | 2.67 | 305 |
| `Suspense` | substring-identifier | 0.51 | 0.61 | 20 | 487.01 | 501.94 | 20 | 2.69 | 2.99 | 52 |
| `flushPassiveEffects` | rare-camelcase | 0.14 | 0.27 | 11 | 597.10 | 622.17 | 20 | 1.32 | 1.52 | 4 |
| `enableTransitionTracing` | rare-flag | 0.18 | 0.28 | 20 | 593.63 | 604.01 | 20 | 1.52 | 1.94 | 162 |
| `scheduleCallback` | camelcase-identifier | 0.17 | 0.26 | 20 | 545.70 | 565.42 | 20 | 1.40 | 1.68 | 38 |
| `concurrent` | lowercase-word | 0.23 | 0.34 | 20 | 557.32 | 565.11 | 20 | 1.97 | 2.47 | 222 |
| `function` | lang-keyword | 17.88 | 29.38 | 20 | 472.26 | 488.21 | 20 | 8.84 | 10.03 | 69 |
| `set` | short-trigram-exact | 3.95 | 4.13 | 20 | 468.22 | 505.09 | 20 | 5.84 | 8.15 | 103 |
| `ReactDOMRoot` | rare-camelcase | 0.25 | 0.36 | 8 | 632.26 | 680.78 | 12 | 1.84 | 2.36 | 26 |
| `xyzzy_react_does_not_exist` | negative | 0.04 | 0.08 | 0 | 635.35 | 670.92 | 0 | 8.36 | 9.74 | 0 |

43 changes: 43 additions & 0 deletions benchmarks/search-shootout/results/2026-05-21/regex-run1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# search-shootout — regex

**Date:** 2026-05-21 10:27
**Corpus:** `/Users/blackfloofie/codedb-bench/regex`
**Indexed files:** 285
**Corpus bytes:** 5.9 MB
**Iterations:** 20 warm

## Build phase

| Backend | Cold index time | On-disk size |
|---|---|---|
| fts5_trigram | 0.54s | 20.3 MB |
| fts5_unicode61 | 0.12s | 7.8 MB |
| codedb | 0.15s | 13.5 MB |
| lean-ctx | 8.17s | — |
| codegraph | 2.56s | 17.6 MB |

## Query latency (warm, ms)

> codedb: MCP stdio (one server, many calls).
> fts5_*: persistent SQLite connection.
> lean-ctx: per-call CLI spawn (includes ~700ms binary startup).
> Hit counts are NOT directly comparable across backends — use them as recall sanity (zero vs non-zero).

| query | kind | fts5_tri p50 | fts5_tri p99 | fts5_tri hits | fts5_uni p50 | fts5_uni p99 | fts5_uni hits | codedb p50 | codedb p99 | codedb hits | leanctx p50 | leanctx p99 | leanctx hits | codegraph p50 | codegraph p99 | codegraph hits |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| `useState` | common-identifier | 0.17 | 0.23 | 0 | 0.00 | 0.01 | 0 | 1.87 | 16.57 | 0 | 449.84 | 499.80 | 1 | 1.78 | 2.81 | 0 |
| `useEffect` | common-identifier | 0.04 | 0.05 | 0 | 0.00 | 0.01 | 0 | 0.05 | 0.09 | 0 | 447.89 | 453.75 | 1 | 1.62 | 1.95 | 0 |
| `forwardRef` | camelcase-identifier | 0.05 | 0.06 | 0 | 0.00 | 0.01 | 0 | 0.04 | 0.06 | 0 | 448.38 | 459.98 | 1 | 1.64 | 1.93 | 0 |
| `createElement` | camelcase-identifier | 0.08 | 0.09 | 0 | 0.00 | 0.01 | 0 | 0.04 | 0.07 | 0 | 447.22 | 463.45 | 1 | 1.74 | 2.08 | 0 |
| `Fiber` | substring-identifier | 0.01 | 0.01 | 0 | 0.00 | 0.01 | 0 | 0.04 | 0.07 | 0 | 449.86 | 493.24 | 1 | 1.96 | 2.45 | 11 |
| `Lane` | substring-identifier | 0.01 | 0.03 | 3 | 0.00 | 0.01 | 0 | 0.07 | 0.14 | 4 | 445.58 | 457.24 | 1 | 1.89 | 2.25 | 2 |
| `Suspense` | substring-identifier | 0.04 | 0.05 | 0 | 0.00 | 0.01 | 0 | 2.82 | 3.08 | 0 | 444.82 | 468.38 | 1 | 1.68 | 1.99 | 0 |
| `flushPassiveEffects` | rare-camelcase | 0.06 | 0.07 | 0 | 0.00 | 0.01 | 0 | 0.04 | 0.10 | 0 | 444.97 | 471.07 | 1 | 1.59 | 1.80 | 0 |
| `enableTransitionTracing` | rare-flag | 0.16 | 0.19 | 0 | 0.00 | 0.01 | 0 | 0.04 | 0.10 | 0 | 441.13 | 456.66 | 1 | 1.76 | 2.00 | 0 |
| `scheduleCallback` | camelcase-identifier | 0.09 | 0.10 | 0 | 0.00 | 0.01 | 0 | 0.05 | 0.07 | 0 | 443.44 | 449.59 | 1 | 1.85 | 2.28 | 0 |
| `concurrent` | lowercase-word | 0.05 | 0.06 | 2 | 0.00 | 0.01 | 1 | 0.05 | 0.10 | 4 | 448.29 | 472.74 | 3 | 0.33 | 0.46 | 1 |
| `function` | lang-keyword | 0.07 | 0.10 | 65 | 0.00 | 0.01 | 51 | 0.12 | 0.19 | 20 | 429.55 | 435.06 | 20 | 2.09 | 2.59 | 185 |
| `set` | short-trigram-exact | 0.01 | 0.02 | 154 | 0.01 | 0.01 | 127 | 0.13 | 0.23 | 20 | 430.41 | 438.46 | 20 | 2.34 | 3.06 | 115 |
| `ReactDOMRoot` | rare-camelcase | 0.05 | 0.07 | 0 | 0.00 | 0.01 | 0 | 0.06 | 0.13 | 0 | 465.57 | 526.76 | 1 | 1.71 | 1.90 | 0 |
| `xyzzy_react_does_not_exist` | negative | 0.13 | 0.15 | 0 | 0.00 | 0.01 | 0 | 0.04 | 0.08 | 0 | 470.83 | 479.19 | 1 | 1.49 | 1.82 | 0 |

Loading
Loading