Skip to content

refactor(store): optimize account_codes pruning#2092

Merged
kkovaacs merged 2 commits into
nextfrom
krisztian/improve-prune-account-codes
May 18, 2026
Merged

refactor(store): optimize account_codes pruning#2092
kkovaacs merged 2 commits into
nextfrom
krisztian/improve-prune-account-codes

Conversation

@kkovaacs
Copy link
Copy Markdown
Contributor

Analyzing traces for current testnet pointed out that there's a serious bottleneck in how we prune account_codes table:

image

It turns out that SQLite was not using the right index for the SQL subquery we were running to get the set of code_commitment values that are still required based on account state:

sqlite> SELECT DISTINCT code_commitment from accounts where code_commitment IS NOT NULL AND (block_num >= 1699000 OR is_latest = 1);
╭─────────────────────────────────────────────────────────────────────╮
│                           code_commitment                           │
╞═════════════════════════════════════════════════════════════════════╡
│ x'002f96792ef149703666adbc840029b96c13b8f1157f32249aa66160fbacfaab' │
[...]
│ x'ffe89f315e3021dde591bdcfb2546cd77b0a4cd8b7271a82d5308b6dd9086fe3' │
╰─────────────────────────────────────────────────────────────────────╯
Run Time: real 1.221431 user 0.993738 sys 0.223425

sqlite> EXPLAIN QUERY PLAN SELECT DISTINCT code_commitment from accounts where code_commitment IS NOT NULL AND (block_num >= 1699000 OR is_latest = 1);
QUERY PLAN
`--SEARCH accounts USING INDEX idx_accounts_code_commitment (code_commitment>?)

This PR splits prune_account_codes SQL query into indexed recent-history and latest-state branches to avoid SQLite scanning account history via code_commitment for the full OR predicate.

To make sure that both branch queries are optimal it also adds a migration that removes is_latest from idx_accounts_prune_code for the recent-history branch (not needed) and adds idx_accounts_latest_code_commitment for latest account rows.

With these changes applied:

sqlite> SELECT DISTINCT code_commitment
FROM (
    SELECT code_commitment
    FROM accounts INDEXED BY idx_accounts_prune_code
    WHERE code_commitment IS NOT NULL
      AND block_num >= 1699000
    UNION ALL
    SELECT code_commitment
    FROM accounts INDEXED BY idx_accounts_latest_code_commitment
    WHERE code_commitment IS NOT NULL
      AND is_latest = 1
);
╭─────────────────────────────────────────────────────────────────────╮
│                           code_commitment                           │
╞═════════════════════════════════════════════════════════════════════╡
│ x'002f96792ef149703666adbc840029b96c13b8f1157f32249aa66160fbacfaab' │
[...]
│ x'ffe89f315e3021dde591bdcfb2546cd77b0a4cd8b7271a82d5308b6dd9086fe3' │
╰─────────────────────────────────────────────────────────────────────╯
Run Time: real 0.027463 user 0.023466 sys 0.003874

sqlite> EXPLAIN QUERY PLAN SELECT DISTINCT code_commitment
FROM (
    SELECT code_commitment
    FROM accounts INDEXED BY idx_accounts_prune_code
    WHERE code_commitment IS NOT NULL
      AND block_num >= 1699000
    UNION ALL
    SELECT code_commitment
    FROM accounts INDEXED BY idx_accounts_latest_code_commitment
    WHERE code_commitment IS NOT NULL
      AND is_latest = 1
);
QUERY PLAN
|--CO-ROUTINE (subquery-2)
|  `--COMPOUND QUERY
|     |--LEFT-MOST SUBQUERY
|     |  `--SEARCH accounts USING COVERING INDEX idx_accounts_prune_code (block_num>?)
|     `--UNION ALL
|        `--SEARCH accounts USING COVERING INDEX idx_accounts_latest_code_commitment (code_commitment>?)
|--SCAN (subquery-2)
`--USE TEMP B-TREE FOR DISTINCT

Split `prune_account_codes` SQL query into indexed recent-history and
latest-state branches to avoid SQLite scanning account history via
code_commitment for the full OR predicate.

Add a migration that removes `is_latest` from `idx_accounts_prune_code`
for the recent-history branch (not needed) and adds
`idx_accounts_latest_code_commitment` for latest account rows.
@kkovaacs kkovaacs added the no changelog This PR does not require an entry in the `CHANGELOG.md` file label May 18, 2026
@kkovaacs kkovaacs marked this pull request as ready for review May 18, 2026 20:45
@kkovaacs kkovaacs merged commit fb31dcc into next May 18, 2026
19 checks passed
@kkovaacs kkovaacs deleted the krisztian/improve-prune-account-codes branch May 18, 2026 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no changelog This PR does not require an entry in the `CHANGELOG.md` file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants