Mempool multifetch

[arvidn]

This PR is best reviewed one commit at a time. But the second commit contains the bulk of the changes.

checkpoint merge

Reverse this patch when merging to main:

diff --git a/tests/core/mempool/test_mempool_manager.py b/tests/core/mempool/test_mempool_manager.py
index ce187fe7f..7a7d32899 100644
--- a/tests/core/mempool/test_mempool_manager.py
+++ b/tests/core/mempool/test_mempool_manager.py
@@ -1753,8 +1753,8 @@ async def test_mempool_timelocks(cond1: List[object], cond2: List[object], expec
     )
 
     coin_spends = [
-        make_spend(coins[0], IDENTITY_PUZZLE, Program.to([cond1])),
-        make_spend(coins[1], IDENTITY_PUZZLE, Program.to([cond2])),
+        CoinSpend(coins[0], IDENTITY_PUZZLE, Program.to([cond1])),
+        CoinSpend(coins[1], IDENTITY_PUZZLE, Program.to([cond2])),
     ]
 
     bundle = SpendBundle(coin_spends, G2Element())

Purpose:

This patch improves the disk I/O performance of the mempool by fetching coin records in batches instead of one at a time. It primarily reduces CPU usage when adding spend bundles to the mempool, which is especially critical when updating the mempool through the new-block slow-path.

There are two key changes in this PR:

get_coin_records

The MempoolManager and Mempool objects are passed a function from the CoinStore, get_coin_record, that looks up a single coin record. The first change is to instead of pass in get_coin_records. This function fetches multiple coin records in a single query.

This change requires updates a lot of tests, which ends up being the build of the changes. The core changes, however, are in def validate_spend_bundle() in mempool_manager.py.

new-peak slow-path

The second change (3rd commit) is to, when entering the slow-path to update the peak; fetch all spent coins in a single query, stick the CoinRecords in a dictionary. Pass in a local version of get_coin_records that only looks up coins from the local dictionary to add_spend_bundle(), when adding all the transactions into the new mempool.

This turns thousands of individual CoinRecord lookups into a single query.

Current Behavior:

Mempool takes a function that can fetch one CoinRecord. Coin record are looked up one at a time when validating transactions.

New Behavior:

Mempool takes a function that can fetch multiple CoinRecords. Coin record are looked up in a single query.
When updating the peak through the slow-path, all coin spends are looked up in a single query, to speed up re-adding the transactions to the new mempool instance.

Testing Notes:

This was tested on IPS machines. The wall-clock time was not improved as much as the CPU usage:

patch	mempool size	wall-clock time	time per spend
before	3551	44.42s	12.5ms
after	7069	67.71	9.57ms

Before

After

[coveralls-official]

Pull Request Test Coverage Report for Build 7304023557

• 92 of 92 (100.0%) changed or added relevant lines in 6 files are covered.
• 16 unchanged lines in 5 files lost coverage.
• Overall coverage decreased (-0.002%) to 90.465%

---

Files with Coverage Reduction	New Missed Lines	%
chia/rpc/rpc_server.py	1	88.36%
chia/rpc/util.py	1	93.22%
chia/simulator/setup_nodes.py	1	98.31%
chia/server/node_discovery.py	4	79.55%
chia/full_node/full_node.py	9	84.6%

Totals
Change from base Build 7294432788:	-0.002%
Covered Lines:	93907
Relevant Lines:	103760

---

💛 - Coveralls

[wjblanke]

aok

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[wjblanke]

aok