[Wallet synchronization] Add BlockFilterIterator
#12185
Conversation
kiminuo
force-pushed
the
feature/2024-01-04-BlockFilterIterator
branch
from
d7c7703
to
8f6fda3
Compare
| if (!FiltersCache.TryGetValue(currentHeight, out var filterToProcess)) | ||
| { | ||
| // We don't have the next filter to process, so fetch another batch of filters from the database. | ||
| FiltersCache.Clear(); | ||
|
|
||
| var filtersBatch = await BitcoinStore.IndexStore.FetchBatchAsync(new Height(currentHeight), MaxNumberFiltersInMemory, cancellationToken).ConfigureAwait(false); | ||
| foreach (var filter in filtersBatch) | ||
| { | ||
| FiltersCache[filter.Header.Height] = filter; | ||
| } | ||
|
|
||
| filterToProcess = filtersBatch.First(); | ||
| } |
There was a problem hiding this comment.
This got simplified.
|
|
||
| public interface IIndexStore | ||
| { | ||
| Task<FilterModel[]> FetchBatchAsync(uint fromHeight, int batchSize, CancellationToken cancellationToken); |
There was a problem hiding this comment.
btw: This should help with moving SQLite dependency to Daemon (#10650)
There was a problem hiding this comment.
I found a bug while testing, if you change your Height in the wallet file to something lower than the first filter, it will crash. This is not expected because it's easier when wanting a full resync to set the height to 0, and let the software ignores it until the first filter.
I understand that it's not really the responsibility of this PR; but the behavior change reveals this problem, so I think it should be fixed here
diff --git a/WalletWasabi/Wallets/Wallet.cs b/WalletWasabi/Wallets/Wallet.cs
index 6a06b3511..ffd3d49e1 100644
--- a/WalletWasabi/Wallets/Wallet.cs
+++ b/WalletWasabi/Wallets/Wallet.cs
@@ -9,6 +9,8 @@ using System.Threading.Tasks;
using WalletWasabi.Backend.Models;
using WalletWasabi.Blockchain.Analysis.Clustering;
using WalletWasabi.Blockchain.Analysis.FeesEstimation;
+using WalletWasabi.Blockchain.BlockFilters;
+using WalletWasabi.Blockchain.Blocks;
using WalletWasabi.Blockchain.Keys;
using WalletWasabi.Blockchain.TransactionOutputs;
using WalletWasabi.Blockchain.TransactionProcessing;
@@ -473,11 +475,20 @@ public class Wallet : BackgroundService, IWallet
Height bestKeyManagerHeight = KeyManager.GetBestTurboSyncHeight();
+ // Make sure best height is at least height of segwit activation.
+ var startingSegwitHeight = new Height(SmartHeader.GetStartingHeader(Network, IndexType.SegwitTaproot).Height);
+ if (startingSegwitHeight > bestKeyManagerHeight)
+ {
+ KeyManager.SetBestHeights(startingSegwitHeight, startingSegwitHeight);
+ bestKeyManagerHeight = startingSegwitHeight;
+ }
+
using (BenchmarkLogger.Measure(LogLevel.Info, "Initial Transaction Processing"))
{
TransactionProcessor.Process(BitcoinStore.TransactionStore.ConfirmedStore.GetTransactions().TakeWhile(x => x.Height <= bestKeyManagerHeight));
}
+
// Each time a new batch of filters is downloaded, request a synchronization.
var lastHashesLeft = BitcoinStore.SmartHeaderChain.HashesLeft;
while (BitcoinStore.SmartHeaderChain.HashesLeft > 0) |
…2024-01-04-BlockFilterIterator
Merged master with your PR fixing this |
|
cACK |
This PR attempts to do the following things:
WalletFilterProcessorimplementation as it relies onBlockFilterIteratorwhich has the complexity of iterating blocks one by one.FiltersCachebut the issue with this cache is that it is shared by allSyncTypes and as such if turbo sync works with height X and non-turbo with height Y, it might require unnecessary database lookups (cache misses).BlockFilterIteratoralso removes block filters from memory once they are not needed, this should decrease memory use a bit (not peak memory but average memory use).