You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we always write every filter we need to fetch to disk. This results in high-ish disk and I/O utilization when booting back up after a long dormant period, or aggressive usage of the GetUTXO call. For "normal" wallets, they'll almost never need to re-match a filter once it's been fetched unless they're doing something like a key import. As a result, we should only actually keep a limited (and configurable) set of filters in memory.
Steps To Completion
Modify the GetCfilter method to only write to disk if specified by the caller. If we don't write the filter to disk, then we should write it to the cache.
Create a new generic (as in []byte) LRU cache. This can double for usage to cache blocks, as well as filters. It may be the case that a more advanced eviction strategy will be more optimal, but we'll at least start with this just to pick the low hanging fruit.
Modify the GetCfilter method to first check the cache, then disk if a filter is requested.
The text was updated successfully, but these errors were encountered:
This looks like a relatively good first issue, I'd like to take a stab at it if it isn't assigned to anyone, what do you think @Roasbeef ? If yes,
I looked through the code and instead of putting the caching inside GetCfilter, what do you think of putting it directly inside the FilterStore ? It seems like a logical place and that way GetCfilter logic would stay the same (other than just passing an extra persistToDisk bool to PutFilter).
For the LRU, what do you think of using an existing lib like golang-lru ? It also has Adaptive Replacement Cache and TwoQueueCache implementations that we could experiment with.
Ideally we don't bring in additional lib just to implement an LRU cache. Also this should be done outside of the FilterStore all together. This is a caching layer that sits on top of the filter store.
Currently we always write every filter we need to fetch to disk. This results in high-ish disk and I/O utilization when booting back up after a long dormant period, or aggressive usage of the
GetUTXO
call. For "normal" wallets, they'll almost never need to re-match a filter once it's been fetched unless they're doing something like a key import. As a result, we should only actually keep a limited (and configurable) set of filters in memory.Steps To Completion
Modify the
GetCfilter
method to only write to disk if specified by the caller. If we don't write the filter to disk, then we should write it to the cache.Create a new generic (as in
[]byte
) LRU cache. This can double for usage to cache blocks, as well as filters. It may be the case that a more advanced eviction strategy will be more optimal, but we'll at least start with this just to pick the low hanging fruit.Modify the
GetCfilter
method to first check the cache, then disk if a filter is requested.The text was updated successfully, but these errors were encountered: