-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
accounts_loading: use load account with cache for filtering #1408
base: master
Are you sure you want to change the base?
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #1408 +/- ##
=========================================
- Coverage 82.7% 82.7% -0.1%
=========================================
Files 871 871
Lines 369894 369992 +98
=========================================
+ Hits 306158 306226 +68
- Misses 63736 63766 +30 |
another impl here would be to modify |
That is intentional, we wanted to runt the counter experiment of #1157 (comment).
That is acting as a pre-fetch for the subsequent actual load in transaction account loading. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Sounds like the side effect of putting these accounts in the read cache (if they aren't already in the write cache) is desired.
When I look at the impl for agave/accounts-db/src/accounts_db.rs Lines 5085 to 5120 in 9dd01f5
but we do not put the loaded account into the read cache if it's not there. Is the problem this PR is solving actually to put the account into the read cache? |
Yes, that is the idea. Though, I suspect it will make little difference because in the subsequent transaction account loading we put programs in the read cache, so in the next TX batch they are already in. |
I see. Is this PR still the right direction then? The results from @HaoranYi indicate an improvement. Is that still accurate? |
The profiler from @alessandrod shows that |
Putting into the cache after checking owners helps the account loading later in the same TX batch. Instead of two read-cache misses, we only have one miss. That's probably why read cache miss is lower with this PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep planning to do this today, please don't merge yet |
![]() As you can see the cursed account_matches_owners is 59% of execution. About 3/4 is agave/accounts-db/src/accounts_db.rs Line 5045 in ec9bd79
account_matches_owners , we're probably hitting disk for anything that is in the write cache.
We should probably just remove this cursed function. Load all the accounts, then filter the programs later once you have the accounts. |
That's similar to #1157. The only difference is that we don't put program accounts into read cache in #1157. |
Yeah that sounds better than keeping account_matches_owners |
Problem
account_matches_owner
doesn't put missed account into read-only cache, which can cause one additional read-only cache misse in subsequent transaction accounts loading.Summary of Changes
Replace
account_matches_owner
with regular accounts load.Fixes #