Jump to conversation
Unresolved conversations (1)
@carllin carllin Jul 8, 2021
Primary concerns here are: 1) If a set of hotly updated pubkeys end up in the same bin, this could cause a performance degradation, especially if people are grinding for certain vanity pubkeys. </br> A good benchmark of this would be to generate a bunch of keys with the same first byte so they end up in the same bin, then run a set of transfers from those keys to another set of keys, and compare the perf to master. 2) Contention with lookups in clean: https://github.com/solana-labs/solana/blob/a64ad36697dceb28ac858eab465487b511a60a91/runtime/src/accounts_db.rs#L1809. The larger the active set grows, the more keys will need to be cleaned in every root, which means it's likely a significant number of keys per bin will need to be cleaned every iteration. For example if we assume the set of dirty keys per clean interval (100 slots) is `100,000`, then we will have in estimation `100,000 / number of bins` conflicting keys with the main replay index update here. This could be an issue if you are trying to update a key in a particular bin that is also currently being used by clean, which should happen every `1/number of bins` updates, so likely at least once in a slot if there's an ongoing clean. Ideally in these cases replay would be prioritized over clean, but there's no guarantee of that. 3) RPC lookups (not even scans) for existing accounts like `get_balance` or `get_account` can no longer run concurrently with updates, so a lot of RPC lookups may starve progress here. This would be another good thing to try with an RPC node. I think Stephen has set up a bench for this in the past to test RPC scan + index contention, which we could repurpose for other RPC queries. The metric to look at for contention would be `accounts_db_store_timings::update_index`
Outdated
runtime/src/accounts_index.rs
Resolved conversations (0)