Iterator to reference SuperVersion from a pool, just like the thread-local super version technique used in Get()

[siying]

Right now, we reduce the mutex contention of grabbing and referencing count super version using thread-local super version. See https://github.com/facebook/rocksdb/blob/v5.17.2/db/column_family.cc#L1027-L1070

This works well for Get(), but we can't directly use it in iterators. Instead, when creating a new iterator we still increase the reference count while getting it from the thread local cached super version. The reference count is still a potential scalability bottleneck. It isn't usually exposed as a bottleneck but it's a good idea to clean the design to get rid of it.

To achieve it, we can turn the thread-local cache to a pool of cached super version. While creating an iterator, we can fetch a random one (or a core-local one) from the pool and remember where we should return it to. In the future we can consider whether it makes sense to replace Get() path can migrate to the same solution too.

Anyone who is interested is welcome to contribute.

[jwasinger]

Hi @siying , I'd like to try and take this on.

[siying]

@jwasinger awesome! Let me know if you need any help. CC @ajkr

[siying]

I can't assign it to you. I don't know much about github system, but you are more than welcome to work on it!

[jwasinger]

Thanks :) . I will reach out with questions as they arise.

[siying]

Btw this is @nbronson 's suggestion.

[ajkr]

I think it's a good idea to have a pool of cached super versions for iterators. A couple thoughts:

(1) Before extending this change to Get, let's make sure it doesn't regress performance. I suspect it might not be as fast as thread local.
(2) I think random might make more sense than core-local for this case. If we did core-local, then a long-held iterator could prevent all other iterators created by the same core to not have a superversion available to them.

[siying]

@ajkr how about core ID as a hint and can fall back to random?

[siying]

@ajkr I updated the original description according to your comment.

[ajkr]

sure, that sounds good. then a thread serially creating short-lived iterators might get some benefit.

[jwasinger]

@siying @ajkr , I've done a bit of a dive through the docs and codebase to get myself up to speed. I still have a lot of reading to do to get a full idea of what is going on.

One question: Is the bottleneck here due to lifetime of the SuperVersion reference being held longer for iterators than for Get? Presumably preventing the referenced resources from being garbage collected/compacted as they go stale. Ah whoops... I had misunderstood the code.