Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Idea: improve select-s performance with lockless word_id -> word translations #23

Open
anton-povarov opened this issue May 1, 2020 · 1 comment

Comments

@anton-povarov
Copy link
Contributor

Currently to translate word_id -> word_str (done for each key in each selected row, potentially millions of times per select) - we need to read lock global dictionary shard.

This incurs significant overhead just for locks themselves.
An in cases where contention might be high (when per-repacker caching is inefficient, e.g. nginx urls) - might also slow down new packets processing.

The lock is only needed, because we use std::deque to find a word by offset, and it might get inserted into while we're reading (changing it's structure).

The proposed idea is to rework dictionary shard to use just a contiguous mmap()-ed memory region, enabling fully-lockless read-at-offset path (as the mmap()-ed region pointer never changes).

The word_t size is less than modern x86 CPU's cache line size, but due to the strong x86 cache-coherence model - this might only incur a performance penalty, but not compromise correctness.

As far as i understand it, anyway :)

@anton-povarov
Copy link
Contributor Author

Preliminary testing on live instances shows no significant impact tbh.
It feels that the change is good, but kinda of small, performance wise.

Will keep it open just for visibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant