Even more scalable ETS ordered_set with write_concurrency #1997
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Move lookup from analyze_pattern to callers.
DbTableCATree has no static stack.
Brute force solution will always iterate tree from slot 0 and forward. ToDo1: Yield. ToDo2: Maybe optimize by caching AVL tree size in each base node.
{RouteNodes, BaseNodes, MaxRouteTreeDepth}
to easier generate a routing tree for test without having to spend cpu to provoke actual repeated lock conflicts.
with ets_force_split
to not have to backtrack up on the stack.
Once an iteration key has been found, never fall back to first/last key in next/prev tree as trees may split or join under our feet. I.e we must always use previous key when searching for the next key.
to actually pass the copy to lock checker.
It's possible to first find an empty base node and then retry and find the same base node as invalid. It's a benign race with join which first makes the old invalid 'neighbor' accessible from 'gparent' before replacing it with 'new_neighbor'.
Easier to read and debug, and about the same lines of code.
The original implementation did not do this due to fear of bad performance. But we think the negative effect of "leaking" empty base nodes is more important to fix. To get the bad performance a special kind of access patterns is needed where base nodes are frequently emptied and then repopulated soon again. ets_SUITE:throughput_benchmark for example did not show any negative effect from this commit at all.
for different lock instances.
We no longer lock more than one base node at a time. We do however trylock a second base node at join.
to generate a routing tree with keys that fit each test case.
to provoke iteration over a moving ordered_set with write_concurrency and make sure we hit all "stable" keys.
sverker
deleted the
sverker/erts/ordered_set-select-improvements/OTP-15325
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a continuation on #1952 by @kjellwinblad already merged to master for OTP-22.
#1952 introduced actual scalable
write_concurrencyforordered_settables with a novel data structure called contention adapting search tree.The implementation did however use a fallback method for functions like
ets:selectaccessing multiple keys, where all sub-trees were first merged into one and the old implementation then called. This PR lifts that restriction and alletsfunctions now operate on the contention adapting search tree without collapsing it.Here are some benchmarks showing all access patterns now perform well, including those containing "selectAll" and "partial_select1000".
http://erlang.org/~sverker/ets_ordered_set_nomerge_select.html