Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Consistent hashing with bounded loads algorithm for sharding #16564

Merged
merged 6 commits into from
Jun 5, 2024
Merged

feat: Consistent hashing with bounded loads algorithm for sharding #16564

merged 6 commits into from
Jun 5, 2024

Commits on Jun 4, 2024

  1. Adds consistent hashing with bound loads sharding algorithm 

    Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
    @akram
    akram committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    ca2e87a View commit details
    Browse the repository at this point in the history

  2. Make the assignement consistent accross all clusters 

    - The assignment or running of the algorithm has to be consistent across all the clusters. Changed the function to return a map where the consistent hash will be used to build the map

- Modifications to the createConsistentHashsingWithBoundLoads function. This will create the map for cluster to shard. Note that the list must be consistent across all shards so that is why the cluster list must be sorted before going through the consistent hash algorithm

Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
    @akram
    akram committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    6c48de6 View commit details
    Browse the repository at this point in the history

  3. Extracting constant and simplifying boolean expression 

    Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
    @akram
    akram committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    0cf03cd View commit details
    Browse the repository at this point in the history

  4. Update docs: consistent-hashing sharding algorithm 

    Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
    @akram
    akram committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    b349f81 View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2024

  1. Merge branch 'master' into consistent-sharding-with-bounded-loads-akram

    @ishitasequeira
    ishitasequeira committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    7dd101d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into consistent-sharding-with-bounded-loads-akram

    @ishitasequeira
    ishitasequeira committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    427bd5a View commit details
    Browse the repository at this point in the history