Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Applied consistent hashing as a default function #131

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

nishidy
Copy link

@nishidy nishidy commented May 27, 2022

This is to enable consistent hashing to get a server for a key statically. There is already an issue about this raised at #33. Tests are included but I will add more if necessary. I will add features on this to fully utilize this function like decreasing and increasing the number of nodes.

Summary by CodeRabbit

  • New Features

    • Introduced a new continuum structure for improved key-based connection management.
    • Enhanced client capabilities with methods for adding virtual nodes and selecting connections using key hashes.
  • Refactor

    • Updated the client's internal handling of connections to leverage the new continuum system.
  • Bug Fixes

    • Implemented a placeholder hash function to ensure consistent client behavior during development.

…t hasher to get connection index from a key.
@nishidy nishidy changed the title Applied consistent hashing as a hash function instead of using default hasher to get connection index from a key. Applied consistent hashing as a default function May 27, 2022
src/client.rs Outdated Show resolved Hide resolved
src/client.rs Outdated Show resolved Hide resolved
Comment on lines +144 to +146
if idx == self.continuum.vnodes.len() {
idx = 0;
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this make the first node (index zero) more weight than others?

@aisk
Copy link
Owner

aisk commented Nov 27, 2023

@coderabbitai review

Copy link
Contributor

coderabbitai bot commented Nov 27, 2023

Walkthrough

The update restructures the Client to integrate a Continuum for managing a distributed system's nodes. It introduces a consistent hashing mechanism, where a Continuum holds VNodes, and the Client uses this to route requests. The Client now has a new hash function and leverages the Continuum to determine the target node for a given key based on its hash.

Changes

File(s) Change Summary
src/client.rs Introduced Continuum struct with VNode management and search methods; updated Client with new continuum field and methods for node selection based on key hashes; added empty_hash_function.

🐇✨
To the code, we hop and leap,
A Continuum now the Client keeps.
With hashes and nodes, so deftly we weave,
A distributed dance, in Rust, we achieve. 🌐🔑


Tips

Chat with CodeRabbit Bot (@coderabbitai)

  • If you reply to a review comment from CodeRabbit, the bot will automatically respond.
  • To engage with CodeRabbit bot directly around the specific lines of code in the PR, mention @coderabbitai in your review comment
  • Note: Review comments are made on code diffs or files, not on the PR overview.
  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Note: For conversation with the bot, please use the review comments on code diffs or files.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between b51a81f and 6f411ca.
Files selected for processing (1)
  • src/client.rs (5 hunks)
Additional comments: 7
src/client.rs (7)
  • 50-65: The Client struct has been correctly updated to include the continuum field and the hash_function field. The Continuum struct and VNode struct are also correctly defined to support consistent hashing.

  • 72-77: The empty_hash_function is a placeholder and should be replaced with a proper hash function to ensure a good distribution of keys across the continuum.

  • 86-133: The Continuum struct's methods for managing virtual nodes and searching the continuum are correctly implemented.

  • 142-147: The get_connections_index method correctly uses the continuum to find the connection index, ensuring that keys are distributed based on the hash.

  • 181-189: The get_connection method uses a comparison to check if the hash_function is the empty_hash_function. This is a temporary measure and should be replaced with a proper hash function to ensure consistent hashing.

  • 308-324: The gets method is correctly implemented to handle multiple keys by grouping them by connection index, which reduces network workloads.

  • 563-675: The tests cover the bsearch_continuum_index method and key distribution, which is good. Consider adding a test for the empty_hash_function to ensure it behaves as expected until it is replaced with a proper hash function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants