-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add DYNAMIC_ROUTER setting #6
Conversation
Can I haz review ? |
looking |
lib/carbon/client.py
Outdated
""" | ||
|
||
def __init__(self): | ||
self.queue = deque() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you make it bounded, even if it is a big number.
lib/carbon/client.py
Outdated
log.clients("Destination is down: %s:%d:%s (%d/%d)" % ( | ||
destination[0], destination[1], destination[2], self.retries, | ||
settings.DYNAMIC_ROUTER_MAX_RETRIES)) | ||
if self.retries < settings.DYNAMIC_ROUTER_MAX_RETRIES: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where do you increment retries ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ReconnectingClientFactory does that
The idea is that you want your consistent hashing ring to adapt itself when destinations are down. This change will remove destinations from the router when down and add them back when up. An additional buffer is created to store points when no destination is up, the receiving socket is also paused in this case to apply some back-pressure. When a node is removed from the router its points are re-injected to carbon to go through the new route.
bfbf675
to
80e5f46
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added comment in the code to explain why it looks like that
lib/carbon/client.py
Outdated
log.clients("Destination is down: %s:%d:%s (%d/%d)" % ( | ||
destination[0], destination[1], destination[2], self.retries, | ||
settings.DYNAMIC_ROUTER_MAX_RETRIES)) | ||
if self.retries < settings.DYNAMIC_ROUTER_MAX_RETRIES: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ReconnectingClientFactory does that
👍 |
Will push early next week because it needs to be monitored closely |
The idea is that you want your consistent hashing ring to
adapt itself when destinations are down. This change will
remove destinations from the router when down and add them
back when up.
An additional buffer is created to store points when no
destination is up, the receiving socket is also paused in this
case to apply some back-pressure.
When a node is removed from the router its points are re-injected
to carbon to go through the new route.