Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

External shard allocation strategy via explicit requests #28175

Closed
chbatey opened this issue Nov 15, 2019 · 1 comment
Closed

External shard allocation strategy via explicit requests #28175

chbatey opened this issue Nov 15, 2019 · 1 comment

Comments

@chbatey
Copy link
Member

@chbatey chbatey commented Nov 15, 2019

A number of use cases have come up for a shard allocation strategy that would rely on an API for explicitly setting the location of shards and being able to update the location of a shard dynamically. These include:

  • Aligning shards with Kafka partition assignment from a Kafka consumer, having shards follow kafka partitions
  • More efficient moving of shards when doing a rolling update e.g. a k8s operator that took over shard assignment as it adds and removes nodes
  • Explicit control during auto scaling

One idea is to publish the desired changes to distributed data and a shard allocation strategy that listens listens to these changes and takes action on the next rebalance.

@chbatey

This comment has been minimized.

Copy link
Member Author

@chbatey chbatey commented Nov 20, 2019

Sample of how this may work: akka/akka-samples#155

@patriknw patriknw mentioned this issue Dec 9, 2019
7 of 11 tasks complete
@patriknw patriknw mentioned this issue Jan 8, 2020
13 of 15 tasks complete
@chbatey chbatey added this to the 2.6.2 milestone Jan 21, 2020
@chbatey chbatey closed this Jan 21, 2020
Akka 2.6.x automation moved this from In progress to Done Jan 21, 2020
@patriknw patriknw changed the title Dynamic shard allocation strategy via explicit requests External shard allocation strategy via explicit requests Jan 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Akka 2.6.x
  
Done
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.