akka.cluster.sharding - batch update #4600
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* The rebalance in the LeastShardAllocationStrategy is only comparing the region with most shards with the one with least shards. Makes the rebalance rather slow. By default it's only rebalancing 1 shard at a time. * This new strategy looks at all current allocations to find the optimal number of shards per region and tries to adjust towards that value. Picking from all regions with more shards than the optimal. * Absolute and relative limit on how many shards that can be rebalanced in one round. * It's also not starting a new rebalance round until the previous has completed. * unit tests * second phase for fine grained rebalance, due to rounding it will not be perfect in the first phase * randomized unit test * configuration settings * docs
…a/akka#29548) Adds some level of cluster awareness to both the LeastShardAllocationStrategy implementations: * prefer shard allocations on new nodes during rolling updates * don't rebalance during rolling update * don't rebalance when there are joining nodes * don't allocate to leaving, downed, exiting and unreachable nodes * When allocating when there are joining, unreachable, are leaving are de-prioritized to decrease the risk that a shard is allocated just to directly need to be re-allocated on a different node.
* Otherwise it will continue to deallocate when there is no traffic that triggers allocations
… from akka/akka#29463) Avoiding the need for rebalance workers to watch shard regions which is expensive as there is one rebalance worker per shard fix race in rebalance worker (partially migrated from akka/akka#29502)
Aaronontheweb
requested changes
Nov 5, 2020
src/contrib/cluster/Akka.Cluster.Sharding.Tests.MultiNode/MultiNodeClusterShardingConfig.cs
Outdated
Show resolved
Hide resolved
|
|
||
| if (settings.TuningParameters.LeastShardAllocationAbsoluteLimit > 0) |
| @@ -18,7 +18,7 @@ namespace Akka.Cluster.Sharding | |||
| /// TBD | |||
| /// </summary> | |||
| [Serializable] | |||
| public class TunningParameters | |||
| public class TuningParameters | |||
There was a problem hiding this comment.
Theoretically, this is a breaking change but I'm not sure how much trouble is worthwhile to avoid it
src/contrib/cluster/Akka.Cluster.Sharding/Internal/AbstractLeastShardAllocationStrategy.cs
Outdated
Show resolved
Hide resolved
src/contrib/cluster/Akka.Cluster.Sharding/Internal/AbstractLeastShardAllocationStrategy.cs
Outdated
Show resolved
Hide resolved
| @@ -569,6 +569,68 @@ public override bool IsLocal | |||
| public override bool IsTerminated { get { return false; } } | |||
| } | |||
|
|
|||
|
|
|||
| /// <summary> | |||
| /// An ActorRef that ignores any incoming messages. | |||
| @@ -53,6 +53,8 @@ public interface IActorRefProvider | |||
| /// <summary>Gets the dead letters.</summary> | |||
| IActorRef DeadLetters { get; } | |||
|
|
|||
| IActorRef IgnoreRef { get; } | |||
|
|
||
| namespace Akka.Tests.Actor | ||
| { | ||
| public class ActorRefIgnoreSpec : AkkaSpec, INoImplicitSender |
| } | ||
|
|
||
| [Fact] | ||
| public void ActorSystem_IgnoreRef_should_return_a_serializable_ActorRef_that_can_be_sent_between_two_ActorSystems_using_remote() |
Aaronontheweb
moved this from In progress
to Awaiting original issuer
in Oct 26 - Nov 6 Sprint
This was referenced Nov 16, 2020
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Mostly Sharding allocation / rebalancing changes making use of the newly introduced AppVersion.
Let me know if you want to split it into smaller parts.
Holistic shard allocation strategy, (migrated from Holistic shard allocation strategy, #29543 akka/akka#29555)
with most shards with the one with least shards. Makes the rebalance rather
slow. By default it's only rebalancing 1 shard at a time.
number of shards per region and tries to adjust towards that value.
Picking from all regions with more shards than the optimal.
in one round.
completed.
Cluster aware shard allocation and rolling updates (migrated from Cluster aware shard allocation and rolling updates akka/akka#29548)
Handle when there are no known regions yet (migrated from Handle when there are no known regions yet akka/akka#29712)
ShardRegion ignores HostShard when shutting down (migrated from ShardRegion ignores HostShard when shutting down akka/akka#29707)
Adds ActorRef.Ignore (Migrated from Adds ActorRef.ignore akka/akka#28630)
Allocate rebalanced shards immediately (migrated from Allocate rebalanced shards immediately, #29543 akka/akka#29551)
that triggers allocations
Forward terminated from ShardCoordinator to RebalanceWorker (migrated from Forward terminated from ShardCoordinator to RebalanceWorker akka/akka#29463)
expensive as there is one rebalance worker per shard
ClusterSharding: increasing retry interval for registration call (migrated from ClusterSharding: increasing retry interval for registration call, #25191 akka/akka#26756)
Make a distinction between hand-offs for rebalance and region shutdown (migrated from Make a distinction between hand-offs for rebalance and region shutdown akka/akka#29579)
improve logging of shard region shutdown timeout (migrated from improve logging of shard region shutdown timeout, #29711 akka/akka#29747)