Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resource optimized placement strategy #8815

Merged
merged 53 commits into from
Jan 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
a50c16a
wip
Jan 9, 2024
804d767
wip
Jan 10, 2024
737c1ba
added neccessary serives to DefaultSiloServices
Jan 10, 2024
621ccf4
wip
Jan 10, 2024
8990407
.
Jan 10, 2024
76dfc7d
fixing some validations
Jan 10, 2024
2cdbb09
reincoorporated physical ram
Jan 11, 2024
ad4b5c2
some tests for options and validator, and XML docs
Jan 11, 2024
c3831d2
renamed file
Jan 11, 2024
e52ff54
made ResourceOptimizedPlacement strategy public in case users want to…
Jan 11, 2024
aceadec
wip
Jan 9, 2024
7c4cd12
wip
Jan 10, 2024
74d8c87
added neccessary serives to DefaultSiloServices
Jan 10, 2024
e825819
wip
Jan 10, 2024
66a31ec
.
Jan 10, 2024
0a7237c
fixing some validations
Jan 10, 2024
cb774bd
reincoorporated physical ram
Jan 11, 2024
2a0622c
some tests for options and validator, and XML docs
Jan 11, 2024
7ee2a95
renamed file
Jan 11, 2024
8f6d36a
made ResourceOptimizedPlacement strategy public in case users want to…
Jan 11, 2024
599046b
Merge branch 'resource-based-strategy' of https://github.com/ledjon-b…
Jan 11, 2024
32bc695
removed IlocalSiloDetails as there is no need for it
Jan 11, 2024
eac44aa
Merge branch 'main' into merging
Jan 11, 2024
8fce9f8
xml docs
Jan 11, 2024
db27b61
again xml docs
Jan 11, 2024
cc863e0
incoorporated support for avoiding overloaded silos by definition of …
Jan 11, 2024
6ecb31d
Apply suggestions from code review
ReubenBond Jan 12, 2024
018443f
resolving some comments from reviewer
Jan 12, 2024
b056cf2
addressing further comments
Jan 12, 2024
84fb563
PR feedback
ReubenBond Jan 12, 2024
32b1a61
PR feedback
ReubenBond Jan 12, 2024
890ef27
Clean up changes to GetBestSiloCandidate_V2
ReubenBond Jan 12, 2024
fdad669
More tidying
ReubenBond Jan 12, 2024
0300d95
Filter inputs per-silo
ReubenBond Jan 12, 2024
66f0380
More cleanup
ReubenBond Jan 12, 2024
f74edfa
fixed comment
Jan 12, 2024
eb312c1
Merge branch 'resource-based-strategy' of https://github.com/ledjon-b…
Jan 12, 2024
f05a584
more comments
Jan 12, 2024
f85d341
clarify comment
ReubenBond Jan 12, 2024
1c64854
comment on the rational behind using a dual-mode KF as opposed to sin…
Jan 12, 2024
5443eef
Merge branch 'resource-based-strategy' of https://github.com/ledjon-b…
Jan 12, 2024
2799824
some more changes
Jan 12, 2024
053de96
PhysicalMemoryWeight
Jan 12, 2024
eafe948
switched to GetBestSiloCandidate v2
Jan 12, 2024
bd04f77
#
Jan 12, 2024
77414bc
switched to non-generic CDM-KF
Jan 12, 2024
1a1c5aa
Normalize weights & fix LocalSiloPreferenceMargin
ReubenBond Jan 13, 2024
407a7ef
added some comments + fixed _localSiloPreferenceMargin not being set …
Jan 13, 2024
28e3068
rearranged CalculateScore code a bit, and added assertion for score >…
Jan 13, 2024
a807d8a
fix potential for a 'DivideByZeroException'
Jan 13, 2024
965e01e
fixed tests + changed Options class to use int instead of float to ma…
Jan 13, 2024
145c38d
perf improvements
Jan 14, 2024
e707edc
added 'in' modifier to CalculateScore to avoid potential defensive co…
Jan 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions src/Orleans.Core.Abstractions/Placement/PlacementAttribute.cs
Original file line number Diff line number Diff line change
Expand Up @@ -99,4 +99,16 @@ public SiloRoleBasedPlacementAttribute() :
base(SiloRoleBasedPlacement.Singleton)
{ }
}

/// <summary>
/// Marks a grain class as using the <see cref="ResourceOptimizedPlacement"/> policy.
/// </summary>
/// <inheritdoc cref="ResourceOptimizedPlacement"/>
[AttributeUsage(AttributeTargets.Class, AllowMultiple = false)]
public sealed class ResourceOptimizedPlacementAttribute : PlacementAttribute
{
public ResourceOptimizedPlacementAttribute() :
base(ResourceOptimizedPlacement.Singleton)
{ }
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
namespace Orleans.Runtime;

/// <summary>
/// A placement strategy which attempts to optimize resource distribution across the cluster.
/// </summary>
/// <remarks>
/// <para>It assigns weights to runtime statistics to prioritize different resources and calculates a normalized score for each silo.
/// Following the <u>power of k-choices</u> algorithm, K silos are picked as potential targets, where K is equal to the square root of the number of silos.
/// Out of those K silos, the one with the lowest score is chosen for placing the activation. Normalization ensures that each property contributes proportionally
/// to the overall score. You can adjust the weights based on your specific requirements and priorities for load balancing.
/// In addition to normalization, an <u>online adaptiv</u> algorithm provides a smoothing effect (filters out high frequency components) and avoids rapid signal
/// drops by transforming it into a polynomial-like decay process. This contributes to avoiding resource saturation on the silos and especially newly joined silos.</para>
/// <para>Silos which are overloaded by definition of the load shedding mechanism are not considered as candidates for new placements.</para>
/// <para><i>This placement strategy is configured by adding the <see cref="Placement.ResourceOptimizedPlacementAttribute"/> attribute to a grain.</i></para>
/// </remarks>
public sealed class ResourceOptimizedPlacement : PlacementStrategy
{
internal static readonly ResourceOptimizedPlacement Singleton = new();
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
using Microsoft.Extensions.Options;

namespace Orleans.Runtime.Configuration.Options;

/// <summary>
/// Settings which regulate the placement of grains across a cluster when using <see cref="ResourceOptimizedPlacement"/>.
/// </summary>
/// <remarks><i>All 'weight' properties, are relative to each other.</i></remarks>
public sealed class ResourceOptimizedPlacementOptions
{
/// <summary>
/// The importance of the CPU usage by the silo.
/// </summary>
/// <remarks><i>
/// <para>A <u>higher</u> value results in the placement favoring silos with <u>lower</u> cpu usage.</para>
/// <para>Valid range is [0-100]</para>
/// </i></remarks>
public int CpuUsageWeight { get; set; } = DEFAULT_CPU_USAGE_WEIGHT;

/// <summary>
ReubenBond marked this conversation as resolved.
Show resolved Hide resolved
/// The default value of <see cref="CpuUsageWeight"/>.
/// </summary>
public const int DEFAULT_CPU_USAGE_WEIGHT = 40;

/// <summary>
/// The importance of the memory usage by the silo.
/// </summary>
/// <remarks><i>
/// <para>A <u>higher</u> value results in the placement favoring silos with <u>lower</u> memory usage.</para>
/// <para>Valid range is [0-100]</para>
/// </i></remarks>
public int MemoryUsageWeight { get; set; } = DEFAULT_MEMORY_USAGE_WEIGHT;

/// <summary>
ReubenBond marked this conversation as resolved.
Show resolved Hide resolved
/// The default value of <see cref="MemoryUsageWeight"/>.
/// </summary>
public const int DEFAULT_MEMORY_USAGE_WEIGHT = 30;

/// <summary>
/// The importance of the available memory to the silo.
/// </summary>
/// <remarks><i>
/// <para>A <u>higher</u> values results in the placement favoring silos with <u>higher</u> available memory.</para>
/// <para>Valid range is [0-100]</para>
/// </i></remarks>
public int AvailableMemoryWeight { get; set; } = DEFAULT_AVAILABLE_MEMORY_WEIGHT;

/// <summary>
ReubenBond marked this conversation as resolved.
Show resolved Hide resolved
/// The default value of <see cref="AvailableMemoryWeight"/>.
/// </summary>
public const int DEFAULT_AVAILABLE_MEMORY_WEIGHT = 20;

/// <summary>
/// The importance of the physical memory to the silo.
/// </summary>
/// <remarks><i>
/// <para>A <u>higher</u> values results in the placement favoring silos with <u>higher</u> physical memory.</para>
/// <para>This may have an impact in clusters with resources distributed unevenly across silos.</para>
/// <para>Valid range is [0-100]</para>
/// </i></remarks>
public int PhysicalMemoryWeight { get; set; } = DEFAULT_PHYSICAL_MEMORY_WEIGHT;

/// <summary>
/// The default value of <see cref="PhysicalMemoryWeight"/>.
/// </summary>
public const int DEFAULT_PHYSICAL_MEMORY_WEIGHT = 10;

/// <summary>
/// The specified margin for which: if two silos (one of them being the local to the current pending activation), have a utilization score that should be considered "the same" within this margin.
/// <list type="bullet">
/// <item>When this value is 0, then the policy will always favor the silo with the lower resource utilization, even if that silo is remote to the current pending activation.</item>
/// <item>When this value is 100, then the policy will always favor the local silo, regardless of its relative utilization score. This policy essentially becomes equivalent to <see cref="PreferLocalPlacement"/>.</item>
ledjon-behluli marked this conversation as resolved.
Show resolved Hide resolved
/// </list>
/// </summary>
/// <remarks><i>
/// <para>Do favor a lower value for this e.g: 5-10</para>
/// <para>Valid range is [0-100]</para>
/// </i></remarks>
public int LocalSiloPreferenceMargin { get; set; } = DEFAULT_LOCAL_SILO_PREFERENCE_MARGIN;

/// <summary>
ReubenBond marked this conversation as resolved.
Show resolved Hide resolved
/// The default value of <see cref="LocalSiloPreferenceMargin"/>.
/// </summary>
public const int DEFAULT_LOCAL_SILO_PREFERENCE_MARGIN = 5;
}

internal sealed class ResourceOptimizedPlacementOptionsValidator
(IOptions<ResourceOptimizedPlacementOptions> options) : IConfigurationValidator
{
private readonly ResourceOptimizedPlacementOptions _options = options.Value;

public void ValidateConfiguration()
{
if (_options.CpuUsageWeight < 0 || _options.CpuUsageWeight > 100)
{
ThrowOutOfRange(nameof(ResourceOptimizedPlacementOptions.CpuUsageWeight));
}

if (_options.MemoryUsageWeight < 0 || _options.MemoryUsageWeight > 100)
{
ThrowOutOfRange(nameof(ResourceOptimizedPlacementOptions.MemoryUsageWeight));
}

if (_options.AvailableMemoryWeight < 0 || _options.AvailableMemoryWeight > 100)
{
ThrowOutOfRange(nameof(ResourceOptimizedPlacementOptions.AvailableMemoryWeight));
}

if (_options.PhysicalMemoryWeight < 0 || _options.PhysicalMemoryWeight > 100)
{
ThrowOutOfRange(nameof(ResourceOptimizedPlacementOptions.PhysicalMemoryWeight));
}

if (_options.LocalSiloPreferenceMargin < 0 || _options.LocalSiloPreferenceMargin > 100)
{
ThrowOutOfRange(nameof(ResourceOptimizedPlacementOptions.LocalSiloPreferenceMargin));
}

static void ThrowOutOfRange(string propertyName)
=> throw new OrleansConfigurationException($"{propertyName} must be inclusive between [0-100]");
}
}
4 changes: 4 additions & 0 deletions src/Orleans.Runtime/Hosting/DefaultSiloServices.cs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
using System.Collections.Generic;
using Microsoft.Extensions.Configuration;
using Orleans.Serialization.Internal;
using Orleans.Runtime.Configuration.Options;

namespace Orleans.Hosting
{
Expand Down Expand Up @@ -190,6 +191,7 @@ internal static void AddDefaultServices(ISiloBuilder builder)

// Placement
services.AddSingleton<IConfigurationValidator, ActivationCountBasedPlacementOptionsValidator>();
services.AddSingleton<IConfigurationValidator, ResourceOptimizedPlacementOptionsValidator>();
services.AddSingleton<PlacementService>();
services.AddSingleton<PlacementStrategyResolver>();
services.AddSingleton<PlacementDirectorResolver>();
Expand All @@ -207,6 +209,7 @@ internal static void AddDefaultServices(ISiloBuilder builder)
services.AddPlacementDirector<HashBasedPlacement, HashBasedPlacementDirector>();
services.AddPlacementDirector<ClientObserversPlacement, ClientObserversPlacementDirector>();
services.AddPlacementDirector<SiloRoleBasedPlacement, SiloRoleBasedPlacementDirector>();
services.AddPlacementDirector<ResourceOptimizedPlacement, ResourceOptimizedPlacementDirector>();

// Versioning
services.TryAddSingleton<VersionSelectorManager>();
Expand Down Expand Up @@ -298,6 +301,7 @@ internal static void AddDefaultServices(ISiloBuilder builder)
services.ConfigureFormatter<ClusterMembershipOptions>();
services.ConfigureFormatter<GrainDirectoryOptions>();
services.ConfigureFormatter<ActivationCountBasedPlacementOptions>();
services.ConfigureFormatter<ResourceOptimizedPlacementOptions>();
services.ConfigureFormatter<GrainCollectionOptions>();
services.ConfigureFormatter<GrainVersioningOptions>();
services.ConfigureFormatter<ConsistentRingOptions>();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

namespace Orleans.Runtime.Placement
{

internal class ActivationCountPlacementDirector : RandomPlacementDirector, ISiloStatisticsChangeListener, IPlacementDirector
{
private class CachedLocalStat
Expand Down
Loading
Loading