Question: MS Orleans support?

[VincentH-Net]

Is there any Aspire support/guidance planned for Microsoft Orleans?

• Orleans is a natural fit in the cloud native space - especially for C# devs
• OpenTelemetry observability is already built in

If not, any advice / pointers on how to integrate it?

Example scenario

E.g. a scenario that I will be implementing in the coming months is running Orleans in Azure Container Apps (auto-scale) with both REST endpoints (minimal API's) and SignalR endpoints (BFF for WASM browser and native mobile clients, built in C# with Uno Platform)

I'm looking for ways to make end2end observability easier to achieve and maintain:
Rest API -> Orleans grains / streams chain <-> SignalR <-> client
And I'm wondering if Aspire could help with that.

Any insights appreciated! (@ReubenBond ?)

[ReubenBond]

Hi @VincentH-Net! We've discussed this and we agree that Orleans is a natural fit here. I think we could have two main pieces:

• An Aspire AppHost Orleans cluster resource which consists of:
  • Default cluster, Grain persistence, & Reminder storage (we can pick Azure Table Storage for clustering & reminders, and Azure Blob Storage for grain state)
  • ClusterId / ServiceId values, and any other common configuration
  • The ability to add this resource to the service projects (i.e, where the Orleans silos & clients live)
• An Aspire Orleans Component which pulls in the requisite packages (Eg, the Azure Storage providers) and uses IConfiguration (specified by the AppHost component in the dev case) to configure them.
  • It should also configure Open Telemetry integration, as you mentioned

@benjaminpetit is looking into this. Your input is greatly appreciated.

[benjaminpetit]

Hi @VincentH-Net ,

Quickly discussed with @ReubenBond, and I think for storage we should provision three different storage instances:

• one for clustering
• one for reminders
• one for grain states

For local dev we could start an azurite docker instance. Or even maybe target some real storage if they are available?

[VincentH-Net]

Thanks @ReubenBond, that sounds promising!

I'll follow up with @benjaminpetit - he already mailed me.

[davidfowl]

@benjaminpetit Why 3?

[benjaminpetit]

If I am not mistaken, it doesn't cost more, but you benefit isolation from each other:

• reminders can do a lot of requests
• same for storage
• clustering doesn't, but it's better if it doesn't get throttled because of reminder or storage.

Plus, we could do premium blob storage for grain storage, cosmosdb for reminders, and simple table storage for clustering for example.

[VincentH-Net]

@benjaminpetit quick questions:

• perhaps offer very lightweight and simple option(s) for local dev, e.g. in-memory or SqLite? Devs usually process very little data on local machines
• is blob storage also a good option for many grains with small state size, or would table storage be better then? If yes, offer the dev a simple way to choose?
• why would you need cosmosdb? It is powerful but also potentially complex & expensive - perhaps not a good fit when trying to make things simpler for devs...

[ReubenBond]

is blob storage also a good option for many grains with small state size, or would table storage be better then? If yes, offer the dev a simple way to choose?

Blobs are a good default, especially because they don't have the 1MB limit which table rows do and they have lower overhead (no splitting / joining columns, less encoding), plus there are more price/performance/durability options for blob storage than with tables. The two most appropriate options for grain storage in Azure are Blobs & Cosmos DB, in my opinion.

[danmoseley]

dotnet/aspire-samples#20

[aL3891]

i'd also love orleans support,
for local dev though i'd prefer to only use in memory storage (at least as an option), but maybe you're talking about the opinionated defaults for deploying?

we do in memory clustering when running locally but also do reminders in the local postgres database, (and dont use grain persistence at all) so being able to configure them individually would be great, even if there is also a kitchen-and-sink option that configures everything

[ReubenBond]

@aL3891 understood. I imagine we will offer a set of default choices as well as a way to configure providers in a more finer-grained manner.

[ReubenBond]

Small update, here's the branch for @benjaminpetit's WIP: https://github.com/dotnet/aspire/tree/rebond/bpetit/orleans-hosting

Here's a usage example showing adding Azure Table Storage for clustering and Azure Blob Storage as the "default" grain state store:

var builder = DistributedApplication.CreateBuilder(args);

builder.AddAzureProvisioning();

var storage = builder.AddAzureStorage("storage");
var clusteringTable = storage.AddTables("clustering");
var grainStorage = storage.AddBlobs("grainstate");

var orleans = builder.AddOrleans("my-app")
                     .WithClustering(clusteringTable)
                     .WithGrainStorage("default", grainStorage);

builder.AddProject<Projects.OrleansServer>("silo")
       .WithOrleansServer(orleans)
       .WithReplicas(3);

using var app = builder.Build();

await app.RunAsync();

The pertinent line in the corresponding Orleans server project is builder.UseOrleansAspire(), as you can see in the sample, here:

aspire/samples/orleans/OrleansServer/Program.cs

Line 7 in aaa5d25

builder.UseOrleansAspire();

[VincentH-Net]

@ReubenBond I succeeded in getting OrleansAppHost from https://github.com/dotnet/aspire/tree/rebond/bpetit/orleans-hosting to run. It took some puzzling effort to get Azure rights correct though.

The only change I had to make was in appsettings.json in the OrleansAppHost project, to update the Azure SubscriptionId.

However, I also had to assign the managed identity of the azure VM I was developing on the Storage Blob Data Contributor and Storage Table Data Contributor roles for the generated storage account.

Without those roles, host.StartAsync() throws:

Orleans.Runtime.OrleansException: 'Exception trying to create or connect to the Azure table OrleansSiloInstances'
RequestFailedException: This request is not authorized to perform this operation using this permission.
Status: 403 (Forbidden)
ErrorCode: AuthorizationPermissionMismatch

-> Perhaps good to add some guidance (readme in the sample source?) on this to help devs getting things working quickly?

[VincentH-Net]

Great to see telemetry across the entire chain, including the grain calls!

[VincentH-Net]

@ReubenBond What I would love to see:

• Default options for local development that use the most lightweight Orleans providers (in memory) to have the fastest and least complicated local dev loop possible (no Azure, no Azurite, no Docker, avoid rights issues), but use sensible Azure defaults (that you mentioned earlier) when deploying to other environments
• As we discussed on X, include web - ideally an ASP.NET minimal API example that runs co-hosted with Orleans, with multiple instances within the same Orleans cluster. That will be a sweet spot and a great Orleans onramp for devs who want to scale/distribute an API.

Overall Aspire with Orleans feels like a really attractive combination for the C# developer👍

[davidfowl]

However, I also had to assign the managed identity of the azure VM I was developing on the Storage Blob Data Contributor and Storage Table Data Contributor roles for the generated storage account.

this is a bug in preview1, fixed in preview2

[VincentH-Net]

@ReubenBond I noticed something 'off' in AddKeyedAzureTableService definition versus usage in https://github.com/dotnet/aspire/tree/rebond/bpetit/orleans-hosting:

The Orleans branch adds below overload, which adds the connectionName parameter:

    public static void AddKeyedAzureTableService(
        this IHostApplicationBuilder builder,
        string connectionName,
        string name,
        Action<AzureDataTablesSettings>? configureSettings = null,
        Action<IAzureClientBuilder<TableServiceClient, TableClientOptions>>? configureClientBuilder = null)

However, this overload is not referenced. The code in AspireOrleansServerExtensions invokes AddKeyedAzureTableService like this in 3 placees:

builder.AddKeyedAzureTableService(connectionName);

Which matches to the existing overload:

    public static void AddKeyedAzureTableService(
        this IHostApplicationBuilder builder,
        string name,
        Action<AzureDataTablesSettings>? configureSettings = null,
        Action<IAzureClientBuilder<TableServiceClient, TableClientOptions>>? configureClientBuilder = null)

So the passed in connectionName is used as name - does not seem intentional?

[VincentH-Net]

(how / when) can I use Orleans with public NuGet Aspire bits? If not possible today, any ETA? Thanks!

I'm trying to get Aspire for Orleans co-hosted with a minimal web API to work.

Approach 1

I tried to use the new Aspire.Hosting.Orleans and Aspire.Orleans.Server from https://github.com/dotnet/aspire/tree/rebond/bpetit/orleans-hosting with the existing preview 1 NuGets (just replaced the project references in those 2 Aspire projects with the latest NuGet.org package references, and then referenced the project assemblies and the NuGet packages in a regular Aspire preview 1 project that already contained a web api)

I got it all building without warnings, however when I do azd init (version 1.5.0) it fails with:

ERROR: generating bicep from manifest: unsupported resource type: azure.storage.table.v0

Updating azd to latest daily build (1.6.0 beta) like this issue comment suggests gives the same error.

Approach 2

I then tried to upgrade the Orleans example in the Aspire repo to add a WEB API with Rest and SignalR.
This works locally - I can run and debug on localhost, and have 3 silo's / API endpoints that form one Orleans cluster, where each endpoint hosts a minimal API and a SignalR hub. All work as expected.

However, when I try azd init in the orleans example folder I get:

ERROR: generating bicep from manifest:

Is deploying Orleans to ACA supported at this point?

[zbarrier]

I would like to see the addition of LogConsistencyStorage/JournaledGrain providers too.

In the hosting project, clustering and grain storage resources are tightly coupled to Azure resources. Are there any plans to modify this? What if I want to use CosmosDB or some custom storage provider? Currently, I get around this by adding the projects directly to my solution and modify it to accommodate my defaults.

[davidfowl]

Updating azd to latest daily build (1.6.0 beta) like this issue comment suggests gives the same error.

I would put that comment on the azd issue.

[VincentH-Net]

I would put that comment on the azd issue.

done

[VincentH-Net]

Aspire support for Orleans is now added (requires Orleans nightly feed packages for now), including an example - I tested it, defects were fixed and all works well.

Thanks for this @benjaminpetit @ReubenBond and all who chimed in!
NJoy all!

[davidfowl]

Is that sample deployable to azure using azd?

[VincentH-Net]

Is that sample deployable to azure using azd?

@davidfowl Yes, if you replace the Aspire project references in the sample with the corresponding NuGet package references, and you comment out this line in the example source as indicated in the comment above that line.

Proof:

Note that if you try to deploy the Orleans playground to ACA without above modifications, AZD does not detect Aspire and fails like below:

Btw none of the other playgrounds are deployable to ACA as-is atm (they fail with different errors though).

[davidfowl]

@mitchdenny I think we need to exclude the manifest dashboard project from the manifest automagically as well so this just works.

aspire/playground/orleans/OrleansAppHost/Program.cs

Lines 5 to 12 in 104cbd4

	if (!args.Contains("--publisher")) // AZD UP passes in --publisher manifest
	{
	storage.UseEmulator();
	}
	else
	{
	builder.AddAzureProvisioning();
	}

You should be able to remove this branch. AzureProvisioning only applies when there's no publisher and UseEmulator isn't effective when publishing the manifest.

[VincentH-Net]

You should be able to remove this branch. AzureProvisioning only applies when there's no publisher and UseEmulator isn't effective when publishing the manifest.

@davidfowl As an Orleans developer I never use the emulator. As a dev I code against the Orleans abstraction, and Orleans has providers for azure and in-memory. Locally I am not testing the Orleans providers, just my app code. So it makes more sense to me to use the most lightweight Orleans providers for the local dev loop.

The reason I am bringing this up is that in the common case, an Orleans dev does need to distinguish between local run and azure deployment. For me the most representative example would read like below. Imo this would be more representative of what most Orleans - Aspire devs will want:

var builder = DistributedApplication.CreateBuilder(args);

var orleans = builder.AddOrleans("my-app");

if (!args.Contains("--publisher")) // Local run; see https://github.com/dotnet/aspire/issues/1823
{
    orleans = orleans
        .WithDevelopmentClustering()
        .WithMemoryGrainStorage("Default");
}
else // Azure deployment
{
    var storage = builder.AddAzureStorage("storage");
    var clusteringTable = storage.AddTables("clustering");
    var grainStorage = storage.AddBlobs("grainstate");

    orleans = orleans
        .WithClustering(clusteringTable)
        .WithGrainStorage("Default", grainStorage);
}

builder.AddProject<Projects.OrleansServer>("silo")
       .WithReference(orleans);

using var app = builder.Build();

await app.RunAsync();

[davidfowl]

This is something we are going to add support for. In this particular sample it's unnecessary.

[VincentH-Net]

True. My concern is guidance for devs today, until the new API is there.

I agree it makes sense to either remove the branch in the current example or modify the example to something like above. Since the current way to do it is a bit of an ugly wart, I can understand wanting to avoid that in the example ;-)

If we keep the example as-is (minus the branch) I will point devs to #1823 which shows the current workaround and allows to track the new API

[VincentH-Net]

@davidfowl I created a PR for the Orleans playground to address your comments:
#1904