Durable Client Factory (DurableOrchestrationClient outside Azure Functions )

[davidrevoledo]

Ref : #161

This is intended to init azure durable functions outside of an azure function orchestrators host.
It could have extensive applications to init durable functions from different applications event from other azure functions applications.

Usability is straight forward for a client. Let's take an Asp.Net Core application.

To configure the application just call AddDurableTask, this will use service collection instead of webjob extension builder.

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
            services.AddDurableTask();
        }

To create the client just inject IDurableClientFactory under the hood this will create a client using the same approach Orchestrator host is doing to avoid breaking changes, we could configure hub name or the connection string for the storage account

private readonly IDurableClientFactory _clientFactory;

        public SampleController(IDurableClientFactory clientFactory)
        {
            _clientFactory = clientFactory;
        }

        [HttpGet]
        public async Task<IActionResult> Get()
        {
            var client = _clientFactory.CreateClient(new DurableClientOptions
            {
                IsExternalClient = true
            });

            var instanceId = await client.StartNewAsync("Chaining");
            return Ok(instanceId);
        }

The implementation on the orchestrator host is exactly the same without any client needed to start the flow.

  [FunctionName("Chaining")]
        public static async Task<object> Chaining(
            [OrchestrationTrigger] IDurableOrchestrationContext context)
        {
            var x = await context.CallActivityAsync<object>("F1", null);
            var y = await context.CallActivityAsync<object>("F2", x);
            var z = await context.CallActivityAsync<object>("F3", y);
            return await context.CallActivityAsync<object>("F4", z);
        }

Let me know your thoughts and will complete a few unit tests required for the external client flag to avoid the validation of know function names.
https://github.com/Azure/azure-functions-durable-extension/pull/1125/files#diff-b4be2ec3bf4e13899ce8445628aa091aR109

Any suggestion to control that validation is welcome.

[davidrevoledo]

It also could be CreateExternalClient and remove the Boolean property and set always true in the attribute to avoid this overhead, I thought it also can be used inside an azure function somewhere not only from external applications.

[cgillum]

I like this idea of having a first-class way to initialize an IDurableClient from an ASP.NET Core web app! I would love to get thoughts from @ConnorMcMahon and @anthonychu on this PR. Personally I would try to simplify a few things and fix a few namespaces, but the high-level approach seems reasonable to me.

It might be useful to see a few more examples, such as the ASP.NET Core setup required to use a non-default task hub name and Azure Storage account.

[anthonychu]

This is great. Thanks!

I wonder if we should publish this as a separate package that references Microsoft.Azure.WebJobs.Extensions.DurableTask and move the ASP.NET specific stuff like public static IServiceCollection AddDurableTask(this IServiceCollection serviceCollection) to it.

[davidrevoledo]

@cgillum ok will do also I think it might be a good idea allow configure IOption to directly configure it in the appsetting.json.

What are you namespace suggestions to clean up the code ? I've only created the factory under ContextImplementations namespace and configuration provider at the same level the webjob configuration provider is, but I can do both changes and create a few mores samples as well.

• Asp.Net Application using non default connections.
• Console App starting a durable function.
• Azure Function starting a durable function hosted in a separated Azure Function Instance.

[davidrevoledo]

@anthonychu great thanks ! :)

you mean something like Microsoft.Extensions.DependencyInjection.DurableExtensions ?

[anthonychu]

Maybe. I haven’t thought this far ahead 🙂 but if we head down this path, we should figure out the right package name to publish under.

…

________________________________ From: David Revoledo <notifications@github.com> Sent: Thursday, December 26, 2019 9:35:30 AM To: Azure/azure-functions-durable-extension <azure-functions-durable-extension@noreply.github.com> Cc: Anthony Chu <Anthony.Chu@microsoft.com>; Mention <mention@noreply.github.com> Subject: Re: [Azure/azure-functions-durable-extension] Durable Client Factory (DurableOrchestrationClient outside Azure Functions ) (#1125) @anthonychu<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fanthonychu&data=02%7C01%7CAnthony.Chu%40microsoft.com%7C7e94c71c1572446729a708d78a2a0153%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637129785322212513&sdata=BBcfaWh4F3F4XCT2hdMilWAZ5PrTv5V5uxC1lNs52rI%3D&reserved=0> great thanks ! :) you mean something like Microsoft.Extensions.DependencyInjection.DurableExtensions ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-functions-durable-extension%2Fpull%2F1125%3Femail_source%3Dnotifications%26email_token%3DAA6MF7LBEWXJA3TCEQEBWVDQ2TTOFA5CNFSM4J7HCFUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHV4EEA%23issuecomment-569098768&data=02%7C01%7CAnthony.Chu%40microsoft.com%7C7e94c71c1572446729a708d78a2a0153%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637129785322212513&sdata=82fcY%2Bnw2EYoNH8Trxho69IdUI%2BjPfGicDXv82ZlQDs%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAA6MF7L3D252EVZWYKWKU6TQ2TTOFANCNFSM4J7HCFUA&data=02%7C01%7CAnthony.Chu%40microsoft.com%7C7e94c71c1572446729a708d78a2a0153%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637129785322222530&sdata=fgx%2BgGx88Ueof59IuieoyAlYNyymsrfk9e2VORsOUsI%3D&reserved=0>.

[davidrevoledo]

Sorry guys I was out for the holiday season of new year, will address this during the week to get a new version more friendly with @cgillum suggestions.

[davidrevoledo]

@cgillum @anthonychu

I've tried to use a connectionstring instead for configuration but to do that we should changes several providers so I still think the best way is keep configure the connection name and read it from configurations.

With changes I made it's easy to configure in the asp.net app and we can call create client w/o params as well have the flexibility to call different clients if any need to reach different durable functions.

 public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);

            services
                .AddDurableTask(options =>
                {
                    options.ConnectionName = "Default";
                    options.TaskHub = "name";
                });
        }

        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseHsts();
            }

            app.UseHttpsRedirection();
            app.UseMvc();
        }
    }

then to use is extremely straight forward.

 [Route("api/[controller]")]
    [ApiController]
    public class ValuesController : ControllerBase
    {
        private readonly IDurableClientFactory _clientFactory;

        public ValuesController(IDurableClientFactory clientFactory)
        {
            _clientFactory = clientFactory;
        }


        [HttpGet]
        public async Task<IActionResult> Get()
        {
            var client = _clientFactory.CreateClient();
            var instanceId = await client.StartNewAsync("Chaining");
            return Ok(instanceId);
        }
    }

and if any one want to reach a specific azure durable function who is not the default configured in the startup (it will be only for complicated and long structures) the code have the flexibility to call like this.

 [HttpGet]
        public async Task<IActionResult> Get()
        {
            var client = _clientFactory.CreateClient(new DurableClientOptions
            {
                ConnectionName = "InvoicingAppStorage",
                TaskHub = "InvoicingHub"
            });
            var instanceId = await client.StartNewAsync("Chaining");
            return Ok(instanceId);
        }

Finally I agree with @anthonychu to move:

• AddDurableTask
• StandardConnectionStringProvider (we can rename it if so)
• DurableClientFactory

outside of this project and create some like Microsoft.Extensions.DependencyInjection.DurableExtensions to be clear that this works using Microsoft DependencyInjection, this make sense because is dangerous using this and manually using the same package inside an azure functions, if so the connection string resolver will be WebJobsConnectionStringProvider and this won't work as excepted, however if we move out this implementation to a different package is clear enough how to use in a console app, asp.net core app or even in an azure function that it'll use Configurations to read the connection.

Let me know your thoughts, I'm very open to do any changes you think it will be necessary and looking forward to collaborate more often in the future :).

[davidrevoledo]

any thoughts ?

[ConnorMcMahon]

@davidrevoledo

Sorry for the delay here. I am going to try to review this in the next day or two.

[ConnorMcMahon]

I gave some initial rounds of feedback. I think we will need to do a bit of refactoring to ensure we don't need to import all of the services, when we really only need a few to properly create the functionality of DurableClient in external/web apps.

[ConnorMcMahon]

We will have to do something similar in SignalEntityAsyncInternal.

[ConnorMcMahon]

@cgillum, is there any reason for us to have this check at all? A DurableClient can be created with a different task hub already, and there is no guarantee that the app that is calling this DurableClient will have the same functions as the app with the TaskHub the client is referencing.

[ConnorMcMahon]

I think this method should be AddDurableTaskFactory(), and honestly might be better served in it's own namespace/package.

[ConnorMcMahon]

As discussed elsewhere in this review, we will keep it in this namespace/package.

There are questions to be brought up about other function apps that don't want to use Durable except for the external client and how we support that, as the functions runtime will automatically start up an instance of the extension, but that can be addressed later.

@bachuv, we may still want to rename this before the v2.4.0 release.

[davidrevoledo]

@ConnorMcMahon thanks for your feedback I'll do the proper changes shortly.

Also do you want I create the separete project to put the IOC and ClientFactory as suggested ? any recommendation for naming it ?

[cgillum]

Personally I'd prefer to not create a separate package. I think the reason to have a separate package is if that package contained additional dependencies that don't make sense in the core Microsoft.Azure.WebJobs.Extensions.DurableTask. It doesn't look like you're adding any new dependencies, so I vote that we keep this all in the same package.

[cgillum]

Hey @davidrevoledo were you still interested in completing this PR? I was hoping to include this in our v2.3.0 release, but we're wrapping that up now and I see there are some merge conflicts that need to be resolved. I'd be happy to include this in our v2.4.0 release if you're up to finishing it.

[davidrevoledo]

sure thing @cgillum , I've been so busy sorry to forget this PR, I've resolved the conflict but let me check if I've changed all the mentioned points in here.

One thing to let you know is that I needed to call durable functions from a asp.net core api so I've been using this for a while, will be great to delete the internal code and having from the library itself :D

[ConnorMcMahon]

@davidrevoledo So sorry for all of the delays getting this work in. We are going to merge in now for our v2.4.0 release, and I know that we will have a lot of very excited customers to use this. We will make sure to give you credit for these new features in our next release.

@bachuv is going to help us drive this to production, with some samples of usage in a console app as well as some tests and validation. Let us know if you are interested in being involved as a reviewer in those PRs!

[davidrevoledo]

Not a problem ! @ConnorMcMahon honoured to contribute, is part of my commitment as an Azure MVP
ok will be ready in case something needs to be addressed @bachuv I was using it for too long (embedded in a project code) however it was the old implementation (from the firsts commits)

Ready here for anything you might need !

[olitomlinson]

Will this also allow me to inject IDurableEntity ?

[ConnorMcMahon]

@olitomlinson, to clarify, do you mean IDurableEntityClient?

This PR does not explicitly support that, but the good news is that the IDurableClient is by definition also a IDurableEntityClient by definition, so you can just cast it as soon as you get the object returned.

[olitomlinson]

Sorry yes, I do mean ‘IDurableEntityClient’ -

I’m having to use the Entity HTTP API from ASP.NET project to signal and read entity state which isn’t terribly easy, so being able to use the durable entity client will be brilliant!

[mronnblom]

This is awesome! 🥇 However, what is still missing is documenting this so people know it's possible. I only noticed after a very specific web search and finding this old PR. I would suggest documenting it under the section "Orchestration client" here:
https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-bindings?tabs=csharp%2C2x-durable-functions#orchestration-client