Unit Testing Isolated Model Azure Durable Function Orchestrator in .Net 8

[Kruti-Joshi]

The documentation for Durable Function Testing only talks about the in-proc model - https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-unit-testing

I have a timer-triggered orchestrator as below -

 public class Orchestrator
 {
     private IMapper mapper;
     private IRepository repository;

     public Orchestrator(IMapper mapper, IRepository repository)
     {
         this.mapper = mapper;
         this.repository = repository;
     }

     [Function(nameof(Orchestrator))]
     public async Task RunOrchestrator(
         [OrchestrationTrigger] TaskOrchestrationContext context)
     {
         ILogger logger = context.CreateReplaySafeLogger(nameof(ConnectorOrchestrator));

         IEnumerable<Result> results;

         try
         {
             results = await repository.GetAllResultsAsync();
         }
         catch (Exception ex)
         {
             logger.LogError(ex, $"Error getting results.");
             throw;
         }

         foreach (var result in results)
         {
             try
             {
                 _ = context.CallActivityAsync<string>(nameof(Activity), result);
             }
             catch (Exception ex)
             {
                 logger.LogError(ex, $"Error calling activity.");
                 throw;
             }
         }
     }

     [Function(nameof(Activity))]
     public void ProcessAlerts([ActivityTrigger] Result result, FunctionContext executionContext)
     {

         logger.LogInformation($"Activity started.");

         logger.LogInformation($"Activity completed");
     }

     [Function("Orchestrator_ScheduledStart")]
     public async Task ScheduledStart(
         [TimerTrigger("* */15 * * * *")] TimerInfo timerInfo,
         [DurableClient] DurableTaskClient client,
         FunctionContext executionContext)
     {
         ILogger logger = executionContext.GetLogger("Orchestrator_ScheduledStart");

         string instanceId = await client.ScheduleNewOrchestrationInstanceAsync(
             nameof(ConnectorOrchestrator));

         logger.LogInformation("Started orchestration with ID = '{instanceId}'.", instanceId);
     }
 }

In the below test, I get an error that DurableTaskClient cannot be mocked -

public OrchestratorTests()
{
    mapper = new Mock<IMapper>();
    repository = new Mock<IRepository>();
    durableClient = new Mock<DurableTaskClient>();
    connectorOrchestrator = new ConnectorOrchestrator(mapper.Object, repository.Object);
}

[Fact]
public async Task ScheduledStart_ShouldTriggerOrchestrator()
{
    TimerInfo timerInfo = new TimerInfo();
    Mock<FunctionContext> functionContext = new Mock<FunctionContext>();
    await connectorOrchestrator.ScheduledStart(timerInfo, durableClient.Object, functionContext.Object);

    durableClient.Verify(client => client.ScheduleNewOrchestrationInstanceAsync(nameof(Orchestrator), null, null, default), Times.Once);
}

Is there any way to test isolated durable orchestrators today?

[arnaudleclerc]

This can be achieved with a bit of boilerplate code and some fakes. What I personally did is :

• Create a FakeDurableTaskClient class implementing DurableTaskClient.
• Instantiate a Mock<FakeDurableTaskClient> and pass its Object field (here of type FakeDurableTaskClient) to the method I want to test.

My fake class looks as follows :

public class FakeDurableTaskClient : DurableTaskClient
{
    public FakeDurableTaskClient() : base("fake")
    {
    }

    public override Task<string> ScheduleNewOrchestrationInstanceAsync(TaskName orchestratorName, object input = null, StartOrchestrationOptions options = null,
        CancellationToken cancellation = new())
    {
        return Task.FromResult(options?.InstanceId ?? Guid.NewGuid().ToString());
    }

    public override Task RaiseEventAsync(string instanceId, string eventName, object eventPayload = null, CancellationToken cancellation = new())
    {
        return Task.CompletedTask;
    }

    public override Task<OrchestrationMetadata> WaitForInstanceStartAsync(string instanceId, bool getInputsAndOutputs = false,
        CancellationToken cancellation = new())
    {
        return Task.FromResult(new OrchestrationMetadata(Guid.NewGuid().ToString(), instanceId));
    }

    public override Task<OrchestrationMetadata> WaitForInstanceCompletionAsync(string instanceId, bool getInputsAndOutputs = false,
        CancellationToken cancellation = new())
    {
        return Task.FromResult(new OrchestrationMetadata(Guid.NewGuid().ToString(), instanceId));
    }

    public override Task TerminateInstanceAsync(string instanceId, object output = null, CancellationToken cancellation = new())
    {
        return Task.CompletedTask;
    }

    public override Task SuspendInstanceAsync(string instanceId, string reason = null, CancellationToken cancellation = new())
    {
        return Task.CompletedTask;
    }

    public override Task ResumeInstanceAsync(string instanceId, string reason = null, CancellationToken cancellation = new())
    {
        return Task.CompletedTask;
    }

    public override Task<OrchestrationMetadata> GetInstancesAsync(string instanceId, bool getInputsAndOutputs = false,
        CancellationToken cancellation = new())
    {
        return Task.FromResult(new OrchestrationMetadata(Guid.NewGuid().ToString(), instanceId));
    }

    public override AsyncPageable<OrchestrationMetadata> GetAllInstancesAsync(OrchestrationQuery filter = null)
    {
        return new FakeOrchestrationMetadataAsyncPageable();
    }

    public override Task<PurgeResult> PurgeInstanceAsync(string instanceId, CancellationToken cancellation = new())
    {
        return Task.FromResult(new PurgeResult(1));
    }

    public override Task<PurgeResult> PurgeAllInstancesAsync(PurgeInstancesFilter filter, CancellationToken cancellation = new())
    {
        return Task.FromResult(new PurgeResult(Random.Shared.Next()));
    }

    public override ValueTask DisposeAsync()
    {
        return ValueTask.CompletedTask;
    }
}

I also had to create a fake of AsyncPageable<OrchestrationMetadata> :

internal class FakeOrchestrationMetadataAsyncPageable : AsyncPageable<OrchestrationMetadata>
{
    public override IAsyncEnumerable<Page<OrchestrationMetadata>> AsPages(string continuationToken = null, int? pageSizeHint = null)
    {
        return AsyncEnumerable.Empty<Page<OrchestrationMetadata>>();
    }
}

If I take your example, the following should work using those fakes :

public OrchestratorTests()
{
    mapper = new Mock<IMapper>();
    repository = new Mock<IRepository>();
    durableClient = new Mock<FakeDurableTaskClient>();
    connectorOrchestrator = new ConnectorOrchestrator(mapper.Object, repository.Object);
}

[Fact]
public async Task ScheduledStart_ShouldTriggerOrchestrator()
{
    TimerInfo timerInfo = new TimerInfo();
    Mock<FunctionContext> functionContext = new Mock<FunctionContext>();
    await connectorOrchestrator.ScheduledStart(timerInfo, durableClient.Object, functionContext.Object);

    durableClient.Verify(client => client.ScheduleNewOrchestrationInstanceAsync(nameof(Orchestrator), null, null, default), Times.Once);
}

I cannot guarantee it works with every use-case (I didn't test it with the Entities so far) and it doesn't really feel natural. I would appreciate a more comfortable out-of-the-box solution. But until now, I don't have any better idea.

[Kruti-Joshi]

Thank you. Yes, this does provide a workaround to test some basic functionalities. I should be able to do the same for the activity function as well.

[Kruti-Joshi]

@arnaudleclerc your example was very helpful.
Now I'm trying to implement the same for TaskOrchestrationContext. I created a fake TaskOrchestration context as below -

 public class FakeTaskOrchestrationContext : TaskOrchestrationContext
 {
     public override TaskName Name => new TaskName("AzureActivityFunction");

     public override string InstanceId => "activityInstanceId";

     protected override ILoggerFactory LoggerFactory => new Mock<ILoggerFactory>().Object;

     public override Task<TResult> CallActivityAsync<TResult>(TaskName name, object? input = null, TaskOptions? options = null)
     {
         if (typeof(TResult) == typeof(string))
         {
             return Task.FromResult((TResult)(object)InstanceId);
         }
         else
         {
             return Task.FromResult(default(TResult));
         }
     }

     public override ParentOrchestrationInstance? Parent => throw new NotImplementedException();

     public override DateTime CurrentUtcDateTime => DateTime.UtcNow;

     public override bool IsReplaying => false;

     public override Task<TResult> CallSubOrchestratorAsync<TResult>(TaskName orchestratorName, object? input = null, TaskOptions? options = null)
     {
         throw new NotImplementedException();
     }

     public override void ContinueAsNew(object? newInput = null, bool preserveUnprocessedEvents = true)
     {
         throw new NotImplementedException();
     }

     public override Task CreateTimer(DateTime fireAt, CancellationToken cancellationToken)
     {
         throw new NotImplementedException();
     }

     public override T? GetInput<T>() where T : default
     {
         throw new NotImplementedException();
     }

     public override Guid NewGuid()
     {
         throw new NotImplementedException();
     }

     public override void SendEvent(string instanceId, string eventName, object payload)
     {
         throw new NotImplementedException();
     }

     public override void SetCustomStatus(object? customStatus)
     {
         throw new NotImplementedException();
     }

     public override Task<T> WaitForExternalEvent<T>(string eventName, CancellationToken cancellationToken = default)
     {
         throw new NotImplementedException();
     }
 }

I'm not sure if I need to implement all of the methods, but I just decided to implement what I would be using directly.
In the Unit Test, I'm creating a mock of this fake.

var taskOrchestrationContextMock = new Mock<FakeTaskOrchestrationContext>();
and passing this object.

But when the code reaches
ILogger logger = taskOrchestrationContext.CreateReplaySafeLogger(nameof(OrchestratorFunction));
it throws a null reference exception because it finds that ILoggerFactory, which is internally used in CreateReplaySafeLogger is null, even though I have initialized it in the FakeTaskOrchestrationContext.

Am I doing something wrong there?

[cgillum]

@Kruti-Joshi the null reference exception might be related to how you're mocking the TaskOrchestrationContext.LoggerFactory property. Can you instead try using NullLoggerFactory instead of Mock<ILoggerFactory>().Object?

[Fazer01]

Following :)

[Fazer01]

@Kruti-Joshi
Maybe this is something you are after?
Azure/azure-functions-dotnet-worker#281 (comment)