PowerShell - parallel processing / scaling without concurrency

[JulianHayward]

Hi,
Trying with the default durable functions templates (from the portal) to achieve parallel processing/scaling as per article.
In the fan out/fan in pattern, you execute multiple functions in parallel and then wait for all functions to finish.

However it seems that the Activity Function is queuing up..

Orchestrator Function:

run.ps1
param($Context) $output = @() $inputs = @('tokyo','london','berlin') $ParallelTasks = foreach ($input in $inputs) { Invoke-ActivityFunction -FunctionName 'Hello2' -Input $input -NoWait } $output = Wait-ActivityFunction -Task $ParallelTasks $output

function.json
{ "bindings": [ { "name": "Context", "type": "orchestrationTrigger", "direction": "in" } ] }

Activity Function (Hello2):
run.ps1
param($name) write-host "$name .. sleep" start-sleep -seconds 10 "Hello $name!"

Function.json
{ "bindings": [ { "name": "name", "type": "activityTrigger", "direction": "in" } ] }

Activity Output:

2021-03-11T21:50:34.303 [Information] Executing 'Functions.Hello2' (Reason='(null)', Id=6f5ab392-4818-417b-8515-12f2b69218cb)
2021-03-11T21:50:34.303 [Information] Executing 'Functions.Hello2' (Reason='(null)', Id=e1580f91-129b-44f6-a0da-28cb2c786560)
2021-03-11T21:50:34.306 [Warning] Your function 'Hello2' is queuing requests as there are no available runspaces. You may be able to increase your throughput by following the best practices on https://aka.ms/functions-powershell-concurrency.
2021-03-11T21:50:34.308 [Information] Executing 'Functions.Hello2' (Reason='(null)', Id=61afd729-3415-496a-b6fb-8a7df3d39a60)
2021-03-11T21:50:34.310 [Information] INFORMATION: berlin .. sleep
2021-03-11T21:50:44.311 [Information] OUTPUT: Hello berlin!
2021-03-11T21:50:44.317 [Information] Executed 'Functions.Hello2' (Succeeded, Id=e1580f91-129b-44f6-a0da-28cb2c786560, Duration=10014ms)
2021-03-11T21:50:44.325 [Warning] Your function 'Hello2' is queuing requests as there are no available runspaces. You may be able to increase your throughput by following the best practices on https://aka.ms/functions-powershell-concurrency.
2021-03-11T21:50:44.326 [Information] INFORMATION: tokyo .. sleep
2021-03-11T21:50:54.339 [Information] OUTPUT: Hello tokyo!
2021-03-11T21:50:54.347 [Information] Executed 'Functions.Hello2' (Succeeded, Id=6f5ab392-4818-417b-8515-12f2b69218cb, Duration=20047ms)
2021-03-11T21:50:54.349 [Information] INFORMATION: london .. sleep
2021-03-11T21:51:04.354 [Information] OUTPUT: Hello london!
2021-03-11T21:51:04.355 [Information] Executed 'Functions.Hello2' (Succeeded, Id=61afd729-3415-496a-b6fb-8a7df3d39a60, Duration=30047ms)

Concurrency
Depending on your use case, Durable Functions may significantly improve scalability. To learn more, see Durable Functions application patterns. (back to link article)

[AnatoliB]

The short answer is: this behavior is expected by design; if you want your activities to run in parallel, you also need to enable concurrency. We do not promise parallel processing without concurrency, even when using the Fan-out/Fan-in pattern.

In this case, how do Durable Functions help scalability, and what is the point of the Fan-out/Fan-in pattern? These are excellent questions, thank you.

There are two layers here:

• The programming model layer. This is where we write our code and assume certain rules on how this code may be executed.
• The execution layer. This layer contains actual compute resources (CPUs, processes, runspaces, threads, etc.), and it tries to optimize work distribution across the allocated resources, scale out if possible, etc. Depending on multiple factors, this layer may decide to execute some code in parallel, but it is not allowed to ever break the programming model assumptions.

It is important to understand the difference between these two layers.

When we author a regular (non-durable) PowerShell function that does not manipulate runspaces, jobs, processes, etc., the following constraints are implied, whether we consciously consider them or not:

1. The entire function must be executed in the context of a single dedicated PowerShell runspace, by a single process, running on a single machine. This is what the PowerShell programming model demands for any PowerShell script, and this is applicable to any PowerShell function. There is no opportunity for concurrency or scaling within a single function invocation. Multiple function invocations can indeed be parallelized (if enough resources are available on the execution layer), but any given function invocation cannot be distributed, regardless of the available resources.
2. All the PowerShell commands must be executed in a certain deterministic order. This order is strictly defined by the PowerShell language. Before the next command starts executing, the previous command execution must complete.

These two constraints significantly limit parallelization and scale out opportunities on the execution layer. Regardless of the number of workers available, exactly one runspace on exactly one worker can be allocated for any given function invocation. This runspace cannot be reused for any other invocation until the previous one completes. Adding more workers will not make any given execution faster, as this function execution cannot be distributed. Also, at any moment, the number of function invocations in progress cannot exceed the number of runspaces across all the worker processes and machines. It does not matter how sophisticated and powerful the execution layer is: the programming model constraints must be respected.

The Durable Functions programming model allows the function author lift these constraints.

First of all, the Function Chaining pattern relaxes the first constraint. By calling Invoke-FunctionActivity without the -NoWait switch, the function author effectively says: "I don’t insist on these chunks of work being executed on the same runspace. Go ahead and optimize when this helps. I still require them to be executed in this order, though." As a result, the execution layer may (but does not have to!) distribute these smaller chunks of work across multiple runspaces, processes, or even machines. Also, every worker can interleave executing activities that belong to different orchestrations, so two or multiple orchestrations can keep making progress.

However, the execution order constraint is still present. When the function author calls Invoke-FunctionActivity with the -NoWait switch (see the Fan-out/Fan-in pattern), this also means: "I don't care about the execution order. Go ahead and optimize by executing these activities in any order, potentially even in parallel if you find this possible and beneficial." As a result, the execution layer will consider the available resources (the number of allocated runspaces, processes, and worker machines) and distribute the work, potentially even scaling out by allocating more resources, and it may (again, it does not have to!) run some activities in parallel. The effective level of parallelism still depends on many factors on the execution layer. But the initial constraint imposed by the programming model is lifted.

So, as you can see, the level of parallelism depends on both the programming model and the execution layer. On the programming model side, make sure you don't introduce unnecessary ordering constraints and use Invoke-FunctionActivity with the -NoWait switch as appropriate. On the execution layer side, pay attention to the concurrency and scaling configurations. For example, if a single worker process with a single PowerShell runspace is all you have on the execution layer, do not expect any activities to run in parallel. Consider increasing the number of runspaces per worker process (PSWorkerInProcConcurrencyUpperBound), the number of worker processes within the same instance (FUNCTIONS_WORKER_PROCESS_COUNT), or even scaling out to multiple instances.

I hope this helps.

[AnatoliB]

Closing this issue as I believe the question is answered. Please feel free to continue the discussion here if you have more questions or concerns on this topic.