Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/designs/PowerShell-AzF-Overall-Design.md
Original file line number Diff line number Diff line change
Expand Up @@ -551,17 +551,17 @@ We had a prototype of Durable Functions in PowerShell worker to enable the '_Fun
2. Make the orchestrator function able to be stopped at certain point safely.
3. Make the PowerShell worker able to replay an orchestrator function, namely skipping the actions that are done in previous runs/replays based on the save logs.

They were solved by introducing the new cmdlet "_Invoke-ActivityFunction_" plus having the worker invoke the orchestrator function with the async PowerShell API.
They were solved by introducing the new cmdlet "_Invoke-DurableActivity_" plus having the worker invoke the orchestrator function with the async PowerShell API.

Internally, the worker shares context information with the cmdlet, including _the existing saved logs_ sent from the host about the running orchestrator function and _a wait handler_ - let's call it A - which the cmdlet will set after it triggers an activity function. The async API used to start the orchestrator function returns _another wait handler_ - let's call it B - which will be set when the invocation finishes. Then the invoking thread will call '_WaitHandler.WaitAny_' on those two wait handlers.

- If the call to '_WaitAny_' returns because the wait handler A was set, then that means an activity function was just triggered, and the orchestrator function should be stopped now (it will be triggered again later after the activity function finishes). So, in this case, the invoking thread will stop the orchestrator function that is running asynchronously.
- If the call to '_WaitAny_' returns because the wait handler B was set, then that means the orchestrator function has run to its completion.

The cmdlet '_Invoke-ActivityFunction_' has the following syntax, the '_-FunctionName_' being the name of the activity function to invoke and '_-Input_' being the argument to the activity function.
The cmdlet '_Invoke-DurableActivity_' has the following syntax, the '_-FunctionName_' being the name of the activity function to invoke and '_-Input_' being the argument to the activity function.

```powershell
Invoke-ActivityFunction [-FunctionName] <string> [[-Input] <Object>] [<CommonParameters>]
Invoke-DurableActivity [-FunctionName] <string> [[-Input] <Object>] [<CommonParameters>]
```

When it's invoked to trigger an activity function, it first checks the existing logs shared by the worker to see if this invocation of the activity function has already done previously. If so, the cmdlet simply returns the result. If not, the cmdlet will
Expand All @@ -570,7 +570,7 @@ When it's invoked to trigger an activity function, it first checks the existing
- set the wait handler shared by the worker to notify the worker that the activity function is triggered;
- wait on a private wait handler that will only be set when the '_StopProcessing_' method of the cmdlet is called. That method gets called only when the pipeline where this cmdlet is running in is being stopped.

The third step is very important in this stop-and-replay model of Durable Functions, because when stopping an invocation that is running asynchronously, we don't want that to interrupt arbitrary code execution that is happening in the pipeline. By having the cmdlet '_Invoke-ActivityFunction_' wait for '_StopProcessing_' to be called, we know for sure that the pipeline execution pauses at a safe place, ready for being stopped by the invoking thread.
The third step is very important in this stop-and-replay model of Durable Functions, because when stopping an invocation that is running asynchronously, we don't want that to interrupt arbitrary code execution that is happening in the pipeline. By having the cmdlet '_Invoke-DurableActivity_' wait for '_StopProcessing_' to be called, we know for sure that the pipeline execution pauses at a safe place, ready for being stopped by the invoking thread.

The following is an example of PowerShell Durable Function that runs in the Function Chaining pattern:

Expand All @@ -583,9 +583,9 @@ param($context)

$output = @()

$output += Invoke-ActivityFunction -FunctionName "E1_SayHello" -Input "Tokyo"
$output += Invoke-ActivityFunction -FunctionName "E1_SayHello" -Input "Seattle"
$output += Invoke-ActivityFunction -FunctionName "E1_SayHello" -Input "London"
$output += Invoke-DurableActivity -FunctionName "E1_SayHello" -Input "Tokyo"
$output += Invoke-DurableActivity -FunctionName "E1_SayHello" -Input "Seattle"
$output += Invoke-DurableActivity -FunctionName "E1_SayHello" -Input "London"

return $output
```
Expand Down
6 changes: 3 additions & 3 deletions examples/durable/DurableApp/CustomStatusOrchestrator/run.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@ Write-Host 'CustomStatusOrchestrator: started.'
$output = @()

Set-DurableCustomStatus -CustomStatus 'Processing Tokyo'
$output += Invoke-ActivityFunction -FunctionName 'SayHello' -Input 'Tokyo'
$output += Invoke-DurableActivity -FunctionName 'SayHello' -Input 'Tokyo'

Set-DurableCustomStatus -CustomStatus @{ ProgressMessage = 'Processing Seattle'; Stage = 2 }
$output += Invoke-ActivityFunction -FunctionName 'SayHello' -Input 'Seattle'
$output += Invoke-DurableActivity -FunctionName 'SayHello' -Input 'Seattle'

Set-DurableCustomStatus -CustomStatus @('Processing London', 'Last stage')
$output += Invoke-ActivityFunction -FunctionName 'SayHello' -Input 'London'
$output += Invoke-DurableActivity -FunctionName 'SayHello' -Input 'London'

Set-DurableCustomStatus 'Processing completed'

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Write-Host 'FanOutFanInOrchestrator: started.'

$parallelTasks =
foreach ($Name in 'Tokyo', 'Seattle', 'London') {
Invoke-ActivityFunction -FunctionName 'SayHello' -Input $Name -NoWait
Invoke-DurableActivity -FunctionName 'SayHello' -Input $Name -NoWait
}

$output = Wait-DurableTask -Task $parallelTasks
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ Write-Host 'FunctionChainingOrchestrator: started.'

$output = @()

$output += Invoke-ActivityFunction -FunctionName 'SayHello' -Input 'Tokyo'
$output += Invoke-ActivityFunction -FunctionName 'SayHello' -Input 'Seattle'
$output += Invoke-ActivityFunction -FunctionName 'SayHello' -Input 'London'
$output += Invoke-DurableActivity -FunctionName 'SayHello' -Input 'Tokyo'
$output += Invoke-DurableActivity -FunctionName 'SayHello' -Input 'Seattle'
$output += Invoke-DurableActivity -FunctionName 'SayHello' -Input 'London'

Write-Host 'FunctionChainingOrchestrator: finished.'

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ $retryOptions = New-DurableRetryOptions `
-FirstRetryInterval (New-Timespan -Seconds 1) `
-MaxNumberOfAttempts 7

$output += Invoke-ActivityFunction -FunctionName 'FlakyActivity' -Input 'Tokyo' -RetryOptions $retryOptions
$output += Invoke-ActivityFunction -FunctionName 'FlakyActivity' -Input 'Seattle' -RetryOptions $retryOptions
$output += Invoke-ActivityFunction -FunctionName 'FlakyActivity' -Input 'London' -RetryOptions $retryOptions
$output += Invoke-DurableActivity -FunctionName 'FlakyActivity' -Input 'Tokyo' -RetryOptions $retryOptions
$output += Invoke-DurableActivity -FunctionName 'FlakyActivity' -Input 'Seattle' -RetryOptions $retryOptions
$output += Invoke-DurableActivity -FunctionName 'FlakyActivity' -Input 'London' -RetryOptions $retryOptions

$output
4 changes: 2 additions & 2 deletions examples/durable/DurableApp/HttpStart/run.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ param($Request, $TriggerMetadata)

Write-Host 'HttpStart started'

$InstanceId = Start-NewOrchestration -FunctionName $Request.Params.FunctionName -InputObject $Request.Query.Input
$InstanceId = Start-DurableOrchestration -FunctionName $Request.Params.FunctionName -InputObject $Request.Query.Input
Write-Host "Started orchestration $($Request.Params.FunctionName) with ID = '$InstanceId'"

$Response = New-OrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
$Response = New-DurableOrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
Push-OutputBinding -Name Response -Value $Response

Write-Host 'HttpStart completed'
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ $duration = New-TimeSpan -Seconds $Context.Input.Duration
$managerId = $Context.Input.ManagerId
$skipManagerId = $Context.Input.SkipManagerId

$output += Invoke-ActivityFunction -FunctionName "RequestApproval" -Input $managerId
$output += Invoke-DurableActivity -FunctionName "RequestApproval" -Input $managerId

$durableTimeoutEvent = Start-DurableTimer -Duration $duration -NoWait
$approvalEvent = Start-DurableExternalEventListener -EventName "ApprovalEvent" -NoWait
Expand All @@ -19,10 +19,10 @@ $firstEvent = Wait-DurableTask -Task @($approvalEvent, $durableTimeoutEvent) -An

if ($approvalEvent -eq $firstEvent) {
Stop-DurableTimerTask -Task $durableTimeoutEvent
$output += Invoke-ActivityFunction -FunctionName "ProcessApproval" -Input $approvalEvent
$output += Invoke-DurableActivity -FunctionName "ProcessApproval" -Input $approvalEvent
}
else {
$output += Invoke-ActivityFunction -FunctionName "EscalateApproval" -Input $skipManagerId
$output += Invoke-DurableActivity -FunctionName "EscalateApproval" -Input $skipManagerId
}

Write-Host 'HumanInteractionOrchestrator: finished.'
Expand Down
4 changes: 2 additions & 2 deletions examples/durable/DurableApp/HumanInteractionStart/run.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ Write-Host 'HumanInteractionStart started'

$OrchestratorInputs = @{ Duration = 45; ManagerId = 1; SkipManagerId = 2 }

$InstanceId = Start-NewOrchestration -FunctionName 'HumanInteractionOrchestrator' -InputObject $OrchestratorInputs
$InstanceId = Start-DurableOrchestration -FunctionName 'HumanInteractionOrchestrator' -InputObject $OrchestratorInputs
Write-Host "Started orchestration with ID = '$InstanceId'"

$Response = New-OrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
$Response = New-DurableOrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
Push-OutputBinding -Name Response -Value $Response

Write-Host 'HumanInteractionStart completed'
4 changes: 2 additions & 2 deletions examples/durable/DurableApp/MonitorOrchestrator/run.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ $pollingInterval = New-TimeSpan -Seconds $Context.Input.PollingInterval
$expiryTime = $Context.Input.ExpiryTime

while ($Context.CurrentUtcDateTime -lt $expiryTime) {
$jobStatus = Invoke-ActivityFunction -FunctionName 'GetJobStatus' -Input $jobId
$jobStatus = Invoke-DurableActivity -FunctionName 'GetJobStatus' -Input $jobId
if ($jobStatus -eq "Completed") {
# Perform an action when a condition is met.
$output += Invoke-ActivityFunction -FunctionName 'SendAlert' -Input $machineId
$output += Invoke-DurableActivity -FunctionName 'SendAlert' -Input $machineId
break
}

Expand Down
4 changes: 2 additions & 2 deletions examples/durable/DurableApp/MonitorStart/run.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ Write-Host 'MonitorStart started'

$OrchestratorInputs = @{ JobId = 1; MachineId = 1; PollingInterval = 10; ExpiryTime = (Get-Date).ToUniversalTime().AddSeconds(60) }

$InstanceId = Start-NewOrchestration -FunctionName 'MonitorOrchestrator' -InputObject $OrchestratorInputs
$InstanceId = Start-DurableOrchestration -FunctionName 'MonitorOrchestrator' -InputObject $OrchestratorInputs
Write-Host "Started orchestration with ID = '$InstanceId'"

$Response = New-OrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
$Response = New-DurableOrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
Push-OutputBinding -Name Response -Value $Response

Write-Host 'MonitorStart completed'
4 changes: 2 additions & 2 deletions examples/durable/LongRunningHttpApp/HttpTrigger/run.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ param($Request, $TriggerMetadata)

Write-Host "HttpTrigger started"

$InstanceId = Start-NewOrchestration -FunctionName 'MyOrchestrator' -InputObject $Request.Query
$InstanceId = Start-DurableOrchestration -FunctionName 'MyOrchestrator' -InputObject $Request.Query
Write-Host "Started orchestration with ID = '$InstanceId'"

$Response = New-OrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
$Response = New-DurableOrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
Push-OutputBinding -Name Response -Value $Response

Write-Host "HttpTrigger completed"
2 changes: 1 addition & 1 deletion examples/durable/LongRunningHttpApp/MyOrchestrator/run.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ param($Context)

Write-Host "MyOrchestrator: started. Input: $($Context.Input)"

$activityResult = Invoke-ActivityFunction -FunctionName "LongRunningActivity" -Input $Context.Input
$activityResult = Invoke-DurableActivity -FunctionName "LongRunningActivity" -Input $Context.Input
Write-Host "MyOrchestrator: Returned from LongRunningActivity: '$activityResult'"

Write-Host "MyOrchestrator: finished."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ namespace Microsoft.Azure.Functions.PowerShellWorker.Durable.Commands
using Microsoft.Azure.Functions.PowerShellWorker.Durable.Tasks;

/// <summary>
/// Invoke an activity function.
/// Invoke a durable activity.
/// </summary>
[Cmdlet("Invoke", "ActivityFunction")]
public class InvokeActivityFunctionCommand : PSCmdlet
[Cmdlet("Invoke", "DurableActivity")]
public class InvokeDurableActivityCommand : PSCmdlet
{
/// <summary>
/// Gets and sets the activity function name.
Expand Down
2 changes: 1 addition & 1 deletion src/Durable/DurableController.cs
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ internal DurableController(
public void BeforeFunctionInvocation(IList<ParameterBinding> inputData)
{
// If the function is an orchestration client, then we set the DurableClient
// in the module context for the 'Start-NewOrchestration' function to use.
// in the module context for the 'Start-DurableOrchestration' function to use.
if (_durableFunctionInfo.IsDurableClient)
{
var durableClient =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,14 @@ NestedModules = @('Microsoft.Azure.Functions.PowerShellWorker.psm1', 'Microsoft.
# Functions to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no functions to export.
FunctionsToExport = @(
'New-DurableRetryOptions',
'New-OrchestrationCheckStatusResponse',
'New-DurableOrchestrationCheckStatusResponse',
'Send-DurableExternalEvent',
'Start-NewOrchestration')
'Start-DurableOrchestration')

# Cmdlets to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no cmdlets to export.
CmdletsToExport = @(
'Get-OutputBinding',
'Invoke-ActivityFunction',
'Invoke-DurableActivity',
'Push-OutputBinding',
'Set-DurableCustomStatus',
'Set-FunctionInvocationContext',
Expand All @@ -72,6 +72,9 @@ VariablesToExport = @()

# Aliases to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no aliases to export.
AliasesToExport = @(
'Invoke-ActivityFunction',
'New-OrchestrationCheckStatusResponse',
'Start-NewOrchestration',
'Wait-ActivityFunction')

# Private data to pass to the module specified in RootModule/ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@

# Set aliases for cmdlets to export
Set-Alias -Name Wait-ActivityFunction -Value Wait-DurableTask
Set-Alias -Name Invoke-ActivityFunction -Value Invoke-DurableActivity
Set-Alias -Name New-OrchestrationCheckStatusResponse -Value New-DurableOrchestrationCheckStatusResponse
Set-Alias -Name Start-NewOrchestration -Value Start-DurableOrchestration

function GetDurableClientFromModulePrivateData {
$PrivateData = $PSCmdlet.MyInvocation.MyCommand.Module.PrivateData
Expand All @@ -22,7 +25,7 @@ function GetDurableClientFromModulePrivateData {
.DESCRIPTION
Start an orchestration Azure Function with the given function name and input value.
.EXAMPLE
PS > Start-NewOrchestration -DurableClient Starter -FunctionName OrchestratorFunction -InputObject "input value for the orchestration function"
PS > Start-DurableOrchestration -DurableClient Starter -FunctionName OrchestratorFunction -InputObject "input value for the orchestration function"
Return the instance id of the new orchestration.
.PARAMETER FunctionName
The name of the orchestration Azure Function you want to start.
Expand All @@ -31,7 +34,7 @@ function GetDurableClientFromModulePrivateData {
.PARAMETER DurableClient
The orchestration client object.
#>
function Start-NewOrchestration {
function Start-DurableOrchestration {
[CmdletBinding()]
param(
[Parameter(
Expand Down Expand Up @@ -78,7 +81,7 @@ function GetUrlOrigin([uri]$Url) {
$fixedOriginUrl.ToString()
}

function New-OrchestrationCheckStatusResponse {
function New-DurableOrchestrationCheckStatusResponse {
[CmdletBinding()]
param(
[Parameter(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,58 @@ public async Task DurableClientFollowsAsyncPattern()
}
}

[Fact]
public async Task LegacyDurableCommandNamesStillWork()
{
var initialResponse = await Utilities.GetHttpTriggerResponse("DurableClientLegacyNames", queryString: string.Empty);
Assert.Equal(HttpStatusCode.Accepted, initialResponse.StatusCode);

var initialResponseBody = await initialResponse.Content.ReadAsStringAsync();
dynamic initialResponseBodyObject = JsonConvert.DeserializeObject(initialResponseBody);
var statusQueryGetUri = (string)initialResponseBodyObject.statusQueryGetUri;

var startTime = DateTime.UtcNow;

using (var httpClient = new HttpClient())
{
while (true)
{
var statusResponse = await httpClient.GetAsync(statusQueryGetUri);
switch (statusResponse.StatusCode)
{
case HttpStatusCode.Accepted:
{
var statusResponseBody = await GetResponseBodyAsync(statusResponse);
var runtimeStatus = (string)statusResponseBody.runtimeStatus;
Assert.True(
runtimeStatus == "Running" || runtimeStatus == "Pending",
$"Unexpected runtime status: {runtimeStatus}");

if (DateTime.UtcNow > startTime + _orchestrationCompletionTimeout)
{
Assert.True(false, $"The orchestration has not completed after {_orchestrationCompletionTimeout}");
}

await Task.Delay(TimeSpan.FromSeconds(2));
break;
}

case HttpStatusCode.OK:
{
var statusResponseBody = await GetResponseBodyAsync(statusResponse);
Assert.Equal("Completed", (string)statusResponseBody.runtimeStatus);
Assert.Equal("Hello Tokyo", statusResponseBody.output[0].ToString());
return;
}

default:
Assert.True(false, $"Unexpected orchestration status code: {statusResponse.StatusCode}");
break;
}
}
}
}

[Fact]
public async Task ActivityExceptionIsPropagatedThroughOrchestrator()
{
Expand Down
4 changes: 2 additions & 2 deletions test/E2E/TestFunctionApp/CurrentUtcDateTimeClient/run.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ Write-Host "CurrentUtcDateTimeClient started"

$ErrorActionPreference = 'Stop'

$InstanceId = Start-NewOrchestration -FunctionName 'CurrentUtcDateTimeOrchestrator' -InputObject 'Hello'
$InstanceId = Start-DurableOrchestration -FunctionName 'CurrentUtcDateTimeOrchestrator' -InputObject 'Hello'
Write-Host "Started orchestration with ID = '$InstanceId'"

$Response = New-OrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
$Response = New-DurableOrchestrationCheckStatusResponse -Request $Request -InstanceId $InstanceId
Push-OutputBinding -Name Response -Value $Response

Write-Host "CurrentUtcDateTimeClient completed"
Loading