AzureFunction: The term 'New-AzureStorageContext' is not recognized

[knikol]

Repro steps

Issue occurs randomly: no real steps to reproduce.

• Azure function, PowerShell, HTTP trigger
• Invoke by HTTP-POST

Expected behavior

No errors occur, HTTP call succeeds

Actual behavior

• Function fails with 500 HTTP error

In Azure Portal dev-console:

System.Management.Automation.CommandNotFoundException: The term 'New-AzureStorageContext' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)

HTTP response:

{
"id":"b2a79469-5fcd-4e26-bbdb-4c402da94652",
"requestId":"6ea2ca65-e587-4457-b1c8-8f419dd5c8ff",
"statusCode":500,
"errorCode":0,
"messsage":"An error has occurred. For more information, please check the logs for error ID b2a79469-5fcd-4e26-bbdb-4c402da94652"
}

Known workarounds

Issue is resolved for a while by restarting of Azure Function (AppService) from Azure Portal UI

Related information

• PowerShell code

$secpasswd = ConvertTo-SecureString $config.Key -AsPlainText -Force;
$mycreds = New-Object System.Management.Automation.PSCredential ($config.ApplicationId, $secpasswd)
Login-AzureRmAccount -ServicePrincipal -Tenant $config.TenantId -Credential $mycreds -SubscriptionName $config.AzureSubscriptionName

[tohling]

Assuming that your Function is using the default installed Azure PowerShell version 1.4, could you kindly add the following to the beginning of your PowerShell run.ps1 script, and see if the issue goes away?

Import-Module "D:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Azure.psd1" -Global

[knikol]

The Import-Module fails. Error in dev-console (from try/catch)

2017-04-15T18:03:35.874 System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
at System.Collections.ArrayList.set_Item(Int32 index, Object value)
at System.Collections.Specialized.OrderedDictionary.set_Item(Object key, Object value)
at Microsoft.Azure.Commands.Common.Authentication.Factories.ClientFactory.AddHandler[T](T handler)
at Microsoft.Azure.Commands.ResourceManager.Common.AzureRMCmdlet..ctor()
at Microsoft.Azure.Commands.Profile.AddAzureRMAccountCommand..ctor()

But the error for Login-AzureRmAccount has changed:

2017-04-15T18:03:36.438 Login-AzureRmAccount : The 'Login-AzureRmAccount' command was found in the module 'AzureRM.Profile', but the module could not be loaded. For more information, run 'Import-Module AzureRM.Profile'.

[tohling]

Seems like there is something not quite right in the environment that your Function App landed on. I apologize for the churn, but could you also add the following to your script?

Import-Module "D:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ResourceManager\AzureResourceManager\AzureRM.Profile\AzureRM.Profile.psd1" -Global;
Import-Module "D:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ResourceManager\AzureResourceManager\AzureRM.Resources\AzureRM.Resources.psd1" -Global;

I'll be monitoring this thread so let me know if you see any more errors.

[tohling]

FYI, to disover the path to any required Azure PowerShell *.psd1 files, you may use the Kudu PowerShell console to navigate the directory structure of the installed Azure PowerShell modules directory. Here are the steps:

1. Navigate to the Azure Functions Portal UI.
2. Click on your Function App and select Platform features->Advanced tools(Kudu)
3. You should see a new browser tab that shows you the Kudu Diagnostic Console. Click on Debug console->PowerShell
4. On the command prompt, type, cd "D:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell" . You should now be able to navigate through all the child folders for the Azure PowerShell module.

If you see anymore errors about missing [xxxxxxx].psd1 files, locate the absolute path for those files and add them to your script. For certain cmdlets, the order that they are imported matters as well.

[knikol]

All three Import-Module fails with the same System.ArgumentOutOfRangeException error.
Error for Login-AzureRmAccount is also the same: The 'Login-AzureRmAccount' command was found in the module 'AzureRM.Profile'.

Now the function has this issue as I did not restart the function AppSerivce. That's why I was able to play with it and followed your recommendations: modified the code online from Portal UI and did HTTP call to see the result.

[tohling]

Could you kindly share your Function App name, either directly or indirectly? Also, I will need to know the name of the Function that is failing. This will help me investigate further. Thanks!

[knikol]

Function url: https://m360nebula-mno547ob4xooc.azurewebsites.net/api/RetrieveVMStatus

[tohling]

@knikol, I was not able to reproduce the error with my own setup. I created a HTTP trigger that reads from a config.json file and uses the Login-AzureRmAccount cmdlet. I triggered the Function many times and did not see errors.

I will need to see this repro in your environment. Is it possible for me to run your RetriveVMStatus Function or could you create another Function that is very similar to RetrieveVMStatus but only have the code segment that uses the Login-AzureRmAccount and give me permission to run the Function?

[knikol]

@tohling , please, try to send POST request to the url with JSON body:

{
TenantName: "Foo",
ResourceGroup: "Foo",
Key: "FooKey"
}

[tohling]

@knikol, yes I can see the 500 error in the request response. Unfortunately, since I cannot repro this elsewhere, I basically need permission from you to troubleshoot from within your Function App.

Would you agree to me creating another Function in this Function App that only calls the Login-AzureRmAccount cmdlet? I would like to isolate the code-path as much as possible.

[knikol]

Yes, go ahead please.

[tohling]

@knikol, I have created a new Function named TestFunction within your Function App. I was able to reproduce the error in that state. But I noticed that when I restarted the Function App, the error went away, both for TestFunction as well as RetrieveVMStatus.

My suspicion is that your m360nebula-mno547ob4xooc Function App process may have been in a bad state that manifested itself into this error. I am re-reading your first post on this thread. I have some follow-up questions:

1. Are you seeing the behavior that your Function RetriveVMStatus would work for a while and then randomly fail with this error message and then go back to being workable again?

2. How long have you been seeing this behavior?

3. If you encounter this error again, could you restart your Function App and see if the error goes away?

[knikol]

@tohling

1. Yes, but it goes back mostly after restart only
2. It has started to occur 2 weeks ago or so when we started implementation of the function
3. This is how we used to resolve the issue. But we cannot do it as the FunctionApp has other functions which are long-running operations (30-40 minutes): restart stops the execution with state lost. My suspicion that this long running operation may result in the bad state of process.

[tohling]

@knikol - Thank you for your quick response. I gather that you are using an App Service Plan and not the Consumption Plan since you are able to execute the long running operations.

Unfortunately, at this experimental stage, we have not tested the impact of long running operations for the Azure PowerShell scenarios. One possibility is to start the long running script using the Start-Job cmdlet. That long running script can write to a storage queue when it is done. You can then have a Queue-Triggered Function notify you when the message is detected in the queue.

If this becomes too cumbersome for your workflow, we highly recommend using Azure Automation.

[leeberg]

FYI - I see this was added to the May 2017 Milestone, great. I am adding to this post as we also have a number of PowerShell Functions added in our production environment with some intermittent issues. We deployed about 2 weeks ago in production with NO issues but in the last few days we have had many issues.

Our functions simply need to Login to Azure and execute a single command. We are using functions as a scheduler. When everything is working, functions take apx. 3-5 Seconds to complete. However, like above there seem to be some instances of error/inconsistency depending on the environment the Function App landed on:

Scirpt:

$ARMA = Add-AzureRmAccount -ServicePrincipal -TenantId $TenantId -ApplicationId $ApplicationID -CertificateThumbprint $AutomationCertThumbprint
$ARMC = Set-AzureRMContext -SubscriptionId $AzureSubscriptionID

1: From Cold Start - my functions often time out / fail when using Add-AzureRmAccount - Second run is fine - but this is troublesome from a monitoring / alerting perspective.
2: In some instances, the Azure Modules do not seem to be present at all:

Add-AzureRmAccount : The term 'Add-AzureRmAccount' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Like you mentioned above, a stop/start of the function will pretty much always fix the issue, but these functions need to be running on schedule and a failure that requires a person to login and restart the function app causes customer impact / service outage.

I see you made a reference to try importing the Azure Module manually using the direct path - this does work fine in our DEV environment so I will likely migrate this step as a "just in case" - Thanks for the kudu debug steps as well. I will run some more tests and keep you posted.

UPDATE:

Hello after fighting lots of intermittent issues today, this seem to have done the trick!

Import-Module "D:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ResourceManager\AzureResourceManager\AzureRM.Profile\AzureRM.Profile.psd1" -Global; Import-Module "D:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ResourceManager\AzureResourceManager\AzureRM.Resources\AzureRM.Resources.psd1" -Global; Import-Module "D:\Program Files (x86)\Microsoft SDKs\Azure\PowerShell\ResourceManager\AzureResourceManager\AzureRM.Automation\AzureRM.Automation.psd1" -Global;

also apologies that I ended up here on the "Azure-Webjobs-SDK-Script" issues list, there are a few responces on this issue over at the Azure/Azure-Functions repo:

#305
#294

I am going to continue to monitor and blog on things to watch out for when using Powershell Azure Functions and the AzureRM Modules