-
Notifications
You must be signed in to change notification settings - Fork 432
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Random errors across every single Python HttpTrigger Azure Function in Function App #1758
Comments
I experienced the similar issue.
|
Could we please have an invocation id and a region so we can take a look at the app? Please note that Python is experimentally supported and not fit for production scenarios. |
@christopheranderson
|
I have developed an Azure function that has ran into exactly same error. "Exception while executing function: Functions.RadiaPythonPoc The region is South Central US & with invocation id, do you mean the subscription id? |
I'm having the same issue, previously working code just stopped working. Invocation: 12a6e044-5ab1-40ee-9705-e81ad59671f5 Region: UK West Exception while executing function: Functions.elevate |
@fabiocav Would you be able to confirm this issue is still under investigation please? |
I am also facing the same exact issue: Location: Canada Central Exception while executing function: Functions.getMissingStocks |
Ditto - I am seeing the same thing in UKSouth from this morning - any update? Exception while executing function: Functions.HttpTriggerPython31 |
I was able to resolve the issue once I created a virtual environment and imported libraries inside it. You would require to navigate to the folder of your function on the debug console and do so. Below are the steps:
Now, you can go ahead and install all the packages that you need via 'python -m pip install '. The only change would be in the way you are trying to read the packages in the script. They should now be imported with a different statement as below: I took reference from https://stackoverflow.com/questions/43970307/azure-functions-installing-python-modules-and-extensions-on-consumption-plan |
This thing affects us too on a time triggered function on consumption plan. Makes using Azure Functions impossible. But it used to work for some time. The function's output is like so: Exception while executing function: Functions.RecountRecommendations Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: Functions.RecountRecommendations ---> System.ApplicationException at async Microsoft.Azure.WebJobs.Script.Description.ScriptFunctionInvoker.ExecuteScriptAsync(String path,String arguments,Object[] invocationParameters,FunctionInvocationContext context) at C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Description\Script\ScriptFunctionInvoker.cs : 114 at async Microsoft.Azure.WebJobs.Script.Description.ScriptFunctionInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Description\Script\ScriptFunctionInvoker.cs : 55 at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Description\FunctionInvokerBase.cs : 95 at async Microsoft.Azure.WebJobs.Host.Executors.VoidTaskMethodInvoker`2.InvokeAsync[TReflected,TReturnType](TReflected instance,Object[] arguments) at async Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync[TReflected,TReturnValue](Object instance,Object[] arguments) at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeAsync(IFunctionInvoker invoker,ParameterHelper parameterHelper,CancellationTokenSource timeoutTokenSource,CancellationTokenSource functionCancellationTokenSource,Boolean throwOnTimeout,TimeSpan timerInterval,IFunctionInstance instance) at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstance instance,ParameterHelper parameterHelper,TraceWriter traceWriter,CancellationTokenSource functionCancellationTokenSource) at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.Execute… |
Hi everyone, |
@sergei3000 Thanks! This fixed it for me too. Were you using a custom installed version of python? |
@sergei3000 Thhanks alot that worked for me |
@StephenThomson I did not do that personally, but a later version of Python and some additional libraries were installed into the app I am using by a colleague of mine. This is what he used as a reference: |
@StephenThomson apologies for the delay on this. The notification was completely missed. @sergei3000 so you started experiencing this on the version immediately after 1.0.11027? @StephenThomson based on your initial report, I'm assuming this issue started more recently for you, correct? |
@StephenThomson would you be able to provide a repro for this? |
@fabiocav I started experiencing this several weeks ago, but I did not use Functions so much before that. So I should say I started experiencing this error right after starting to use Azure Functions extensively. I chose version 1.0.11027, because it was the second release back from the date this issue was opened. Also I found that the error mentions WebJobs in logs, and there was WebJobs SDK release on August 2, which is the next release after 1.0.11027. This could be a foolish thing to make this kind of a connection, but I just tried to find some clues to make things work for me :). |
@fabiocav : Thanks for getting back to me :) Yes I started experiencing this last week. It seems to be solved by setting FUNCTIONS_EXTENSION_VERSION=1.0.11296 I am currently trying to reproduce the error but it is tricky. When I first encountered it I redeployed onto a fresh function-app. This ran initially but then stopped working the next day. I've created a new function app today and am trying to get it to break. I did notice that the bug only happens for me when I am using python 3.6 from D:\home\site\tools. My steps so far are: New Function App - Consumption - UK West Output: "hello world from 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)]" Kudu: Rerun function: "hello world from 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:14:34) [MSC v.1900 32 bit (Intel)]" I will update as soon as I can. UPDATE: Approx 4h later rerun function: TEST LOGS: Invocation context : feebf67b-64a7-46b4-98a6-0cc053b363f8 Monitor Logs: |
Set FUNCTIONS_EXTENSION_VERSION=1.0.11296 and it started working. Will have to wait to see if they continue to work but my production code was changed over ~30h ago and is still working. |
The test code stayed working on 1.0.11296 for >24h. Switched it to ~1 and it stop functioning in a few hours. |
Clearing milestone and assignment for triage and further investigation |
Hi everyone. The drawback here is that for a time/http trigger to work the app should be always on. But if the price is not too high for you, it's a good solution actually, my code works much faster there. Also it imports additional libraries very fast, unlike functions working on the Consumption plan which take way too long to do that. At the same time, I really hope that Microsoft will fix this bug soon. It is really critical, since it is affecting so many people and makes it impossible using the very attractive Consumption plan. |
Hello, I believe that using a custom version of python is interacting poorly with our "placeholder" feature which is designed to optimize cold start scenarios. Thanks to @StephenThomson for clear repro steps. One of the important points to note is that placeholders are disabled when you specify a custom function version. So while on first look it might appear that 1.0.11296 solves the problem, actually all that is happening is that you are disabling placeholders. In fact, if you go to "Function App Settings" you'll probably see that your host version is something newer like this: This is because we retire older versions, and 11296 is already retired. You can confirm my theory by disabling placeholders directly. Remove your app setting for the function version, and replace it with This setting is documented here - please let me know how it goes. I do not know why placeholders are interacting poorly with a custom python version. However I want to point out that we are no longer investing in the experimental implementation of python that is currently available for functions v1. For more information about language support, see here. So we are unlikely to get to the bottom of this issue, but it looks like we have a workaround - disabling placeholders as described above. I will close this issue as "won't fix, but workaround available" once I hear from a few of you. |
We have started encountering random, intermittent errors across every single one of our HttpTrigger Azure Functions which are coded in Python. The log streaming service shows only this vague Exception when the issue occurs while executing a given function:
Functions.. Microsoft.Azure.WebJobs.Script: .
In looking at the logs under Monitoring, I was able to dig out this call stack:
The text was updated successfully, but these errors were encountered: