Failed to sync triggers for function app . Error: BadRequest - Encountered an error (ServiceUnavailable) from host runtime. (CODE: 400) #16942
Comments
github-actions
bot
added
Area: Release
bug
environment:need-to-triage
|
Getting this issue sporadically also, sometimes consistently for long periods of time |
|
I am only getting this for function apps, that use consumption plans. |
|
Im also getting this issue sometimes. |
|
Me too ( Edit: Now, one day later, it worked for whatever reason. Can the action be made more robust please? We need to deploy production soon :D |
|
Suddenly working for us aswell. Would like some clarification around this issue as we cannot know for certain it will not happen again. |
|
@nadesu Would you be so kind and give an update on how likely it is that the problem will persist or continue to occur over the next week(s)? Thank you very much. |
|
@nadesu Still happening often. Any update? With this error it is unfeasible to use function apps on consumption plans in production environments |
|
@nadesu Happening for me aswell |
|
Hi all, This is an issue for me too, I managed to resolve it by recreating my storage account, service plan and function app the running the config again.
Prior to rebuilding I was getting the issue consistently, I had changed my plan from Y1 to B1 a few times and also been doing docker deployment to the same functions app. I wonder if the storage account got some garbage in. When running the deploy with --debug the following error was shown.
|
|
It's hard to tell where the issue lies, I however attempted calling the synctriggers api directly. https://management.azure.com/subscriptions/*********/resourceGroups/rgName/providers/Microsoft.Web/sites/functionApp/syncfunctiontriggers?api-version=2022-03-01. It works for the first couple of calls, however after a while it starts to return 400, same as in the deployment, with the same error message as seen in the deployment logs. I've also noted that the api-version differs from the Azure task, and the latest available, hard to say if it makes any difference. |
|
Any news here of how to fix this? We are having the same issue. |
|
We ended up moving back to app services on windows with dedicated plans. |
|
I am facing the same problem with my new function. |
|
Changed to B1, it works for me, it seems to only happen on the Linux Y1 Consumption (Serverless) plan. |
|
Hello, I will investigate and come back to this thread. |
Thank you very much! And I've already tried to remove the app setting, it also works well Azure/functions-action#147 (comment) |
|
Don't know if it will help you, but In my case, the problem was caused because I tried to deploy it while the app was stopped. |
Have tried both |
|
@FinVamp1 isn't it problem with this line in the log:
Run from package is not supported for linux azure functions on consumption app service plan. |
|
It's actually required, according to documentation.
|
|
Same error for me. It seems to happen without any reason, without anything that has changed from our side. It's a big blocker, it prevents us from deploying updates to our functions. |
|
Hello, will investigate with priority today. The task deploys using WEBSITE_RUN_FROM_PACKAGE_URL by default for the Linux Consumption plan. |
|
@FinVamp1 does the task also override @Palving Are you sure? Here is shown that consumption linux functions support both run from package with URI and simple zip deploy. Edit: Zip deploy for consumption linux function is supported only when performing remote build:
|
|
In our case, with .NET 6, Azure Function v4, Microsoft Identity, the problem was that we had a Function with a reference to the WebApp, used to access the DB Identity context (because we had the necessity to retrieve users when the function was running). After spending a lot of time looking for a solution, we decided to remove the WebApp reference from the Function, and we created a library with a separated scaffolding (DB first). The release process started working again. |
|
Deploying a Python functions app (v4) from VSCode's built-in integration has started failing with the same error. I tried removing the managed identity, per the above suggestions, but that did not help. Neither did removing the I've been trying several times for the last hour. Same app deployed ok yesterday. |
|
We've changed the function from Linux to Windows and it instantly worked. This was without any other changes in our pipeline and configuration. |
|
Unfortunately, it looks like Python functions are not supported on Windows. |
|
Completely deleting the resource group, function app, storage, etc was still not enough. Something in the deployment framework is busted. For help pinpointing, note that I was wrong about it working yesterday; last succesful deploy was actually October 9. |
|
@richard-berg I've reproduced your issue and this is related to setting the TZ app setting which isn't supported on Linux Consumption SKU's this is only supported on dedicated and Elastic Premium SKU's. This is discussed at Azure/azure-functions-host#9203 . I'll be adding Checks to show this as a configuration issue in Diagnose and Solve, |
|
Thanks @FinVamp1! I would've never figured that out from the (lack of) deployment logs alone. I was able to get my app deployed, with the triggers reconfigured to UTC times. They'll be 1hr off during Daylight Savings, but that's better than not running at all. |
|
Same problem here. It looks like this one is being open for over a year and it's a real big trouble. |
|
Just hit this deploying a linux function app to consumption plan using the v2 release task in azure devops. I had removed a timer trigger from the function app and the new build failed with the "Failed to sync triggers" error. I was still able to deploy a previous build. The fix for me was to delete all the |
|
Has anyone solved this? its been occurring on all our production deployments and our systems are completely offline and we are at a complete loss what to do right now. 8:43:13 PM PROD-XXX: Syncing triggers... |
|
@lyadeski have you tried switching the app host to Windows instead of Linux? |
i did but that introduces new issues. i dont understand what has changed, everything was working fine until a couple days ago, we've made no code changes or infrastructure changes and suddenly everything stops working - very frustrating |
|
I encountered this error when I had neglected to configure Azure Private Endpoint DNS. This prevented the Function App runtime from resolving the hostname of the Azure Storage private endpoint. I had to open a support case with Microsoft to get to my solution. I highly recommend you do the same if you are encountering this error. They have additional views into the engine that can help pinpoint the issue. |
We had the exact same problem yesterday. Function App was completely down, couldn't redeploy, and I open a support ticket with Microsoft to only be told that I either needed to switch away from consumption plan, or implement a timer function to keep the app from going to cold start. Absolutely ridiculous answer to the problem... Sounds to me like theres a bug with the consumption plan currently, and no one wants to address it. Our function app worked perfectly fine before the weekend, and we made no code changes or configuration changes, it just stopped working. For now, I moved to the Premium plan, and its back working, but we're going to be looking into alternative options, since we can't have this happen again, and not get proper answers from the support team. |
|
Hello all, I'm Finbar and I'm part of the Functions engineering team at Microsoft and I work on the features related to Application Code Deployment. This thread is long and so I will summarize the symptoms people get and the potential causes for each. If this update doesn't help you resolve the issue then please consider opening a support case or providing me with an app name and timeframe for when your issue occurs. Summary of issues related to Sync Triggers
More Details
Investigating Sync Trigger issues
If this doesn't help then please contact Support or provide an app name and timeframe in this thread or email to me finbarr at microsoft.com Next Steps from MicrosoftWe understand that this error can be cryptic and we are looking at how best to resolve this in two areas.
Thanks, Finbar |
|
Here's what worked for me: I was trying to zip deploy to a Linux/Node.js-based Function App. I had a working instance created through the Azure Portal and another created via a Bicep template where the ServiceUnavailable error would happen. I've used the following commands to get the settings for each instance:
Diffing the outputs I noticed that my new function app was missing a definition for the Here's a part of the Bicep template for reference: |
|
I was getting the Syncing Triggers BadRequest error during the deployment. I am using Linux Y1 service plan. The runtime was set to 3.0. I swtiched to 4.0, and there were no more errors. You can access the runtime setting by visiting:
|
|
For me the issue was related to the storage account, the keys in the azure function app setting wasn't correct and pointed to a deleted azure storage accounts, and the application needed that to deploy.
Then it worked! |
Please see my comment, the health check couldn't see that my settings had storage account that didn't exist. |
|
We have found that we get this error when deployment throws an exception. E.g. when our (tirgger) configuration was wrong. This is with dotnet-isolated runtime. This seems to be different behavior than we had before. |
Same for me. Also, log stream does not work, so I can't even find which exception do I have. This is infuriating ;-( |
|
@VladislavsKugelevics we only looked at the log stream, sorry to hear that doesn't work for you |
|
I have advice for anyone having the same issue. If you are not being able to see the error in log stream or application insights.
|
|
We managed to switch the Linux App Service Plan from Y1 (Consumption) to B1 and it resolved it for us. It broke for us after adding AAD auth. I understand this has been already posted as a solution working for some people and not others. Nevertheless this is the Az CLI we ran: If Microsoft divulges any info on why this is happening, I'll be sure to share with a follow up. |
|
For others who may be struggling with this error message, I have run into this issue for the past 6 weeks and finally was able to identify what was causing it for my Function App. With all the correct configuration settings set up for vnet integration etc with the storage account, I was still experiencing incosistent deployment success/failures. After opening a ticket with Microsoft, they were able to look at back end logs - not visible to end users during deployment - and there was an error related to the Python packed named "typing". This was in the requirements.txt file we were trying to deploy but we were not aware that package comes default with Python and was not needed to be added. Removal of this package from the requirements.txt file fixed the issue completely and deployments have been 100% successful since. "Why does this cause Syncing of Triggers to fail sometimes?" I am told by Microsoft support that it installs the package successfully, but after the function app restarts after deployment of code (successfully), it hits an error that prevents the restart from completing in time. Sometimes it progresses past the error fast enough and Syncing of Triggers is able to complete, and other times it doesn't. "What could be done to help avoid or debug this issue?" Better error logging to the end users during deployment via VS Code deployment logs. This is my understanding so far, so am happy to be corrected anywhere I may have mis-spoken. This is also likely an isolated incident with the "typing" package, but it might give clues for others out there regarding the requirements.txt file and packages being installed. |
|
I also encountered with this issue recently. But I found below issue in my situation:
It got worked for me after solving those. Hope this might be helpful to anyone. |
FinVamp1
added
FunctionsTaskBacklog
and removed
environment:need-to-triage
|
I ran into this issue because I had not compiled my custom handler for the correct runtime, for example we are using golang so we had to do: Once i did that, sync triggers worked right away with |
|
We didn't specify the python version in our azure-pipelines.yml file. We fixed this issue by adding the following code in our build step: - task: UsePythonVersion@0
inputs:
versionSpec: '$(python.version)'
displayName: 'Use Python $(python.version)' |
and others
Note
Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo
For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks
If you have an issue or request for the Azure Pipelines service, use developer community instead:
https://developercommunity.visualstudio.com/spaces/21/index.html )
Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name: AzureFunctionApp@1
list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks
Environment
Azure pipelines
Agent - Hosted or Private:
If using Hosted agent, provide agent queue name:
Agent name: 'Azure Pipelines 2'
Agent machine name: 'fv-az313-661'
Current agent version: '2.210.1'
Operating System
Ubuntu
20.04.5
LTS
If using private agent, provide the OS of the machine running the agent and the agent version:
Issue Description
Deployment to function apps worked in the beginning but has however started to consistently fail across deployments. Even after deletion and re-deployment of the function app, plans and storage account it fails when zip-deploying. The task itself seems to bring the app down, and it takes a while for it to be available again.
Task logs
This is a linux function app I am trying to deploy with consumption plan
Troubleshooting
Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting
Error logs
The text was updated successfully, but these errors were encountered: