Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Functions do not work when using the dotnet-isolated:4-dotnet-isolated6.0 base image. #810

Closed
elliotchaim opened this issue Feb 15, 2022 · 24 comments
Assignees

Comments

@elliotchaim
Copy link

I'm trying to deploy an Azure Functions app (net6.0 isolated functions v4) to Azure (Linux P1v2 app service plan). the app is containerized with mcr.microsoft.com/azure-functions/dotnet-isolated:4-dotnet-isolated6.0 base image. i'm deploying and see logs in deployment center showing that the image was downloaded and container started. but no functions have been found and i get this error notification in the azure portal:

Microsoft.Azure.WebJobs.Script: WorkerConfig for runtime: dotnet-isolated not found.

i can't find any docs whatsover around setting up deployments to azure for functions v4 aside from deploying straight from vs code using the extension (not what i want to do). im trying to setup deployment of a production application from CI/CD using terraform

thanks

@elliotchaim elliotchaim changed the title Microsoft.Azure.WebJobs.Script: WorkerConfig for runtime: dotnet-isolated not found. Functions do not work when using the dotnet-isolated:4-dotnet-isolated6.0 base image. Feb 16, 2022
@elliotchaim
Copy link
Author

I have some new information for this which you may find useful.

  • Adding the app setting WEBSITES_ENABLE_APP_SERVICE_STORAGE=false on our Azure app service resolved the WorkerConfig runtime error.
  • Updated worker nugets to the following versions
<PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.3.0" OutputItemType="Analyzer" />
<PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.6.0" />

Now when running the container I can see the following output.

info: Host.Function.Console[0]
      It was not possible to find any compatible framework version
info: Host.Function.Console[0]
      The framework 'Microsoft.AspNetCore.App', version '6.0.0' (x64) was not found.
info: Host.Function.Console[0]
        - No frameworks were found.
info: Host.Function.Console[0]
info: Host.Function.Console[0]
      You can resolve the problem by installing the specified framework and/or SDK.
info: Host.Function.Console[0]
info: Host.Function.Console[0]
      The specified framework can be found at:
info: Host.Function.Console[0]
        - https://aka.ms/dotnet-core-applaunch?framework=Microsoft.AspNetCore.App&framework_version=6.0.0&arch=x64&rid=debian.11-x64

After this output, the host starts successfully. BUT when a function is triggered (in my case a Timer triggered function) I can see the following error:

fail: Function.ProcessTransactionFiles[3]

      Executed 'Functions.ProcessTransactionFiles' (Failed, Id=068bc539-f63c-4f3a-9d1a-fac8c03c8c87, Duration=44738ms)

      Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.ProcessTransactionFiles

       ---> System.InvalidOperationException: Did not find any initialized language workers

         at Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcherLoadBalancer.GetLanguageWorkerChannel(IEnumerable`1 rpcWorkerChannels) in /src/azure-functions-host/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcherLoadBalancer.cs:line 27

         at Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InvokeAsync(ScriptInvocationContext invocationContext) in /src/azure-functions-host/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs:line 389

         at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in /src/azure-functions-host/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:line 95

         at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionInvokerBase.cs:line 82

         at Microsoft.Azure.WebJobs.Host.Executors.VoidTaskMethodInvoker`2.InvokeAsync(TReflected instance, Object[] arguments) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\VoidTaskMethodInvoker.cs:line 20

         at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync(Object instance, Object[] arguments) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs:line 52

         at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 596

         at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance, ParameterHelper parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 542

         at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 320

         --- End of inner exception stack trace ---

         at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 367

         at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 108

@ChrisKlug
Copy link

I've got the same issue. However, it actually used to work for us. But after a redeploy, we started getting this issue.

Our function isn't containerized though. So it is not only for containerized function. However, we are using .NET 6.0, dotnet-isolated and extension version 4, so it sounds a LOT like the same issue.

I've tried re-deploying the whole app service, but it just comes back with the same issue. Any information would be appreciated!

@ChrisKlug
Copy link

It actually seemed to sort itself out when I added runtimeStack: 'DOTNET-ISOLATED|6.0' to the AzureFunctionApp@1 task I used to deploy it... Not sure why it worked before... But I hope that helps!

@drmathias
Copy link

I'm deploying with Bicep. Was working fine until I deployed it again this morning, now getting this error.

@ChrisKlug
Copy link

So am I. It has worked fine previously, but stopped working for our deployments today. However, adding the above mentioned runtimeStack setting seems to have solved to problem for us

@drmathias
Copy link

Unfortunately didn't solve the issue for me.

Worth mentioning that I'm deploying to a Windows app service, so this isn't exclusive to app service for linux.

@ChrisKlug
Copy link

Apparently it didn't solve it for me either. Every time I re-deploy the bicep files, the function dies. However, re-deploying the code seems to reset it. Not a really good solution though...

@ghost ghost assigned brettsam Feb 24, 2022
@michaelpeng36 michaelpeng36 transferred this issue from Azure/azure-functions-core-tools Feb 24, 2022
@xtellurian
Copy link

xtellurian commented Feb 28, 2022

I'm running into a similar issue. Deploying a dotnet 6.0 functions project. The Azure resource (linux app) is created using Pulumi then the code is deployed with functions CLI func azure functionapp publish

The problem occurs when I update the infrastructure. Redeploying the code with func azure functionapp deploy fixes the issue - but it's a dangerous place to be.

@fabiocav
Copy link
Member

fabiocav commented Mar 1, 2022

Adding @kshyju here, but we have a deployment that should be completing in the next couple of weeks that will address this issue.

Will keep this open until that is done, but in the meantime, you follow the guidance in this comment

@jkdmyrs
Copy link

jkdmyrs commented Mar 4, 2022

I am experiencing the same issue, however, I am just running a dotnet publish and using the AzureFunctionApp@1 Azure DevOps Pipelines task to deploy the net6 functions app to a Linux functions app in Azure. I'm not using docker so I do not have the ability to specify App Service images as suggested in the workaround referenced by @fabiocav's comment.

What is the guidance in this case?

@ihordyrman
Copy link

ihordyrman commented Mar 7, 2022

We are facing this issue as well, but in our case, we don't use a docker image. We are creating and configuring Azure resources via the ARM template.
A similar issues are here #821, #819

image

@fabiocav any updates on this? The comment you've linked is more than two weeks old and the issue is still here.

@BinishPeter
Copy link

BinishPeter commented Mar 7, 2022

I had same issue, I got it fixed by changing the linux-fx-version to DOTNET-ISOLATED|6.0
check stackoverflow on how to set it

Few other settings might also be interesting
image

@ChrisKlug
Copy link

We are setting the linux-fx-version, which works...ish. However, re-deploying the infrastructure using Bicep causes the function to fail again until it is re-deployed

@jkdmyrs
Copy link

jkdmyrs commented Mar 8, 2022

I was able to resolve the issues on my end via this comment in issue #556.

Basically, I'm using Azure App Configuration with keyvault linked values, and the keyvault access policy did not exist.

This exception is extremely generic and seems to be thrown for ANY startup issue. There is definitely a need for the logging/exception handling during isolated functions startup to be drastically imrproved.

@kshyju kshyju assigned kshyju and unassigned brettsam Mar 9, 2022
@fabiocav
Copy link
Member

fabiocav commented Mar 9, 2022

@ihordyrman did the workaround in the comment I've linked to help address the issue? Azure/azure-functions-docker#600 (comment), if not, it would be a good idea to have a separate issue opened with your application details so we can take a closer look, as this may have a different root cause.

@kshyju
Copy link
Member

kshyju commented Mar 9, 2022

@ihordyrman Could you try deploying using function core tools and see that makes a difference?

@elliotchaim The "The framework 'Microsoft.AspNetCore.App', version '6.0.0' (x64) was not found" error is caused by this issue, which was originally fixed, but only for app service images. Now we fixed it for all images and the deployment for that is in-progress. Will share an update here when it is done (hopefully in the next couple of days)

@drmathias
Copy link

@ihordyrman Could you try deploying using function core tools and see that makes a difference?

Deploying with func azure functionapp publish seems to work. However deploying the resource with Bicep/ARM and then using the AzureFunctionApp@1 Azure pipelines task to deploy the function app results in the error.

@fabiocav
Copy link
Member

Just wanted to provide an update here. The full deployment with updated images addressing this issue should complete this week.

We'll close this issue with an update when that is done.

@tnsholding
Copy link

Any updates on this? - really stuck on this one, and anxiously waiting for a fix...

@arrudanathan
Copy link

arrudanathan commented Apr 4, 2022

I dont know what happened, but in last friday i had the same problem, bu now he dont happen again. had any update on this weekend?

@elliotchaim
Copy link
Author

I fixed this issue by changing the base image to mcr.microsoft.com/azure-functions/dotnet-isolated:4-dotnet-isolated6.0-appservice.

I only figured this out after finding some obscure documentation somewhere. Really need to improve the docs for this!

@kshyju
Copy link
Member

kshyju commented May 9, 2022

Deployment with the fix (for non-app service images) has been completed. The Microsoft.AspNetCore.App framework should be present in the mcr.microsoft.com/azure-functions/dotnet-isolated:4 image now.

Let us know if you run into issues.

@kshyju
Copy link
Member

kshyju commented May 9, 2022

I fixed this issue by changing the base image to mcr.microsoft.com/azure-functions/dotnet-isolated:4-dotnet-isolated6.0-appservice.

I only figured this out after finding some obscure documentation somewhere. Really need to improve the docs for this!

@elliotchaim You should be able to use the non app-service images now.

@ghost
Copy link

ghost commented May 13, 2022

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

@ghost ghost closed this as completed May 16, 2022
@ghost ghost removed the no-recent-activity label May 18, 2022
@Azure Azure locked as resolved and limited conversation to collaborators Jun 18, 2022
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests