Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.NET Core 3.0 Availability on App Service #118

Open
fabiocav opened this issue Oct 4, 2019 · 57 comments

Comments

@fabiocav
Copy link
Member

commented Oct 4, 2019

Discussion for .NET Core 3.0 Availability on App Service.

@lanehoy

This comment has been minimized.

Copy link

commented Oct 4, 2019

Sound good. And what about App Service Linux ?

@fabiocav

This comment has been minimized.

Copy link
Member Author

commented Oct 4, 2019

App Service Linux is also coming very soon. We'll post an update there with details once that information is available.

@sjkp

This comment has been minimized.

Copy link

commented Oct 8, 2019

What else is included in this update? My Lets Encrypt site extension, that host a few webpages on the are starting to get 502 gateway errors. The errors starting being reported by users in US regions about a week ago, but now all regions seems to be affected, West Europe being hit today. Given that the errors have slowly spread I think it is on the infrastructure level, so I have no chance to debug this.
People are reporting it here: sjkp/letsencrypt-siteextension#331

@sjkp

This comment has been minimized.

Copy link

commented Oct 8, 2019

People are reporting similar issues for the standard kudu portal: Azure/azure-webjobs-sdk#2343
I also get the 502 gateway error on POST request, all my get requests work fine.

@fabiocav

This comment has been minimized.

Copy link
Member Author

commented Oct 8, 2019

@sjkp do you have more details on the exception/error seen? Would you be able to share a site name for investigation?

@tmenier

This comment has been minimized.

Copy link

commented Oct 8, 2019

@fabiocav I can provide some details on what I'm seeing:

  • Continuous WebJobs
  • .NET Framework
  • WebJobs SDK
  • [NoAutomaticTrigger] (or any trigger attribute that makes them show up in Kudu)

Repro: Click "Run" button on any function in Kudu. Spins for 2 minutes and returns this:

So far Azure support is stumped and so am I. But the timing seems to line up perfectly with this release. Note that we're seeing it with multiple completely isolated production WebJobs that haven't had code changes in a while, as well as totally trivial test WebJobs deployed solely to troubleshoot this issue. All suddenly started experiencing this last Friday (10/4) morning.

@sjkp

This comment has been minimized.

Copy link

commented Oct 8, 2019

@fabiocav you can browse my test site here (with a dummy extension that does a simple post request which fails)
https://webappcfmv5fy7lcq7o.scm.azurewebsites.net/dummyapp/

I just deployed an empty MVC app with no functionality other than a controller that accepts a POST request and it fails too. So it has nothing to do with my extension.

My sample site extension that shows the problem is uploaded here
https://github.com/sjkp/azure-site-extension-502

It is also available as a site-extension here https://www.nuget.org/packages/dummyapp/

Here is a failed request log https://github.com/sjkp/azure-site-extension-502/blob/master/https____1webAppcfmv5fy7lcq7o_80_dummyapp_%2C%20STATUS_CODE%20500%2C%20124750%20ms%2C%20POST.xml

image

There is a 2 minute gap between the different modules, I have no clue how to debug that futher.

@fabiocav

This comment has been minimized.

Copy link
Member Author

commented Oct 8, 2019

@sjkp thank you for all the details. We're taking a look at the issue and will provide updates here.

@fabiocav

This comment has been minimized.

Copy link
Member Author

commented Oct 8, 2019

@sjkp was this test site extension deployed onto an existing resource group/app? do you see the same issue if you deploy to a different app/resource group/plan?

@mathewc

This comment has been minimized.

Copy link

commented Oct 9, 2019

@sjkp I'm able to browse the https://webappcfmv5fy7lcq7o.scm.azurewebsites.net/dummyapp/ site without error - the ASP.NET page loads for me. Similarly for https://webappcfmv5fy7lcq7o.scm.azurewebsites.net/letsencrypt

@sjkp

This comment has been minimized.

Copy link

commented Oct 9, 2019

@mathewc @fabriocav get requests are working fine, so you can browse the two extension, but when you try a POST request they both fail. You can click the "next" button to initiate a POST request.

@tuin007

This comment has been minimized.

Copy link

commented Oct 9, 2019

Got any fix forecasts? I can no longer add to new domains.
I have this same problem here Southern Brazil

@mathewc

This comment has been minimized.

Copy link

commented Oct 9, 2019

@tuin007 what are your repro steps? Trying to determine if this is occurs only in site extensions.

@tuin007

This comment has been minimized.

Copy link

commented Oct 9, 2019

Playback: Click the "Run" button in any Kudu function. Turns for 2 minutes and returns: Server Error 502
Capturar

Next:
image

Return error:
image

@mathewc

This comment has been minimized.

Copy link

commented Oct 9, 2019

@sjkp The POST requests via the "Next" buttons in your webappcfmv5fy7lcq7o app all succeed for me when trying:

image

@tmenier

This comment has been minimized.

Copy link

commented Oct 9, 2019

@mathewc I don't know if my issue is related or not, but this is the call that is failing for us, invoked by simply clicking the "Run" button on any WebJob function in Kudu:

POST https://myappservice.scm.azurewebsites.net/azurejobs/function/Run?functionId=xxxx

Can you confirm with any trivial WebJobs function using the SDK that clicking "Run" in Kudu works? I initially reported this here and have since gotten a couple other reports of the same behavior. If this is in fact a different issue than the one being discussed here, would you mind replying to that issue as well? Thanks!

@fabiocav

This comment has been minimized.

Copy link
Member Author

commented Oct 9, 2019

Just as an update; the issue above is not related to the .NET Core 3.0 deployment. The deployment is moving forward while we continue to investigate the problem.

@sjkp

This comment has been minimized.

Copy link

commented Oct 9, 2019

@mathewc that is very strange. Because it doesn't work for me whatsoever.
image

@mathewc

This comment has been minimized.

Copy link

commented Oct 9, 2019

We are able to repro with the repro project you posted above however, and we’re investigating. Repros on a machine without .NET Core 3.

@sjkp

This comment has been minimized.

Copy link

commented Oct 9, 2019

Okay - it just started around the rollout of .net core 3. But it can ofc, be something else like a windows update.

@shibayan

This comment has been minimized.

Copy link

commented Oct 10, 2019

It seems that .NET Core 3.0 Runtime is available in some regions 👍 , but will the .NET Core 3.0 SDK not be updated at the same time?

@CarljohanDanling

This comment has been minimized.

Copy link

commented Oct 10, 2019

It seems that .NET Core 3.0 Runtime is available in some regions 👍 , but will the .NET Core 3.0 SDK not be updated at the same time?

Where can you find/see this information?

@pseabury

This comment has been minimized.

@pseabury

This comment has been minimized.

Copy link

commented Oct 10, 2019

Also - Is the Runtime selection box for App Services tied to the SDK, or the installed runtimes? It appears that for all the US regions that have the Core 3 runtime installed, you can still only choose 2.2 runtime.

image

@Kormyr

This comment has been minimized.

Copy link

commented Oct 10, 2019

Thanks, we have our first Asp.net Core 3 application running on a West EU App Service 👍

@justdmitry

This comment has been minimized.

Copy link

commented Oct 10, 2019

@Kormyr build locally and published via VS, or auto-build in AppService after push into git ?

@Kormyr

This comment has been minimized.

Copy link

commented Oct 10, 2019

@justdmitry Auto-built on ADO and auto-deployed in App Service after push into Git

@Link-69

This comment has been minimized.

Copy link

commented Oct 11, 2019

The deployment is said as complete but I still can't find 3.0 when creating a new web app on Azure?
image

@bradygaster

This comment has been minimized.

Copy link

commented Oct 11, 2019

@Link-69 - adding @btardif here to comment on the portal availability. The portal generally takes a few days beyond the runtime availability.

Also @fabiocav - whilst it is true that .NET Core 3.0 is available, the SDK is still rolling out to the App Service fabric. Is it possible that we could have a separate issue or announcement for when the .NET Core SDK roll-out is also complete?

Thanks in advance to everyone who made the deployment as efficient as possible.

@avoxm

This comment has been minimized.

Copy link

commented Oct 14, 2019

@fabiocav I am still having issues with publishing my app on East US.
I get HTTP Error 500.30 - ANCM In-Process Start Failure. Also VS 2019 complains while manual publishing saying that .net 3.0.0 does not exist on server however when it lists available runtimes it included 3.0.0 into that list.
Any idea when the deployment will be properly finalized and why this is happening ?

@marchy

This comment has been minimized.

Copy link

commented Oct 14, 2019

We are 3 full weeks after the supposed 'release' of .NET Core 3.0, and Azure, the first-party, Microsoft-owned cloud platform still has NO word, plans, communications or expectations set about when we can actually deploy this "released" platform on our servers.

It would be fine if you set the expectation that you need another, week, month, or heck... a full quarter to publish this. But instead you jeopardize your customers and drag them through hell and shamefully stay completely silent despite the multitude of comments, both on GitHub and on the .NET blogs asking about the confusion and keep us stuck with codebases that are half-migrated that we can't fully test and actually deploy.

It is COMPLETELY unacceptable to be this dis-coordinated, both executionally and more important – communicationally. I hope this gets brought up in every Azure team member's performance reviews this year and heads roll after this blotchery.

Cannot wait to exit this mess of an ecosystem
#dotnetfail

@fabiocav

This comment has been minimized.

Copy link
Member Author

commented Oct 14, 2019

@marchy the timeline was set in the announcement linked to this discussion (Azure/app-service-announcements#204), which was also updated once the deployment was globally completed, ahead of the estimated date. We are working to improve communication on those deployments to ensure those details are clearer and not missed.

As of that date, you should be able to deploy .NET Core 3.0 applications to all App Service regions.

@avoxm

This comment has been minimized.

Copy link

commented Oct 14, 2019

@marchy the timeline was set in the announcement linked to this discussion (Azure/app-service-announcements#204), which was also updated once the deployment was globally completed, ahead of the estimated date. We are working to improve communication on those deployments to ensure those details are clearer and not missed.

As of that date, you should be able to deploy .NET Core 3.0 applications to all App Service regions.

@fabiocav I am afraid that is not true. I am still not able to deploy .NET Core 3.0 app on East US.

@fabiocav

This comment has been minimized.

Copy link
Member Author

commented Oct 14, 2019

@avoxm deployment is completed. We can look at your specific issue to understand what is happening there.

If you open the Kudu Console (https://blogs.msdn.microsoft.com/benjaminperkins/2017/11/08/how-to-access-kudu-scm-for-an-azure-app-service-environment-ase/) and check the look at the output of dotnet --info, you should be able to see .NET Core 3.0 as an installed runtime. Can you confirm?

@fabiocav

This comment has been minimized.

Copy link
Member Author

commented Oct 14, 2019

@avoxm

This comment has been minimized.

Copy link

commented Oct 14, 2019

@avoxm deployment is completed. We can look at your specific issue to understand what is happening there.

If you open the Kudu Console (https://blogs.msdn.microsoft.com/benjaminperkins/2017/11/08/how-to-access-kudu-scm-for-an-azure-app-service-environment-ase/) and check the look at the output of dotnet --info, you should be able to see .NET Core 3.0 as an installed runtime. Can you confirm?

Thanks @fabiocav . So this is what I got it kudu.

Type 'exit' then hit 'enter' to get a new CMD process.
Type 'cls' to clear the console

Microsoft Windows [Version 10.0.14393]
(c) 2016 Microsoft Corporation. All rights reserved.

D:\home>dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   2.2.109
 Commit:    586f23c400

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x86
 Base Path:   D:\Program Files (x86)\dotnet\sdk\2.2.109\

Host (useful for support):
  Version: 3.0.0
  Commit:  7d57652f33

.NET Core SDKs installed:
  1.1.14 [D:\Program Files (x86)\dotnet\sdk]
  2.1.509 [D:\Program Files (x86)\dotnet\sdk]
  2.1.701 [D:\Program Files (x86)\dotnet\sdk]
  2.2.108 [D:\Program Files (x86)\dotnet\sdk]
  2.2.109 [D:\Program Files (x86)\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.12 [D:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.13 [D:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [D:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [D:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.12 [D:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.13 [D:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [D:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [D:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [D:\Program Files (x86)\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.16 [D:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.13 [D:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [D:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.12 [D:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.13 [D:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [D:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [D:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [D:\Program Files (x86)\dotnet\shared\Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

However this is what I am getting in VS2019.

image

And this is the deployed app.

image

@pseabury

This comment has been minimized.

Copy link

commented Oct 14, 2019

@fabiocav - When should we expect the SDK and AspNetCore.All to 3.0, if ever? Under what circumstances would these be required for AppServices deployments? In other words, when will the deployment map show all regions as current? The announcement says the deployment is complete, but it looks otherwise.

Also, like @Link-69 and I asked earlier, when will the portal be updated to reflect the new runtime?

@patricklee2

This comment has been minimized.

Copy link

commented Oct 14, 2019

AspNetCore.All is Deprecated and there is no 3.0.0 release
See aspnet/Announcements#314

@avoxm can you share the name of your site?

@bradygaster

This comment has been minimized.

Copy link

commented Oct 14, 2019

@pseabury - to be clear, it says the shared assemblies and ANCM components are there, and that the SDK is not yet there. Once the SDKs start landing in the App Service regions, the icons will appear green, reflecting full roll-out to those regions is complete.

Re-adding @btardif to comment on the portal availability of LTS and Current releases in the stacks drop-down.

@avoxm

This comment has been minimized.

Copy link

commented Oct 15, 2019

@patricklee2 @fabiocav there is a chance that the error is on my end due to not porting Cert/Pfx files.
Let me me try to validate it and get back to you. Error message is kinda misleading though :).
I'll post an update here soon. Thank you for being available to help, I do appreciate it much.

@abhisheksiddhu

This comment has been minimized.

Copy link

commented Oct 15, 2019

I am also being shown this error in VS 2019. I know self contained deployment can fix it, but the error itself says 3.0.0 is supported. How are these 2 different?

image

@oliviergareau

This comment has been minimized.

Copy link

commented Oct 15, 2019

Using the sample Web API (the weather forecast example) from VS 2019 and publishing from Visual Studio gives me the same error, and results in an App Service that does not work! Running in localhost works fine.

image

@timheuer

This comment has been minimized.

Copy link

commented Oct 15, 2019

@abhisheksiddhu @oliviergareau thanks for the note on this issue/dialog you are seeing. We're aware of this bug and looking to have it fixed in an upcoming service release. In this instance where the App Service has the runtime, you can click OK here and continue to publish. Apologies.

@oliviergareau

This comment has been minimized.

Copy link

commented Oct 15, 2019

@timheuer The App Service that is automatically created after clicking OK still ends up producing a 500.30 ANCM Start failure 😭 (I chose "Create New" when publishing)

@abhisheksiddhu

This comment has been minimized.

Copy link

commented Oct 15, 2019

@oliviergareau Are you using the single file deployment. I was also getting this error when doing a single file deployment, but normal deployment worked fine for me

@SimonOrdo

This comment has been minimized.

Copy link

commented Oct 15, 2019

@timheuer So if the AppServer has the runtime, but not the SDK, are you saying we should be able to do a standard deployment that's not a self-contained build? And when you say you're hoping to "have the bug fixed", does that mean you're looking to have the 3.x SDK deployed to AppService? And, what's the general timeline for "Service Releases"?

image

@Pilchie

This comment has been minimized.

Copy link

commented Oct 15, 2019

You should be able to deploy a framework dependent application as long as it is built before deployment. That means that a normal publish, or an Azure Pipelines CI/CD publish should work. The thing that won’t work without the SDK is when you push source code and the app is compiled in Azure App Service.

The bug being is the error dialog that says that 3.0 is missing when it isn’t.

Including the SDK is being tested, and should start rolling out soon.

Hope this helps.

@marchy

This comment has been minimized.

Copy link

commented Oct 16, 2019

@marchy the timeline was set in the announcement linked to this discussion (Azure/app-service-announcements#204), which was also updated once the deployment was globally completed, ahead of the estimated date. We are working to improve communication on those deployments to ensure those details are clearer and not missed.

As of that date, you should be able to deploy .NET Core 3.0 applications to all App Service regions.

@fabiocav Thanks for the reply Fabio. The only problem with this mindset – and stance – is that Deployment Center / pulling from repo STILL doesn't work. This isn't some fringe mechanism, it is THE first-party mechanism Azure has had for years and this little big detail was never once mentioned anywhere. Not in the ticket you mention and not until @shibayan, @pseabury and @Link-69 dropped comments on the surprise mismatch between runtime and SDK.

@bradygaster's suggestion seems to make the most sense - if internally you guys want/must keep these aspects separate.

Just understand that from a customer standpoint these details do NOT matter. Our .NET Core 3.0 isn't building when we push changes to our repo / Development Center in Azure --> .NET Core 3.0 is not ready.

Look at all the folks who have been burning developer hours simply because you guys are quietly trying to hide the simple fact that you didn't make the public release timeline – nor the posted one on October 11th. Transparency works 10x better.

Please institute a process so that this doesn't happen again (release expectations/estimates go out the SAME day as the public release, and you ask for as much time as is comfortably needed to make it available). The Azure deployment timelines have been a pain for all past .NET Core releases – but this one has been more especially damaging due to the size of the release and the fact that 3rd party EF 6 migrations have been completely broken on .NET Core 2.2 (mrahhal/Migrator.EF6#62) which means we've been stuck between outdated 2.1 codebases and a 3.0 codebase we can't deploy, and having to learn this once we've already done the codebase upgrades and things failing in the last stages while pushing to our servers. Just put yourselves in the shoes of people using this stuff instead of your internal, 'up-the-chain' milestones.

I share this all constructively. Much appreciate it guys.

@stay2be

This comment has been minimized.

Copy link

commented Oct 16, 2019

Is .NET Core 3 also available in App Service for Linux?

We have the following configuration and also getting a 503 error:

  • Azure App Service (Linux)
  • Deployment via Azure Dev Ops Build & Release pipeline
    • Build: dotnet publish (NOT self-contained)
    • Release: Azure App Service deploy

Already tried several re-deploys and restarting the app.

bug_2

bug_1

@timheuer

This comment has been minimized.

Copy link

commented Oct 16, 2019

@stay2be Yes it is. What your describing (using pipelines to build with CLI, release with azure app service deploy) is exactly what I'm doing today on a live site and is working fine. A few questions:

  • What region are you deploying to?
  • In your build pipeline (maybe can you share the yml?) are you building targeting 3.0 SDK?
  • Is your App Service Linux instance using a specific docker image you are providing?
@valeriob

This comment has been minimized.

Copy link

commented Oct 16, 2019

same problem as @stay2be on France central, deploying a framework dependent application on linux

@stay2be

This comment has been minimized.

Copy link

commented Oct 16, 2019

@timheuer

Ok, I'll try to give you a whole picture:

  • Azure Region: West Europe
  • We deploy the compiled project (dotnet publish) directly via App Service Deploy, so I think Docker isn't involved here. At least I haven't configured anything to create a Docker image to be used within the app service. :)
  • The first step in the build pipeline is "Use .NET Core" to select the correct sdk version (in this case 3.0)

Hope the yml helps. Thanks for your quick response!

pool:
  name: Azure Pipelines

steps:
- task: UseDotNet@2
  displayName: 'Use .Net Core sdk 3.0.x'
  inputs:
    version: 3.0.x

- task: DotNetCoreCLI@2
  displayName: Restore
  inputs:
    command: restore
    projects: '$(Parameters.RestoreBuildProjects)'

- task: DotNetCoreCLI@2
  displayName: Build
  inputs:
    projects: '$(Parameters.RestoreBuildProjects)'
    arguments: '--configuration $(BuildConfiguration)'

- task: DotNetCoreCLI@2
  displayName: Test
  inputs:
    command: test
    projects: '$(Parameters.TestProjects)'
    arguments: '--configuration $(BuildConfiguration)'

- task: DotNetCoreCLI@2
  displayName: Publish
  inputs:
    command: publish
    publishWebProjects: True
    arguments: '--configuration $(BuildConfiguration) --output $(build.artifactstagingdirectory) /p:AssemblyVersion=$(Build.BuildNumber)'
    zipAfterPublish: True

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact'
  inputs:
    PathtoPublish: '$(build.artifactstagingdirectory)'
  condition: succeededOrFailed()
@timheuer

This comment has been minimized.

Copy link

commented Oct 16, 2019

@stay2be in your release pipeline what are you using as startup command for your app?

@fabiocav

This comment has been minimized.

Copy link
Member Author

commented Oct 16, 2019

@marchy sincerely, thank you for the feedback! Those are very valid points and we're working closely with the different teams involved to ensure this process is a bit smoother going forward.

For Linux: The deployment is rolling out and the estimated completion date is Friday (10/18). As the deployment progresses, you'll be able to see that option under the Application Configuration in the portal:
config

For Windows: The portal option should be available by 10/25, but this is not a deployment blocker. Runtime has been globally deployed.

We'll continue to provide updates on the SDK deployment once we have validation results.

Thank you again for the patience and all the feedback.

@stay2be

This comment has been minimized.

Copy link

commented Oct 17, 2019

@timheuer The start up command field is empty. As I understood @fabiocav, we just have to wait to get this working.
Annotation 2019-10-17 090230

I was a bit confused, because of the comment above:

@avoxm deployment is completed. We can look at your specific issue to understand what is happening there.

@timheuer

This comment has been minimized.

Copy link

commented Oct 17, 2019

@stay2be -- if you are deploying from DevOps, VIsual Studio, or other automation that already builds your app, you can deploy now. But for your deployment step you need to specify a startup command which is usually mapped to the app in your wwwroot. For example mine is /home/site/wwwroot/TacticView

What @fabiocav was also referring to is the full rollout of the SDK which would allow code-push to App Service and the service builds the app for you as well.

@timheuer

This comment has been minimized.

Copy link

commented Oct 17, 2019

@stay2be mind contacting me off-thread here and we can see if we can get yours working and report back findings here? (just reducing back-and-forth noise of questions) timheuer@microsoft.com

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.