IIS requests hang after closing Visual Studio with Development Time IIS Support (outOfProcess)

[Grabauskas]

Description

All requests from IIS will be hung if the console was flooded with messages.
This issue only occurs when running with IIS without Visual Studio. And it looks like because there is no active console.
Seems like Console Logger has a queue and if it's overcrowded then all requests will be hung.

To Reproduce

1. Create a new ASP.NET web application with API project template.
2. Set LogLevel to Trace in Development Environment.
   appsettings.Development.json

   {
     "Logging": {
       "LogLevel": {
         "Default": "Trace",
         "Microsoft": "Trace",
         "Microsoft.Hosting.Lifetime": "Trace"
       }
     }
   }

3. Create IIS launch profile.
   launchSettings.json

   {
     "$schema": "http://json.schemastore.org/launchsettings.json",
     "iisSettings": {
       "windowsAuthentication": false,
       "anonymousAuthentication": true,
       "iis": {
         "applicationUrl": "https://localhost/Demo",
         "sslPort": 0
       }
     },
     "profiles": {
       "IIS": {
         "commandName": "IIS",
         "launchBrowser": true,
         "environmentVariables": {
           "ASPNETCORE_ENVIRONMENT": "Development"
         },
         "ancmHostingModel": "OutOfProcess"
       }
     }
   }

4. Build and Deploy to IIS (Start without Debuging).
5. Close Visual Studio.
6. Repeatedly call endpoint.
   request.bat

   @echo off
   set /a a=1
   :loop
   set /a a+=1
   curl https://localhost/Demo/weatherforecast
   echo %a%
   goto loop

Further technical details

• dotnet --info

.NET Core SDK (reflecting any global.json):
 Version:   3.1.101
 Commit:    b377529961

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.101\

Host (useful for support):
  Version: 3.1.1
  Commit:  a1388f194c

.NET Core SDKs installed:
  2.2.207 [C:\Program Files\dotnet\sdk]
  2.2.401 [C:\Program Files\dotnet\sdk]
  3.0.100 [C:\Program Files\dotnet\sdk]
  3.0.101 [C:\Program Files\dotnet\sdk]
  3.1.100 [C:\Program Files\dotnet\sdk]
  3.1.101 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

• Using:
  • ASP.NET Core 3.1
  • Visual Studio 16.4.3
  • ASP.NET and web development
    • Development Time IIS Support

[analogrelay]

@Grabauskas can you post your repro app? It would help ensure we are looking at the same code in the same context. For example, the ancmHostingModel field in launchSettings.json doesn't apply to published apps, it's only about F5 launches.

There is definitely a queue in the Console Logger that can cause a backlog, but ANCM should be flushing the console regularly.

[Grabauskas]

This issue only occurs in the Development Environment.

I created a demo repository: Grabauskas/dotnet-aspnetcore-issues-18850
And created gif: demo.gif

[zduoba]

After some debugging. I found out that after few bombardments of requests Interop.Kernel32.WriteFile takes a lot of time to process.
Link to CodeFile
Interop.Kernel32.WriteFile(hFile, p + offset, count, out numBytesWritten, IntPtr.Zero)

[shirhatti]

The only in IIS and without Visual Studio is a red herring here.

Unfortunately, this is expected behavior with the console logger when the volume of logging is too high. If you really need that level of tracing in production you might be better off using a file logger. (Try Serilog)