Hosting multiple instances of an app published with PublishSingleFile and IncludeNativeLibrariesForSelfExtract does not work as expected.

[qcc-na]

Description

When compiling a web project to a single file application with IncludeNativeLibrariesForSelfExtract you can only run one instance of the application in IIS. If you try to run a second instance, it will try to extract from the same directory and fail to do so. This is contrary to the documented behavior I've read in one of dotnet's design documents. Specifically the linked document says,

Check if the [extraction directory](Extraction Location) exists for this app:
If not, perform a new extraction:
Extract appropriate files to a temporary location // . This ensures that concurrent launches of the app do not interfere with each other.

This isn't the case in my experience.

If you remove the IncludeNativeLibrariesForSelfExtract option this problem goes away.

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <GenerateAssemblyInfo>false</GenerateAssemblyInfo>
    <AspNetCoreHostingModel>OutOfProcess</AspNetCoreHostingModel>
    <PublishSingleFile>true</PublishSingleFile>
    <SelfContained>true</SelfContained>
    <RuntimeIdentifier>win-x64</RuntimeIdentifier>
    <PublishReadyToRun>true</PublishReadyToRun>
    <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract>
  </PropertyGroup>

Reproduction Steps

0. Open Visual Studio Version 16.10.3

1. File -> New -> Project -> ASP NET Core Web App -> Next -> .Net 5.0 -> Create

2. Paste this into the project file.

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <GenerateAssemblyInfo>false</GenerateAssemblyInfo>
    <AspNetCoreHostingModel>OutOfProcess</AspNetCoreHostingModel>
    <PublishSingleFile>true</PublishSingleFile>
    <SelfContained>true</SelfContained>
    <RuntimeIdentifier>win-x64</RuntimeIdentifier>
    <PublishReadyToRun>true</PublishReadyToRun>
    <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract>
  </PropertyGroup>

3. Folder publish
4. Copy to c:\inetpub\wwwroot\testproject1
5. Copy to c:\inetpub\wwwroot\testproject2
6. Convert each of these to web application in their own application pool
7. Attempt to view each of these applications
8. One will succeed and one will show a hosting error.
9. Remove <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract> from the project file.
10. Republish to respective directories
11. Both applications work

Expected behavior

The expected behavior is for both instances of the application to work properly when <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract> is in the project file prior to publishing.

Actual behavior

The actual behavior is only a single instance working properly when <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract> is included in the project file prior to publishing.

Regression?

I do not know if this worked in prior releases as this was my first attempt.

Known Workarounds

Removing <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract> from the project file prior to publishing enables both applications to work but does NOT produce a single file build output.

Renaming one of the .exe files and it's key in the associated web.config file allows this to work.

Configuration

Visual Studio:
Microsoft Visual Studio Professional 2019
Version 16.10.3

OS:
Edition Windows 10 Pro
Version 21H1
Installed on ‎5/‎20/‎2021
OS build 19043.1237
Experience Windows Feature Experience Pack 120.2212.3530.0

Other information

No response

Tagging subscribers to this area: @agocke, @vitek-karas, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

When compiling a web project to a single file application with IncludeNativeLibrariesForSelfExtract you can only run one instance of the application in IIS. If you try to run a second instance, it will try to extract from the same directory and fail to do so. This is contrary to the documented behavior I've read in one of dotnet's design documents. Specifically the linked document says,

Check if the [extraction directory](Extraction Location) exists for this app:
If not, perform a new extraction:
Extract appropriate files to a temporary location // . This ensures that concurrent launches of the app do not interfere with each other.

This isn't the case in my experience.

If you remove the IncludeNativeLibrariesForSelfExtract option this problem goes away.

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <GenerateAssemblyInfo>false</GenerateAssemblyInfo>
    <AspNetCoreHostingModel>OutOfProcess</AspNetCoreHostingModel>
    <PublishSingleFile>true</PublishSingleFile>
    <SelfContained>true</SelfContained>
    <RuntimeIdentifier>win-x64</RuntimeIdentifier>
    <PublishReadyToRun>true</PublishReadyToRun>
    <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract>
  </PropertyGroup>

Reproduction Steps

0. Open Visual Studio Version 16.10.3

1. File -> New -> Project -> ASP NET Core Web App -> Next -> .Net 5.0 -> Create

2. Paste this into the project file.

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <GenerateAssemblyInfo>false</GenerateAssemblyInfo>
    <AspNetCoreHostingModel>OutOfProcess</AspNetCoreHostingModel>
    <PublishSingleFile>true</PublishSingleFile>
    <SelfContained>true</SelfContained>
    <RuntimeIdentifier>win-x64</RuntimeIdentifier>
    <PublishReadyToRun>true</PublishReadyToRun>
    <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract>
  </PropertyGroup>

3. Folder publish
4. Copy to c:\inetpub\wwwroot\testproject1
5. Copy to c:\inetpub\wwwroot\testproject2
6. Convert each of these to web application in their own application pool
7. Attempt to view each of these applications
8. One will succeed and one will show a hosting error.
9. Remove <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract> from the project file.
10. Republish to respective directories
11. Both applications work

Expected behavior

The expected behavior is for both instances of the application to work properly when <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract> is in the project file prior to publishing.

Actual behavior

The actual behavior is only a single instance working properly when <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract> is included in the project file prior to publishing.

Regression?

I do not know if this worked in prior releases as this was my first attempt.

Known Workarounds

Removing <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract> from the project file prior to publishing enables both applications to work but does NOT produce a single file build output.

Renaming one of the .exe files and it's key in the associated web.config file allows this to work.

Configuration

Visual Studio:
Microsoft Visual Studio Professional 2019
Version 16.10.3

OS:
Edition Windows 10 Pro
Version 21H1
Installed on ‎5/‎20/‎2021
OS build 19043.1237
Experience Windows Feature Experience Pack 120.2212.3530.0

Other information

No response

Author:	qcc-na
Assignees:	-
Labels:	area-Single-File, untriaged
Milestone:	-

[agocke]

@VSadov could you take a look?

[VSadov]

I will take a look.

[VSadov]

The extraction mechanism did not change much, so it may not be a new issue, just a new scenario.

@qcc-na - Is this with 5.0?
What kind of error is produced?

[qcc-na]

HTTP Error 502.5 - ANCM Out-Of-Process Startup Failure
Common solutions to this issue:
The application process failed to start
The application process started but then stopped
The application process started but failed to listen on the configured port
Troubleshooting steps:
Check the system event log for error messages
Enable logging the application process' stdout messages
Attach a debugger to the application process and inspect
For more information visit: https://go.microsoft.com/fwlink/?LinkID=808681

Attached Event Logs (cleared before accessing error page):
eventlog.zip

If this file isn't sufficient let me know and I can start pasting the logs in here.

Clarification: Yes .net 5.0

[qcc-na]

@VSadov Any info?

[revuniversal]

Any word on this?

[VSadov]

I will try to take a look today. It is a very strange error.
You can certainly run multiple copies of a singlefile app. Something must be different for how web apps are being run.

[VSadov]

If I run two copies of the same webapp as-is (without IIS, which I do not have on my machine), I do not see any issues.

Both apps run and the native files are extracted into a temp folder that corresponds to the app
%TEMP%\.net\WebApplication1\nreshoqb.fjm in my case

There is nothing special to web apps when running in this mode.
I will try reproducing with IIS, but it may take time since I am not very familiar with IIS.

I wonder - can it be some permissions conflict? How does IIS runs application pools? Do they run with limited permissions or under some special user?

[VSadov]

Another thing to try is to set environment variable COREHOST_TRACE to 1 and capture the output.

If you can do this faster than me, it could be helpful with figuring what is going on.

I suspect it is not a problem with the singlefile model itself, but with the app model in IIS - like the first app extracts files, but the next one does not have permissions to use them.

[qcc-na]

I changed the account in both AppPools to "LocalSystem" and both instances were able to load. I think you are correct about this being a permissions issue. Do you think it is an issue that 2 separate instances are extracting to the same folder? The document seems to indicate that process-id is involved but it doesn't seem like it is. Is it the process id of IIS?

[VSadov]

Do you think it is an issue that 2 separate instances are extracting to the same folder?

That is how it should work. Once the files are extracted, they will be reused if the app runs again or if another copy of the app is launched.

The extraction location is specific to the application and is under %TEMP%, which is different for different users, so there are no problems with permissions. Different users see different extraction directories.
Under IIS it seems permissions may be configured in various ways including to not match the user account that starts the process and that could cause issues with extraction.

I do not think this is actionable on the singlefile app model side, but may be worth mentioning in documentation as what can happen under IIS and what can be done about that.