.NET core app does not start when published with /p:PublishSingleFile=true

[Viir]

How can I use publishing of single-file executables?

I tried to follow the instructions from https://docs.microsoft.com/en-us/dotnet/core/whats-new/dotnet-core-3-0#single-file-executables, but this did not work:

Steps to reproduce

• Clone the repository from https://github.com/Viir/bots
• publish the .NET core app in this directory: https://github.com/Viir/bots/tree/82ef37f3089a3e40bd496f507b691a5c25011b33/implement/engine/BotEngine.Windows.Console , using the following command:

dotnet publish -r win10-x64 /p:PublishSingleFile=true

The publish command produced following output in the console:

Microsoft (R) Build Engine version 16.3.0+0f4c62fea for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 85,07 ms for K:\Source\Repos\bots\implement\engine\BotEngine.Windows.Console\BotEngine.Windows.Console.csproj.
  BotEngine.Windows.Console -> K:\Source\Repos\bots\implement\engine\BotEngine.Windows.Console\bin\Debug\netcoreapp3.0\win10-x64\BotEngine.dll
  BotEngine.Windows.Console -> K:\Source\Repos\bots\implement\engine\BotEngine.Windows.Console\bin\Debug\netcoreapp3.0\win10-x64\publish\

• Run the published app on the same machine, by entering the following command in powershell: ./botengine

Expected behavior

I expected the single-file executable app to behave the same way as when published without the /p:PublishSingleFile=true option.
When published without the /p:PublishSingleFile=true option, the apps output starts as follows:

Please specify a subcommand.
BotEngine console version 2019-09-01
[...]

Actual behavior

When trying to start the app, it outputs the following message:

Error:
  An assembly specified in the application dependencies manifest (BotEngine.deps.json) was not found:
    package: 'LibGit2Sharp.NativeBinaries', version: '2.0.267'
    path: 'runtimes/win-x64/native/git2-572e4d8.pdb'

and then exits.

Environment data

dotnet --info output:

.NET Core SDK (reflecting any global.json):
 Version:   3.0.100
 Commit:    04339c3a26

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18362
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.0.100\

Host (useful for support):
  Version: 3.0.0
  Commit:  7d57652f33

.NET Core SDKs installed:
  1.0.0-preview4-004233 [C:\Program Files\dotnet\sdk]
  1.0.0-rc4-004771 [C:\Program Files\dotnet\sdk]
  1.0.4 [C:\Program Files\dotnet\sdk]
  1.1.0 [C:\Program Files\dotnet\sdk]
  2.0.0-preview2-006497 [C:\Program Files\dotnet\sdk]
  2.0.1-servicing-006957 [C:\Program Files\dotnet\sdk]
  2.1.4 [C:\Program Files\dotnet\sdk]
  2.1.103 [C:\Program Files\dotnet\sdk]
  2.1.201 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.300 [C:\Program Files\dotnet\sdk]
  2.1.403 [C:\Program Files\dotnet\sdk]
  2.1.500 [C:\Program Files\dotnet\sdk]
  2.1.502 [C:\Program Files\dotnet\sdk]
  2.1.503 [C:\Program Files\dotnet\sdk]
  2.1.504 [C:\Program Files\dotnet\sdk]
  2.1.505 [C:\Program Files\dotnet\sdk]
  2.1.602 [C:\Program Files\dotnet\sdk]
  2.1.604 [C:\Program Files\dotnet\sdk]
  2.1.700 [C:\Program Files\dotnet\sdk]
  2.1.701 [C:\Program Files\dotnet\sdk]
  2.2.101 [C:\Program Files\dotnet\sdk]
  2.2.104 [C:\Program Files\dotnet\sdk]
  2.2.202 [C:\Program Files\dotnet\sdk]
  2.2.204 [C:\Program Files\dotnet\sdk]
  2.2.300 [C:\Program Files\dotnet\sdk]
  2.2.301 [C:\Program Files\dotnet\sdk]
  3.0.100 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.0-preview2-25407-01 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.2-servicing-25708-01 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

[dasMulli]

cc @swaroop-sridhar

[Viir]

Reading in the document at https://github.com/dotnet/designs/blob/master/accepted/single-file/design.md : I found this paragraph interesting:

By default, the symbol files are not embedded within the single-file, but remain as separate files in the publish directory. This includes both the IL .pdb file, and the native .ni.pdb / app.guid.map files generated by ready-to-run compiler. Setting the following property causes the symbol files to be included in the single-file.

I found this interesting because the error message from the published app (quoted above) mentions a file with a name ending in .pdb. Here the error message again:

Error:
  An assembly specified in the application dependencies manifest (BotEngine.deps.json) was not found:
    package: 'LibGit2Sharp.NativeBinaries', version: '2.0.267'
    path: 'runtimes/win-x64/native/git2-572e4d8.pdb'

But, on the other hand, the linked document also contains following:

Setting the PublishSingleFileproperty causes the managed app, managed dependencies, platform-specific native dependencies, configurations, etc. (basically the contents of the publish directory when dotnet publish is run without setting the property) to be embedded within the native apphost.

So the git2-572e4d8.pdb should be also included? When I run publish without single exe-file option, I find git2-572e4d8.pdb in the publish directory.

[dasMulli]

I've seen one or two other reports about errors when RID-specific content is involved together with single-file publishing.

the deps.json shouldn't contain links to runtimes/* folders when the app is published rid-specific, which is the case for single-file apps.

cc @dsplaisted @nguerrera

[dasMulli]

Adding

<PropertyGroup>
    <IncludeSymbolsInSingleFile>true</IncludeSymbolsInSingleFile>
</PropertyGroup>

Could help in your situation, but I'm not sure and even if, it shouldn't be necessary in the first place.

[Viir]

I added this to the .csproj file: Viir/bots@30e7533 (https://github.com/Viir/bots/tree/adapt-for-single-file-exe-publish)

I confirm this helps, at least the app seems to start normally. I did not find a problem with the apps behaviour when published from that version.

Adding the IncludeSymbolsInSingleFile resulted in the executable file coming out around 7 MB larger (107.599 kB instead of 100.327 kB). Would be nice if would not need this workaround.

[swaroop-sridhar]

@Viir: This problem here is that the app's definition (as dictated by the deps.json manifest) requires the presence of the runtimes/win-x64/native/git2-572e4d8.pdb file. The .net core host is simply checking that we have all the files required by the manifest, and fails if it doesn't find the PDB embedded within the single-file app.

The correct fix is to not have the PDB file required by deps.json. This dependency likely comes in because of one of the dependencies of the apps (via nuget packages).

This issue is similar to PowerShell/PowerShell#10266

From the host point of view, this bug is a dup of dotnet/core#3455 (comment).

[swaroop-sridhar]

CC: @dsplaisted

[nguerrera]

It is strange that pdbs would ever be in deps.json like that. I gues nuget thinks this is a native library?

[dasMulli]

if I understand the linked issues correctly powershell removes their PDBs.

I don't think that files in native folders should be assumed to be of special type.. it is valid that PDBs shouldn't be runtime libs trying to be loaded by the host. But also removing them from the publish (single file) seems off since the NuGet package should be free to include whatever is necessary - that may be symbols for native libs (pdb,dsym,..) for stack traces or just txt/dat files.

[swaroop-sridhar]

@dasMulli the exclusion of PDB is just a default, and can be overridden by IncludeSymbolsInSingleFile property. I think if we reverse the default, most apps will build with ExcludeSymbolsFromSingleFile setting (or silently take a size penalty without noticing it).

I'm not sure if there is anything else actionable for this issue.

[Viir]

@swaroop-sridhar thank you for the quick clarification.

Does this mean the dotnet publish command should output a message to inform us that the executable we just produced will never work?

So far, I did not find such a warning in the output from dotnet publish.

[swaroop-sridhar]

Sure, the since the SDK generates the deps.json file, it is possible to generate a warning if PDBs are included in deps.json files.

It is also possible to adapt the default to include the required PDBs in the single-file -- but this will silently hide a likely bug in one of the dependent packages and make the app larger unnecessarily.

I filed #3685 to track this.

[livarcocc]

Closing this issue as there is a sdk issue tracking the action item from this thread.