Android Release mode build failure

[elix22]

Release Type: Official Release

Version: 3.0.0.4

Describe the bug
Compilation failure in Android release mode

To Reproduce
Steps to reproduce the behavior:

1. Open the project in Visual studio 2017
2. Choose Solution Platform to be Android
3. Choose Solution Configuration to be Release
4. Press F7 to build the solution (or choose build solution from the Build menu)

Expected behavior
The build should pass

Log and callstacks
build_log.txt

Additional context
Issue is not observed on Xenko version 2.1.2.1 , meaning the compilation is passing in Android/Release configuration

[xen2]

Duplicate of #127
However, since this one using the new issue template, I will keep it open and close the other one.

[aurodev]

Xenko version: 3.0.0.5
We still can not get a project to build for Android in Release mode in VS2017 and it presents with the following output(s) for Jumpy Jet (Debug mode builds and deploys from VS2017 for us)

Also the same behavior with 3.0.0.6-dev including the System.Xml.dll access error.

1>------ Build started: Project: JumpyJet.Android, Configuration: Release Android ------
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\Properties\AndroidManifest.xml : warning XA0101: @(Content) build action is not supported
1>  Assembly [JumpyJet, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] has already been processed, skip it.
1>  JumpyJet.Android -> C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Bin\Android\Release\JumpyJet.dll
1>  info 0.000s: [AssetCompiler] BuildEngine arguments: --disable-auto-compile --project-configuration Release --platform=Android --profile=Android --project-configuration=Release --output-path=C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\obj\Android\Release\assets\data --build-path=C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\..\Cache\data --package-file=C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\..\JumpyJet.xkpkg --log-pipe=
1>  info 0.000s: [AssetCompiler] Starting builder.
1>EXEC : error 0.885s: [AssetCompiler] Unable to build assembly reference [C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Game\bin\Debug\net461\JumpyJet.Game.dll]
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\GraphicsCompositor.xkgfxcomp(103,13): warning 0.885s: [AssetCompiler] Could not deserialize object of type !JumpyJet.JumpyJetRenderer,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 102, Col: 12, Chr: 6040) - (Lin: 103, Col: 8, Chr: 6090): Unable to resolve tag [!JumpyJet.JumpyJetRenderer,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Scene.xkscene(25,55): warning 0.885s: [AssetCompiler] Could not deserialize object of type !JumpyJet.UIScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 24, Col: 54, Chr: 1015) - (Lin: 25, Col: 24, Chr: 1073): Unable to resolve tag [!JumpyJet.UIScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Character.xkprefab(24,55): warning 0.886s: [AssetCompiler] Could not deserialize object of type !JumpyJet.CharacterScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 23, Col: 54, Chr: 1098) - (Lin: 24, Col: 24, Chr: 1163): Unable to resolve tag [!JumpyJet.CharacterScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Scene.xkscene(63,55): warning 0.886s: [AssetCompiler] Could not deserialize object of type !JumpyJet.BackgroundScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 62, Col: 54, Chr: 3132) - (Lin: 63, Col: 24, Chr: 3198): Unable to resolve tag [!JumpyJet.BackgroundScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Scene.xkscene(88,55): warning 0.886s: [AssetCompiler] Could not deserialize object of type !JumpyJet.PipesScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 87, Col: 54, Chr: 4502) - (Lin: 88, Col: 24, Chr: 4563): Unable to resolve tag [!JumpyJet.PipesScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Scene.xkscene(105,55): warning 0.886s: [AssetCompiler] Could not deserialize object of type !JumpyJet.CharacterScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 104, Col: 54, Chr: 5506) - (Lin: 105, Col: 24, Chr: 5571): Unable to resolve tag [!JumpyJet.CharacterScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>  RunTime 00:00:03.18
1>C:\Users\Greg\.nuget\packages\xenko\3.0.0.5\Targets\Xenko.targets(374,5): error MSB3073: The command ""C:\Users\Greg\.nuget\packages\xenko\3.0.0.5\Bin\Windows\Xenko.Core.Assets.CompilerClient.exe"  --disable-auto-compile --project-configuration "Release" --platform=Android --profile=Android --project-configuration=Release --output-path="C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\obj\Android\Release\assets\data" --build-path="C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\..\Cache\data" --package-file="C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\..\JumpyJet.xkpkg" --log-pipe=""" exited with code 1.
========== Build: 0 succeeded, 1 failed, 1 up-to-date, 0 skipped ==========

1>------ Build started: Project: JumpyJet.Android, Configuration: Release Android ------
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\Properties\AndroidManifest.xml : warning XA0101: @(Content) build action is not supported
1>  Assembly [JumpyJet, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] has already been processed, skip it.
1>  JumpyJet.Android -> C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Bin\Android\Release\JumpyJet.dll
1>  info 0.000s: [AssetCompiler] BuildEngine arguments: --disable-auto-compile --project-configuration Release --platform=Android --profile=Android --project-configuration=Release --output-path=C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\obj\Android\Release\assets\data --build-path=C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\..\Cache\data --package-file=C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\..\JumpyJet.xkpkg --log-pipe=
1>  info 0.000s: [AssetCompiler] Starting builder.
1>EXEC : error 0.931s: [AssetCompiler] Unable to build assembly reference [C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Game\bin\Debug\net461\JumpyJet.Game.dll]
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Character.xkprefab(24,55): warning 0.931s: [AssetCompiler] Could not deserialize object of type !JumpyJet.CharacterScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 23, Col: 54, Chr: 1098) - (Lin: 24, Col: 24, Chr: 1163): Unable to resolve tag [!JumpyJet.CharacterScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Scene.xkscene(25,55): warning 0.931s: [AssetCompiler] Could not deserialize object of type !JumpyJet.UIScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 24, Col: 54, Chr: 1015) - (Lin: 25, Col: 24, Chr: 1073): Unable to resolve tag [!JumpyJet.UIScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\GraphicsCompositor.xkgfxcomp(103,13): warning 0.931s: [AssetCompiler] Could not deserialize object of type !JumpyJet.JumpyJetRenderer,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 102, Col: 12, Chr: 6040) - (Lin: 103, Col: 8, Chr: 6090): Unable to resolve tag [!JumpyJet.JumpyJetRenderer,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Scene.xkscene(63,55): warning 0.931s: [AssetCompiler] Could not deserialize object of type !JumpyJet.BackgroundScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 62, Col: 54, Chr: 3132) - (Lin: 63, Col: 24, Chr: 3198): Unable to resolve tag [!JumpyJet.BackgroundScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Scene.xkscene(88,55): warning 0.931s: [AssetCompiler] Could not deserialize object of type !JumpyJet.PipesScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 87, Col: 54, Chr: 4502) - (Lin: 88, Col: 24, Chr: 4563): Unable to resolve tag [!JumpyJet.PipesScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\Assets\Shared\Scene.xkscene(105,55): warning 0.931s: [AssetCompiler] Could not deserialize object of type !JumpyJet.CharacterScript,JumpyJet.Game; replacing it with an object implementing IUnloadable:
1>  (Lin: 104, Col: 54, Chr: 5506) - (Lin: 105, Col: 24, Chr: 5571): Unable to resolve tag [!JumpyJet.CharacterScript,JumpyJet.Game] to type from tag resolution or registered assemblies
1>  RunTime 00:00:03.10
1>C:\Users\Greg\.nuget\packages\xenko\3.0.0.5\Targets\Xenko.targets(374,5): error MSB3073: The command ""C:\Users\Greg\.nuget\packages\xenko\3.0.0.5\Bin\Windows\Xenko.Core.Assets.CompilerClient.exe"  --disable-auto-compile --project-configuration "Release" --platform=Android --profile=Android --project-configuration=Release --output-path="C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\obj\Android\Release\assets\data" --build-path="C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\..\Cache\data" --package-file="C:\Users\Greg\Documents\Xenko_Projects\JumpyJet\JumpyJet\JumpyJet.Android\..\JumpyJet.xkpkg" --log-pipe=""" exited with code 1.
========== Build: 0 succeeded, 1 failed, 1 up-to-date, 0 skipped ==========

Comments
The forums aren't very active or responsive right now so I'll add this opinion here. The build system and project file issues are, in our opinion, one of the highest priorities, even higher than bugs with existing features.

If people can get this thing building for all of it's targets, debug + release, then the community will be a lot more motivated to help. Very little is more discouraging to someone new to a tool or API than complex and (sometimes) indecipherable Microsoft build errors.

Xenko won't succeed if the only entity that can build it is an automated build system on github.

@xen2
@Kryptos-FR
@elix22

[aurodev]

Perhaps @migueldeicaza can offer some insight? :)

[xwellingtonx]

In the last days I'm debugging this issue and I think that I find the bug.

The TrackExecutablePath method runs in another thread that maintain the Exec Server alive even all tasks were done it's just close only after 600 seconds. While It's alive the Xamarin try to build the project and fails with the exception showed above.

https://github.com/xenko3d/xenko/blob/84c7810b9cdf2d1345f1bbbc4ad5f2f0aef3c184/sources/tools/Xenko.ExecServer/ExecServerRemote.cs#L114-L154

Unfortunately I do not understand all the ExecServer code to fix this issue. If someone can fix it from this informations let me know!

One solution that's a find it's to change the Linking option to None in Android Project -> Properties-> Android Options -> Linker Options.
**This will cause that your package gets bigger than usual for Release, you could see more about this on Linker Documentation.

@xen2
@Kryptos-FR
@elix22
@aurodev

[xen2]

few explanations about execserver:

It's a process that stays alive to avoid full appdomain load + JIT every time we compile asset.

• When running from dev directory it uses shadow caching to avoid locking assemblies
• When running in official xenko install it doesn't to be faster (only for Xenko assemblies, but game Assemblies are still supposed to be loaded through Assembly.Load(byte[]))

Maybe a game assembly is loaded differently and file is locked?

If you want to skip using execserver, there is an option:
<DisableAssetCompilerExecServerProxy>true</DisableAssetCompilerExecServerProxy> in executable csproj. It will give you a quick idea if execserver is the issue or not there

Note: with recent changes happening with #12 and #58, I might have to temporarily disable (and have slightly slower compile) or revisit ExecServer at some point

[xwellingtonx]

@xen2 Unfortunately even ignoring the execserver the bug still happens. I will look further after the release of version 3.1.

[aurodev]

@xwellingtonx Good job tracking this down! 👍