New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bundled app initialization error (vs 15.7 preview) #1651

Closed
tranb3r opened this Issue May 7, 2018 · 38 comments

Comments

Projects
None yet
@tranb3r

tranb3r commented May 7, 2018

Steps to Reproduce

  1. Create a xamarin app for Android
  2. Enable "Bundle Assemblies into Native Code" option : true in csproj
  3. Build and launch app in VS 2017 15.7 preview

Expected Behavior

No crash.
This is actually what happens with VS 2017 15.6.7

Actual Behavior

App crashes (when using VS 2017 15.7 preview) with the following error:
bundled app initialization error: dlopen failed: cannot locate symbol "mono_jit_set_aot_mode" referenced by libmonodroid_bundle_app.so

Version Information (causing the error)

Version 15.7.0 Preview 5.0
VisualStudio.15.Preview/15.7.0-pre.5.0+27625.0
Mono Debugging for Visual Studio 4.10.5-pre (ab58725)
Xamarin 4.10.0.440 (943b34d9d)
Xamarin.Android SDK 8.3.0.18 (HEAD/1f8462ef1)

Version Information (no error)

Version 15.6.7
VisualStudio.15.Release/15.6.7+27428.2043
Mono Debugging for Visual Studio 4.9.11-pre (71eb098)
Xamarin 4.9.0.753 (f0f46392f)
Xamarin.Android SDK 8.2.0.16 (HEAD/a78295902)

VS bug #618947

@whitwa

This comment has been minimized.

whitwa commented May 9, 2018

This is also happening with the release of version 15.7.0.

Microsoft Visual Studio Enterprise 2017 Version 15.7.0
VisualStudio.15.Release/15.7.0+27703.1
Mono Debugging for Visual Studio 4.10.5-pre (ab58725)
Xamarin 4.10.0.442 (396b18cef)
Xamarin.Android SDK 8.3.0.19 (HEAD/342b2ce96)

@RajGogri

This comment has been minimized.

RajGogri commented May 9, 2018

This is also happening with the release of VS 2017 community edition. It works fine for the debug build but produces error while running release build.

Setting Enable "Bundle Assemblies into Native Code" option : false in csproj for Release configuration group fixes this issue. But It's a work around.

Development environment info.
Visual Studio Community 2017 for Mac
Version 7.5 (build 1254)
Installation UUID: e1436f78-c6f3-45bb-883a-d0fd263fac97
Runtime:
Mono 5.10.1.47 (2017-12/8eb8f7d5e74) (64-bit)
GTK+ 2.24.23 (Raleigh theme)
Xamarin.Mac 4.4.0.36 (master / 0c7c49a6)

Package version: 510010047

NuGet
Version: 4.3.1.4445

.NET Core
Runtime: /usr/local/share/dotnet/dotnet
Runtime Version: 2.0.5
SDK: /usr/local/share/dotnet/sdk/2.1.4/Sdks
SDK Version: 2.1.4
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.10.1/lib/mono/msbuild/15.0/bin/Sdks

Xamarin.Profiler
Version: 1.6.2
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

Apple Developer Tools
Xcode 9.3 (14154)
Build 9E145

Xamarin.Mac
Version: 4.4.1.176 (Visual Studio Community)

Xamarin.iOS
Version: 11.10.1.177 (Visual Studio Community)
Hash: 7e782c1e
Branch: d15-7
Build date: 2018-04-25 15:27:13-0400

Xamarin.Android
Version: 8.3.0.19 (Visual Studio Community)
Android SDK: /Users/webwerks/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
5.1 (API level 22)
6.0 (API level 23)
7.0 (API level 24)
7.1 (API level 25)
8.0 (API level 26)
8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 27.0.1
SDK Build Tools Version: 28.0.0 rc1

Java SDK: /usr
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

Xamarin Inspector
Version: 1.4.0
Hash: b3f92f9
Branch: master
Build date: Fri, 19 Jan 2018 22:00:34 GMT
Client compatibility: 1

Build Information
Release ID: 705001254
Git revision: 498923ea36d2c7fe440c4e4b8cfb75bd50bbd748
Build date: 2018-05-05 10:35:24-04
Xamarin addins: 219f1c4943b4693b837b4173dd10ea982a47c852
Build lane: monodevelop-lion-d15-7

Operating System
Mac OS X 10.13.4
Darwin 17.5.0 Darwin Kernel Version 17.5.0
Fri Apr 13 19:32:32 PDT 2018
root:xnu-4570.51.2~1/RELEASE_X86_64 x86_64

@jgold6

This comment has been minimized.

jgold6 commented May 10, 2018

I received another report of this and was able to reproduce both on Visual Studio 2017 Windows and VS for Mac (all updated to 15.7) with a template Xamarin.Forms app. I am attaching adb logcat log that show the error and a test project.

VS 2017 Windows version info: Microsoft Visual Studio Enterprise 2017 Version 15.7.1 VisualStudio.15.Release/15.7.1+27703.2000 Microsoft .NET Framework Version 4.7.03056

Installed Version: Enterprise

Architecture Diagrams and Analysis Tools 00369-90253-02232-AA699
Microsoft Architecture Diagrams and Analysis Tools

Visual C++ 2017 00369-90253-02232-AA699
Microsoft Visual C++ 2017

Application Insights Tools for Visual Studio Package 8.12.10405.1
Application Insights Tools for Visual Studio

ASP.NET and Web Tools 2017 15.0.40501.0
ASP.NET and Web Tools 2017

ASP.NET Core Razor Language Services 15.7.31476
Provides languages services for ASP.NET Core Razor.

ASP.NET Web Frameworks and Tools 2012 4.0.21208.0
For additional information, visit https://www.asp.net/

ASP.NET Web Frameworks and Tools 2017 5.2.60419.0
For additional information, visit https://www.asp.net/

Azure App Service Tools v3.0.0 15.0.40424.0
Azure App Service Tools v3.0.0

Azure Data Lake Node 1.0
This package contains the Data Lake integration nodes for Server Explorer.

Azure Data Lake Tools for Visual Studio 2.3.3000.5
Microsoft Azure Data Lake Tools for Visual Studio

Azure Functions and Web Jobs Tools 15.0.40502.0
Azure Functions and Web Jobs Tools

Azure Stream Analytics Tools for Visual Studio 2.3.3000.5
Microsoft Azure Stream Analytics Tools for Visual Studio

C# Tools 2.8.0-beta6-62830-08. Commit Hash: e595ee276d14e14bfb3eb323fb57f2aa668bddea
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Common Azure Tools 1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Fabric.DiagnosticEvents 1.0
Fabric Diagnostic Events

GitHub.VisualStudio 2.5.1.2234
A Visual Studio Extension that brings the GitHub Flow into Visual Studio.

JavaScript Language Service 2.0
JavaScript Language Service

JavaScript Project System 2.0
JavaScript Project System

JavaScript UWP Project System 2.0
JavaScript UWP Project System

Merq 1.1.17-rc (cba4571)
Command Bus, Event Stream and Async Manager for Visual Studio extensions.

Microsoft Azure HDInsight Azure Node 2.3.3000.5
HDInsight Node under Azure Node

Microsoft Azure Hive Query Language Service 2.3.3000.5
Language service for Hive query

Microsoft Azure Service Fabric Tools for Visual Studio 2.1
Microsoft Azure Service Fabric Tools for Visual Studio

Microsoft Azure Stream Analytics Language Service 2.3.3000.5
Language service for Azure Stream Analytics

Microsoft Azure Stream Analytics Node 1.0
Azure Stream Analytics Node under Azure Node

Microsoft Azure Tools 2.9
Microsoft Azure Tools for Microsoft Visual Studio 2017 - v2.9.10420.2

Microsoft Continuous Delivery Tools for Visual Studio 0.3
Simplifying the configuration of continuous build integration and continuous build delivery from within the Visual Studio IDE.

Microsoft JVM Debugger 1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft MI-Based Debugger 1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual C++ Wizards 1.0
Microsoft Visual C++ Wizards

Microsoft Visual Studio Tools for Containers 1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.

Microsoft Visual Studio VC Package 1.0
Microsoft Visual Studio VC Package

Mono Debugging for Visual Studio 4.10.5-pre (ab58725)
Support for debugging Mono processes with Visual Studio.

NuGet Package Manager 4.6.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

OptionsPackage Extension 1.0
OptionsPackage Visual Studio Extension Detailed Info

ProjectServicesPackage Extension 1.0
ProjectServicesPackage Visual Studio Extension Detailed Info

ResourcePackage Extension 1.0
ResourcePackage Visual Studio Extension Detailed Info

SQL Server Data Tools 15.1.61804.210
Microsoft SQL Server Data Tools

ToolWindowHostedEditor 1.0
Hosting json editor into a tool window

TypeScript Tools 15.7.20419.2003
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools 2.8.0-beta6-62830-08. Commit Hash: e595ee276d14e14bfb3eb323fb57f2aa668bddea
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools 10.1 for F# 4.1 15.7.0.0. Commit Hash: 16ecf5a30ad868d183c58e4a71a71c23d4ed3ba9.
Microsoft Visual F# Tools 10.1 for F# 4.1

Visual Studio Code Debug Adapter Host Package 1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Tools for Universal Windows Apps 15.0.27703.01
The Visual Studio Tools for Universal Windows apps allow you to build a single universal app experience that can reach every device running Windows 10: phone, tablet, PC, and more. It includes the Microsoft Windows 10 Software Development Kit.

VisualStudio.Mac 1.0
Mac Extension for Visual Studio

Windows Machine Learning Generator Extension 1.0
Windows Machine Learning Visual Studio Extension Detailed Info

Xamarin 4.10.0.442 (396b18cef)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer 4.12.264 (fc37cd02e)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin.Android SDK 8.3.0.19 (HEAD/342b2ce96)
Xamarin.Android Reference Assemblies and MSBuild support.

Xamarin.iOS and Xamarin.Mac SDK 11.10.1.177 (7e782c1)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.


VS for Mac version info: === Visual Studio Enterprise 2017 for Mac ===

Version 7.5 (build 1254)
Installation UUID: f86726f2-bd5d-4610-867e-44e82f306ca2
Runtime:
Mono 5.10.1.47 (2017-12/8eb8f7d5e74) (64-bit)
GTK+ 2.24.23 (Raleigh theme)
Xamarin.Mac 4.4.0.36 (master / 0c7c49a6)

Package version: 510010047

=== NuGet ===

Version: 4.3.1.4445

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
2.0.5
2.0.0
1.1.1
1.0.4
1.0.0
SDK: /usr/local/share/dotnet/sdk/2.1.4/Sdks
SDK Versions:
2.1.4
2.0.0
1.0.1
1.0.0-preview2-003121
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.10.1/lib/mono/msbuild/15.0/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.6.2
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Apple Developer Tools ===

Xcode 9.3 (14154)
Build 9E145

=== Xamarin.Mac ===

Version: 4.4.1.176 (Visual Studio Enterprise)

=== Xamarin.iOS ===

Version: 11.10.1.177 (Visual Studio Enterprise)
Hash: 7e782c1e
Branch: d15-7
Build date: 2018-04-25 15:27:13-0400

=== Xamarin.Android ===

Version: 8.3.0.19 (Visual Studio Enterprise)
Android SDK: /Users/jongoldberger/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
4.0.3 (API level 15)
4.1 (API level 16)
4.2 (API level 17)
4.3 (API level 18)
4.4 (API level 19)
5.0 (API level 21)
5.1 (API level 22)
6.0 (API level 23)
7.0 (API level 24)
7.1 (API level 25)
8.0 (API level 26)
8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 27.0.1
SDK Build Tools Version: 27.0.3

Java SDK: /usr
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Xamarin Inspector ===

Version: 1.4.0
Hash: b3f92f9
Branch: master
Build date: Fri, 19 Jan 2018 22:00:34 GMT
Client compatibility: 1

=== Build Information ===

Release ID: 705001254
Git revision: 498923ea36d2c7fe440c4e4b8cfb75bd50bbd748
Build date: 2018-05-05 10:35:24-04
Xamarin addins: 219f1c4943b4693b837b4173dd10ea982a47c852
Build lane: monodevelop-lion-d15-7

=== Operating System ===

Mac OS X 10.13.4
Darwin 17.5.0 Darwin Kernel Version 17.5.0
Fri Apr 13 19:32:32 PDT 2018
root:xnu-4570.51.2~1/RELEASE_X86_64 x86_64

=== Enabled user installed addins ===

Internet of Things (IoT) development (Preview) 7.1

Test Project

TestProject.zip

ADB Logcat logs

EmbedAssembliesIntoNativeCodeCrash.txt

@HakanCelikMK1

This comment has been minimized.

HakanCelikMK1 commented May 10, 2018

I'm also suffering from the same problem, after I updated my Visual Studio installation to 15.7.1 this morning. I couldn't get the release version of my Android project to work.

This doesn't seem to be an edge case. How can such an obvious problem escape the testing ?
@RajGogri thank you for poviding a workaround, you've saved me a lot of time.

My environment :
Windows 10 Enterprise x64, 1709, 16299.431
Microsoft Visual Studio Enterprise 2017 Version 15.7.1
VisualStudio.15.Release/15.7.1+27703.2000
Microsoft .NET FrameworkVersion 4.7.03062
Xamarin 4.10.0.442 (396b18cef)
Xamarin Designer 4.12.264 (fc37cd02e)
Xamarin.Android SDK 8.3.0.19 (HEAD/342b2ce96)

@gjkhw

This comment has been minimized.

gjkhw commented May 10, 2018

I also see the issue. I just did a clean install of VS2017 for Mac 7.5 to be sure.

Click for a long list of "About" text from VS2017 for Mac...

Visual Studio Professional 2017 for Mac
Version 7.5 (build 1254)
Installation UUID: b05a38fd-8fe4-4390-b9b3-0c464dbea029
Runtime:
Mono 5.10.1.47 (2017-12/8eb8f7d5e74) (64-bit)
GTK+ 2.24.23 (Raleigh theme)
Xamarin.Mac 4.4.0.36 (master / 0c7c49a6)

Package version: 510010047

NuGet
Version: 4.3.1.4445

.NET Core
Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
2.0.5
2.0.0
1.1.1
1.0.4
SDK: /usr/local/share/dotnet/sdk/2.1.4/Sdks
SDK Versions:
2.1.4
2.0.0
1.0.3
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.10.1/lib/mono/msbuild/15.0/bin/Sdks

Xamarin.Profiler
Version: 1.6.2
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

Apple Developer Tools
Xcode 9.3 (14154)
Build 9E145

Xamarin.Mac
Version: 4.4.1.176 (Visual Studio Professional)

Xamarin.iOS
Version: 11.10.1.177 (Visual Studio Professional)
Hash: 7e782c1e
Branch: d15-7
Build date: 2018-04-25 15:27:13-0400

Xamarin.Android
Version: 8.3.0.19 (Visual Studio Professional)
Android SDK: /Users/e840944/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
6.0 (API level 23)
7.1 (API level 25)
8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 27.0.1
SDK Build Tools Version: 27.0.3

Java SDK: /usr
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

Xamarin Inspector
Version: 1.4.0
Hash: b3f92f9
Branch: master
Build date: Fri, 19 Jan 2018 22:00:34 GMT
Client compatibility: 1

Build Information
Release ID: 705001254
Git revision: 498923ea36d2c7fe440c4e4b8cfb75bd50bbd748
Build date: 2018-05-05 10:35:24-04
Xamarin addins: 219f1c4943b4693b837b4173dd10ea982a47c852
Build lane: monodevelop-lion-d15-7

Operating System
Mac OS X 10.13.4
Darwin 17.5.0 Darwin Kernel Version 17.5.0
Fri Apr 13 19:32:32 PDT 2018
root:xnu-4570.51.2~1/RELEASE_X86_64 x86_64

Enabled user installed addins
Internet of Things (IoT) development (Preview) 7.1

@brendanzagaeski

This comment has been minimized.

Member

brendanzagaeski commented May 10, 2018

Cross-referencing note for the Microsoft team

A matching error message has also been reported in: https://developercommunity.visualstudio.com/content/problem/249954/build-release-xamarin-android-error-if-check-budle.html

@jzeferino

This comment has been minimized.

jzeferino commented May 11, 2018

Having the same problem here.

@jonpryor

This comment has been minimized.

Contributor

jonpryor commented May 11, 2018

The immediate cause of the breakage is this change to mkbundle: mono/mono@ca8b8bd

It altered mkbundle output so that it now had a call to mono_jit_set_aot_mode(), which results in an undefined symbol in mkbundle output:

$ …/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-nm -D obj/Debug/bundles/armeabi-v7a/libmonodroid_bundle_app.so | grep mono_
         U mono_jit_set_aot_mode
00000ed8 T mono_mkbundle_init

This is what results in the dlopen failed: cannot locate symbol "mono_jit_set_aot_mode" message in adb logcat.

Furthermore, that mkbundle change (or an earlier one, I haven't dug deeper) has altered the API between xamarin-android and mkbundle output. Xamarin.Android expects a mono_mkbundle_init() declaration of:

typedef void (*mono_register_bundled_assemblies_fn)(const MonoBundledAssembly **);
typedef void (*mono_register_config_for_assembly_fn)(const char* assembly_name, const char* config_xml);

typedef void (*mono_mkbundle_init_ptr) (mono_register_bundled_assemblies_fn, mono_register_config_for_assembly_fn);

The mono_mkbundle_init that mkbundle emits, meanwhile, uses no parameters.

Furthermore, now that mkbundle wants to invoke mono_jit_set_aot_mode(), that will also need to be provided as a parameter to mono_mkbundle_init, thus removing the import from the generated libmonodroid_bundle_app.so.

@fekberg

This comment has been minimized.

Contributor

fekberg commented May 14, 2018

@jonpryor Any workaround for this (That is NOT turning off embedding assemblies)? Seems to be an issue in both STABLE and Beta.

@jzeferino

This comment has been minimized.

jzeferino commented May 14, 2018

@fekberg atm the only workaround i know is to set "Bundle Assemblies into Native Code" to false or downgrade your mono version.

@ninaada

This comment has been minimized.

ninaada commented May 15, 2018

When will this be fixed? We are stuck, not able to make release builds.

@RajGogri

This comment has been minimized.

RajGogri commented May 15, 2018

@ninaada @fekberg You can set BundleAssemblies flag to false for release config in Android's .csproj file. For the time being you can use this work around for running release build.

@fekberg

This comment has been minimized.

Contributor

fekberg commented May 15, 2018

@RajGogri That's a terrible workaround. There's a reason BundleAssemblies is set to true..

@RajGogri

This comment has been minimized.

RajGogri commented May 15, 2018

@fekberg Yes I know it's terrible, but that's the only work around that I found. I am also waiting for the true fix from Xamarin team.

grendello added a commit to grendello/xamarin-android that referenced this issue May 15, 2018

Fix bundled app crash on startup
Fixes: xamarin#1651

Recent update to mono brought in changes in the code generated by `mkbundle`
when dealing with AOT. Unfortunately, the changes require the bundle to be able
to find the `mono_jit_set_aot_mode` runtime function which isn't exported in the
Xamarin.Android up and thus the loading of the bundle crashes on startup with
the following message:

    dlopen failed: cannot locate symbol "mono_jit_set_aot_mode" referenced by libmonodroid_bundle_app.so

To fix this we need to augment the bundle's `mono_mkbundle_init` function in
generated code to take a third parameter - pointer to the abovementioned
function. With the current code this requires a set of rather ugly changes, as
seen in this commit, but it makes the application not crash on start and is the
fastest way to fix the issue right now.

This is a short-term fix, a long-term one will be presented at a later time.
@grendello

This comment has been minimized.

Contributor

grendello commented May 15, 2018

PR with the proposed fix: #1683

grendello added a commit to grendello/xamarin-android that referenced this issue May 15, 2018

Fix bundled app crash on startup
Fixes: xamarin#1651

Recent update to mono brought in changes in the code generated by `mkbundle`
when dealing with AOT. Unfortunately, the changes require the bundle to be able
to find the `mono_jit_set_aot_mode` runtime function which isn't exported in the
Xamarin.Android up and thus the loading of the bundle crashes on startup with
the following message:

    dlopen failed: cannot locate symbol "mono_jit_set_aot_mode" referenced by libmonodroid_bundle_app.so

To fix this we need to augment the bundle's `mono_mkbundle_init` function in
generated code to take a third parameter - pointer to the abovementioned
function. With the current code this requires a set of rather ugly changes, as
seen in this commit, but it makes the application not crash on start and is the
fastest way to fix the issue right now.

This is a short-term fix, a long-term one will be presented at a later time.
@EmilAlipiev

This comment has been minimized.

EmilAlipiev commented May 16, 2018

This is also happening for VSpre 15.8 and VS15.7.1... question is if it is VS2017 related or xamarin packages? so if I can at least install VS earlier version to update my app until it is fixed

@grendello

This comment has been minimized.

Contributor

grendello commented May 16, 2018

@EmilAlipiev the issue is caused by a breaking change in Mono, but we've fixed it entirely within Xamarin.Android, see #1683

@EmilAlipiev

This comment has been minimized.

EmilAlipiev commented May 16, 2018

@grendello cool. please let us know when it is merged and published in any version. thank you

grendello added a commit to grendello/xamarin-android that referenced this issue May 17, 2018

Fix bundled app crash on startup
Fixes: xamarin#1651

Recent update to mono brought in changes in the code generated by `mkbundle`
when dealing with AOT. Unfortunately, the changes require the bundle to be able
to find the `mono_jit_set_aot_mode` runtime function which isn't exported in the
Xamarin.Android up and thus the loading of the bundle crashes on startup with
the following message:

    dlopen failed: cannot locate symbol "mono_jit_set_aot_mode" referenced by libmonodroid_bundle_app.so

To fix this we need to augment the bundle's `mono_mkbundle_init` function in
generated code to take a third parameter - pointer to the abovementioned
function. With the current code this requires a set of rather ugly changes, as
seen in this commit, but it makes the application not crash on start and is the
fastest way to fix the issue right now.

This is a short-term fix, a long-term one will be presented at a later time.

grendello added a commit to grendello/xamarin-android that referenced this issue May 17, 2018

[WIP] Better mkbundle interop
Xamarin.Android side of the Mono's mkbundle update which makes it easier (and
safer) to introduce changes in the template generated by mkbundle when creating
the application bundle.

Code generated by mkbundle calls into the Mono runtime and in order for this to
work on Xamarin.Android, libmonodroid needs to properly initialize the bundle or
the application won't work (`dlsym` won't be able to find the Mono runtime
symbols). This has been done by patching the generated code using plain
search-and-replace which is very fragile even to the slightest changes and it
doesn't guarantee that any new additions won't break the Xamarin.Android
bundle (see xamarin#1651)

Mono's mkbundle has been updated ([insert_pr_ref_or_commit_ref]) to support
third parties, like Xamarin.Android, which need to use special methods to
initialize the bundle. The third party provides a small bit of source code with
a dispatch structure (mkbundle has its own default version of the
structure) which contains pointers to all the Mono runtime methods required by
the initialization code. The provided code is inserted by mkbundle in place of
its default version. If there are any incompatibilities between the two
structures (such as missing pointers, invalid signature, additional pointers
etc) then the bundle will not build, thus allowing the third party to notice the
problem early and update its version of the structure. Likewise, if the build
succeeds but the pointers aren't correctly initialized (i.e. they are `null`), a
runtime check will discover the fact and fail the application gracefully.

The bit of inserted code also contains platform-specific logging function in
place of the calls to `fprintf` used by mkbundle previously. For Xamarin.Android
it means that all the error messages will end up in logcat.

Xamarin.Android's version of the structure and the error logging function are
found in the `mkbundle-api.h` C header which is used *both* when building
libmonodroid (to ensure that when initializing the generated bundle libmonodroid
is ABI-compatible with it) and is copied to the framework directory to be used
when generating the application bundle.

Sample of XA-generated bundle code:
https://gist.github.com/grendello/465ecec96b3bee801e6bbc5a28019833

grendello added a commit to grendello/mono that referenced this issue May 17, 2018

mkbundle interop changes
mkbundle works by generating a small C source file which, in addition to
embedded assemblies/configs/etc, contains a certain amount of code to initialize
the bundle, uncompress the assemblies and set runtime options (e.g. AOT mode).
This source code is generated based on a number of templates shipped as
resources inside `mkbundle.exe` and is not part of any public, maintained API.

The generated code contains calls to Mono runtime which work just fine when
mkbundle is used to generate a desktop application bundle with the default
template code. However, it breaks when a third party (like Xamarin.Android)
embeds Mono runtime and loads it dynamically **after** the application is
initialized. Such third parties must ensure that the bundle's startup code is
able to access the Mono runtime functions it needs by providing pointers to
them, loaded using e.g. the `dlsym` function.

However, as the Mono calls are hard-coded into the generated source code, the
third party needs to resort to search-and-replace methods (e.g. Xamarin.Android
https://github.com/xamarin/xamarin-android/blob/62e7792ca52949f2fa213330d292865ad78d2bfa/src/Xamarin.Android.Build.Tasks/Tasks/MakeBundleNativeCodeExternal.cs#L159-L166)
which are very fragile and prone to undetected breakage.

One example of such breakage which slipped under the radar is
xamarin/xamarin-android#1651 which was caused by
addition of the `mono_jit_set_aot_mode` call to the generated code. The change
slipped under the radar and was discovered only after release.

This commit attempts to make the situation better by establishing a sort of a
code contract between "upstream" mkbundle and the third party. Since the
template code is not "public" (nor should it be), the third party cannot simply
include a header file with the API definitions. Instead, the idea here is to
ensure that all and any Mono runtime calls needed by the template code are
placed in a dispatch structure as pointers to functions and that the structure
needs to be initialized by the embedding party before calling the
`mono_mkbundle_init()` function. The third party is responsible for maintenance
of its own version of the dispatch structure as well as for its initialization
via the `initialize_mono_api` function in the template code. The generated code
must include the third party's version of the structure, in place of the default
one as shipped with `mkbundle`, and to make it possible this commit adds the
`--mono-api-struct-path` mkbundle parameter which points to a file that's
injected into the generated code, replacing the default structure definition.

Following that, the `mono_mkbundle_init()` code calls the `validate_api_struct`
function which checks each and every pointer in the structure for `null`,
exiting the application with error if any `null` pointer is encountered.
This (along with the `initialize_mono_api` function in the template) has the
additional effect that if the third party's structure is incompatible with the
default one a build error will happen, thus allowing early detection of obvious
incompatibilities/omissions.

As long as all the new Mono runtime calls are added to the structure, the above
should make it possible for issues like the one linked to above to not happen in
the future.
@ninaada

This comment has been minimized.

ninaada commented May 19, 2018

Any time frame as to when we will get this update? We ran out of quota in the app center also & currently have no way of making builds without turning off Bundle assemblies, which we really don't want to do.

@JeroenBer

This comment has been minimized.

JeroenBer commented May 19, 2018

This is also causing big problems for our app now, we need to do a new release but turn off bundle assemblies is not an option. Please fix asap.

@daniels7

This comment has been minimized.

daniels7 commented May 20, 2018

Come on Microsoft, this problem especially affects VS 2017 Enterprise Edition (and is one of the killer features of Xamarin on VS Enterprise) and each license costs several thousand Euros and still the issue is existing since several weeks... That's a no go! I don't understand how this error could ever pass your QA, since it is in normal release versions and not just Insider Previews or something... Fix this asap..please

@brendanzagaeski

This comment has been minimized.

Member

brendanzagaeski commented May 20, 2018

Non-engineering-team cross-referencing notes about continuous builds

Since by chance I have a moment this weekend, I'll add a little information that might be of interest to some users over the weekend.

  • xamarin-android-builds-d15-7/22 lists the preliminary un-tested continuous integration packages from the Xamarin team open source build server that include the candidate fix.
  • See also the commit comparison between these packages and the current version in Visual Studio 2017 version 15.7. In short, there is one new commit for this issue.

The team is of course coordinating to include this change into Visual Studio as soon as possible, so many users will likely prefer to wait for that.

But if some users would like to try the packages earlier, that is possible. Note that these steps do fall outside of the support policy. I also haven't had a chance to try this preliminary .vsix package myself. (I have used other preliminary Xamarin.Android .vsix packages successfully in the past.) If you would like to test this package locally, I would recommend installing it into a separate instance of Visual Studio (such as the Preview instance) if possible to avoid potential .vsix versioning conflicts when the fix is included into Visual Studio in an upcoming servicing update. You can revert the temporary .vsix installation under the Tools > Extensions and Updates > Installed menu if needed. (In theory, reverting the test package before updating is also sufficient to avoid versioning conflicts.) In Visual Studio for Mac, if you wish to revert the test .pkg, you can reinstall the current Stable channel version using the updater as usual.

(These steps are closely related to the guide on Installing Jenkins Build Artifacts.)

@JonDouglas

This comment has been minimized.

Contributor

JonDouglas commented May 21, 2018

At this time, we have a fix that will be available in Visual Studio 2017 version 15.7.3(Service Release 3) as our fix did not make the cutoff window for 15.7.2(Service Release 2). The release history can be found here. Here's a quick note regarding service updates with respect to a timeframe:

Servicing Updates are very targeted releases that typically contain bug fixes and ship more quickly. These servicing updates can ship often (e.g. weekly). You’ll be able to tell which servicing update you’re running by opening the Help, About and reading the third string in the version, for example 15.1.x, 15.2.y.

https://docs.microsoft.com/en-us/visualstudio/productinfo/vs2017-release-rhythm

To unblock app releases, you can follow @brendanzagaeski comment on Installing Jenkins Build Artifacts until this releases through Visual Studio.

@jonpryor jonpryor closed this May 21, 2018

monojenkins added a commit to monojenkins/mono that referenced this issue May 21, 2018

mkbundle interop changes
mkbundle works by generating a small C source file which, in addition to
embedded assemblies/configs/etc, contains a certain amount of code to initialize
the bundle, uncompress the assemblies and set runtime options (e.g. AOT mode).
This source code is generated based on a number of templates shipped as
resources inside `mkbundle.exe` and is not part of any public, maintained API.

The generated code contains calls to Mono runtime which work just fine when
mkbundle is used to generate a desktop application bundle with the default
template code. However, it breaks when a third party (like Xamarin.Android)
embeds Mono runtime and loads it dynamically **after** the application is
initialized. Such third parties must ensure that the bundle's startup code is
able to access the Mono runtime functions it needs by providing pointers to
them, loaded using e.g. the `dlsym` function.

However, as the Mono calls are hard-coded into the generated source code, the
third party needs to resort to search-and-replace methods (e.g. Xamarin.Android
https://github.com/xamarin/xamarin-android/blob/62e7792ca52949f2fa213330d292865ad78d2bfa/src/Xamarin.Android.Build.Tasks/Tasks/MakeBundleNativeCodeExternal.cs#L159-L166)
which are very fragile and prone to undetected breakage.

One example of such breakage which slipped under the radar is
xamarin/xamarin-android#1651 which was caused by
addition of the `mono_jit_set_aot_mode` call to the generated code. The change
slipped under the radar and was discovered only after release.

This commit attempts to make the situation better by establishing a sort of a
code contract between "upstream" mkbundle and the third party. Since the
template code is not "public" (nor should it be), the third party cannot simply
include a header file with the API definitions. Instead, the idea here is to
ensure that all and any Mono runtime calls needed by the template code are
placed in a dispatch structure as pointers to functions and that the structure
needs to be initialized by the embedding party before calling the
`mono_mkbundle_init()` function. The third party is responsible for maintenance
of its own version of the dispatch structure as well as for its initialization
via the `initialize_mono_api` function in the template code. The generated code
must include the third party's version of the structure, in place of the default
one as shipped with `mkbundle`, and to make it possible this commit adds the
`--mono-api-struct-path` mkbundle parameter which points to a file that's
injected into the generated code, replacing the default structure definition.

Following that, the `mono_mkbundle_init()` code calls the `validate_api_struct`
function which checks each and every pointer in the structure for `null`,
exiting the application with error if any `null` pointer is encountered.
This (along with the `initialize_mono_api` function in the template) has the
additional effect that if the third party's structure is incompatible with the
default one a build error will happen, thus allowing early detection of obvious
incompatibilities/omissions.

As long as all the new Mono runtime calls are added to the structure, the above
should make it possible for issues like the one linked to above to not happen in
the future.
@abdu292

This comment has been minimized.

abdu292 commented May 22, 2018

Oh, I see. Thanks for the update!

marek-safar added a commit to mono/mono that referenced this issue May 22, 2018

mkbundle interop changes
mkbundle works by generating a small C source file which, in addition to
embedded assemblies/configs/etc, contains a certain amount of code to initialize
the bundle, uncompress the assemblies and set runtime options (e.g. AOT mode).
This source code is generated based on a number of templates shipped as
resources inside `mkbundle.exe` and is not part of any public, maintained API.

The generated code contains calls to Mono runtime which work just fine when
mkbundle is used to generate a desktop application bundle with the default
template code. However, it breaks when a third party (like Xamarin.Android)
embeds Mono runtime and loads it dynamically **after** the application is
initialized. Such third parties must ensure that the bundle's startup code is
able to access the Mono runtime functions it needs by providing pointers to
them, loaded using e.g. the `dlsym` function.

However, as the Mono calls are hard-coded into the generated source code, the
third party needs to resort to search-and-replace methods (e.g. Xamarin.Android
https://github.com/xamarin/xamarin-android/blob/62e7792ca52949f2fa213330d292865ad78d2bfa/src/Xamarin.Android.Build.Tasks/Tasks/MakeBundleNativeCodeExternal.cs#L159-L166)
which are very fragile and prone to undetected breakage.

One example of such breakage which slipped under the radar is
xamarin/xamarin-android#1651 which was caused by
addition of the `mono_jit_set_aot_mode` call to the generated code. The change
slipped under the radar and was discovered only after release.

This commit attempts to make the situation better by establishing a sort of a
code contract between "upstream" mkbundle and the third party. Since the
template code is not "public" (nor should it be), the third party cannot simply
include a header file with the API definitions. Instead, the idea here is to
ensure that all and any Mono runtime calls needed by the template code are
placed in a dispatch structure as pointers to functions and that the structure
needs to be initialized by the embedding party before calling the
`mono_mkbundle_init()` function. The third party is responsible for maintenance
of its own version of the dispatch structure as well as for its initialization
via the `initialize_mono_api` function in the template code. The generated code
must include the third party's version of the structure, in place of the default
one as shipped with `mkbundle`, and to make it possible this commit adds the
`--mono-api-struct-path` mkbundle parameter which points to a file that's
injected into the generated code, replacing the default structure definition.

Following that, the `mono_mkbundle_init()` code calls the `validate_api_struct`
function which checks each and every pointer in the structure for `null`,
exiting the application with error if any `null` pointer is encountered.
This (along with the `initialize_mono_api` function in the template) has the
additional effect that if the third party's structure is incompatible with the
default one a build error will happen, thus allowing early detection of obvious
incompatibilities/omissions.

As long as all the new Mono runtime calls are added to the structure, the above
should make it possible for issues like the one linked to above to not happen in
the future.
@Foxsterc

This comment has been minimized.

Foxsterc commented May 23, 2018

Downloading the vsix, as suggested by @brendanzagaeski, solved this issue for me on Visual Studio 15.7.2

@ronpf

This comment has been minimized.

ronpf commented May 24, 2018

Seeing this in VSMac 7.5.1 (build 22)
When will the fix be for VSMac? Can't release app...

@fernandovm

This comment has been minimized.

fernandovm commented May 28, 2018

I have tested with the vsix, as suggested by @brendanzagaeski, but it not solve my problem. Unfortunately this is the third time that I update my VS and I'm have blocked without to can generate appstore releases. :((

@HakanCelikMK1

This comment has been minimized.

HakanCelikMK1 commented May 29, 2018

I can confirm that Xamarin.Android.Sdk.8.3.2.0.vsix provided in the @brendanzagaeski's answer fixes the problem in my VS 2017 15.7.2 installation.

My environment :
OS : Windows 10 Enterprise x64, 1709, 16299.431

VS Info Before Install :
Microsoft Visual Studio Enterprise 2017 Version 15.7.2
VisualStudio.15.Release/15.7.2+27703.2018
Microsoft .NET Framework Version 4.7.03062
Xamarin 4.10.0.442 (396b18cef)
Xamarin Designer 4.12.264 (fc37cd02e)
Xamarin.Android SDK 8.3.0.19 (HEAD/342b2ce96)

VS Info After Install :
Microsoft Visual Studio Enterprise 2017 Version 15.7.2
VisualStudio.15.Release/15.7.2+27703.2018
Microsoft .NET Framework Version 4.7.03062
Xamarin 4.10.0.448 (4373404db)
Xamarin Designer 4.12.270 (82d750d12)
Xamarin.Android SDK 8.3.2.0 (HEAD/8be583805)

Cheers.

@mclillill

This comment has been minimized.

mclillill commented May 30, 2018

Thanks. I had an install error. Then VS didn't start. Then tried installing the vsix again. It said "already installed". Then VS started again and I now have 8.3.2.0 installed. I didn't test yet though.

@mclillill

This comment has been minimized.

mclillill commented May 30, 2018

It works in relase mode. But debug mode now only works with "Shared Runtime" turned off. Not sure if related...

@Szymaniuk

This comment has been minimized.

Szymaniuk commented May 30, 2018

Same for me - it's not working with latest stable release of VS on Windows as well as on Mac.
Workaround is to download and install latest pre-release of Xamarin.Android SDK from here https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/lastSuccessfulBuild/Azure/

@xufeipyxis

This comment has been minimized.

xufeipyxis commented Jun 1, 2018

I see v15.7.3 is released but I don't see this fix is included?
https://docs.microsoft.com/en-us/visualstudio/releasenotes/vs2017-relnotes#15.7.3

@abdu292

This comment has been minimized.

abdu292 commented Jun 1, 2018

@JeroenBer

This comment has been minimized.

JeroenBer commented Jun 1, 2018

I installed 15.7.3 and it is working for me now.

@fernandovm

This comment has been minimized.

fernandovm commented Jun 1, 2018

Dear... Now I can generate appstore releases again, but unfortunately I can't explain exactaly the reason to be used as workaround. I have tested with the vsix, as suggested by @brendanzagaeski, but it not solve my problem. Then I have updated my project to Xamarin Forms 3 and indeed, it works, but I had other minor problems. Then I have downgraded my project to previous Xamarin Forms (2.5) and, without neither hope I have tried rebuild again, but, to my surprise, all worked fine.

@EmilAlipiev

This comment has been minimized.

EmilAlipiev commented Jun 14, 2018

Hi, Please reopen this bug. it still doesnt work. this error occurs #1760

This is surely related to this problem. when I see the logs, i am seeing this information. what is that supposed to mean? what ArchiveListViewModel and what key does it expect?


Xamarin.VisualStudio.Progress.ProgressReportService|Information|0|Shared Mono runtime is enabled for ''.
Xamarin.VisualStudio.Publishing.Presentation.ViewModels.ArchiveListViewModel|Error|0|System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Xamarin.VisualStudio.Publishing.Presentation.ViewModels.ArchiveListViewModel.<<SubscribeToEvents>b__56_5>d.MoveNext() in E:\A\_work\5\s\src\Core\VisualStudio.Publishing\Presentation\ViewModels\ArchiveListViewModel.cs:line 262
Xamarin.VisualStudio.Progress.ProgressReportService|Information|0|The selected build configuration is using the shared Mono runtime for faster deployment. Apps cannot be archived with this setting enabled.
Xamarin.VisualStudio.Progress.ProgressReportService|Information|0|Please ensure that you are using a release configuration and that the "Use Shared Mono Runtime" option in your project's build options is unchecked.

grendello added a commit to grendello/xamarin-android that referenced this issue Jun 28, 2018

Better mkbundle interop
Xamarin.Android side of the Mono's mkbundle update which makes it easier (and
safer) to introduce changes in the template generated by mkbundle when creating
the application bundle.

Code generated by mkbundle calls into the Mono runtime and in order for this to
work on Xamarin.Android, libmonodroid needs to properly initialize the bundle or
the application won't work (`dlsym` won't be able to find the Mono runtime
symbols). This has been done by patching the generated code using plain
search-and-replace which is very fragile even to the slightest changes and it
doesn't guarantee that any new additions won't break the Xamarin.Android
bundle (see xamarin#1651)

Mono's mkbundle has been updated (see
mono/mono@170e944)
to support third parties, like Xamarin.Android, which need to use special
methods to initialize the bundle. The third party provides a small bit of source
code with a dispatch structure (mkbundle has its own default version of the
structure) which contains pointers to all the Mono runtime methods required by
the initialization code. The provided code is inserted by mkbundle in place of
its default version. If there are any incompatibilities between the two
structures (such as missing pointers, invalid signature, additional pointers
etc) then the bundle will not build, thus allowing the third party to notice the
problem early and update its version of the structure. Likewise, if the build
succeeds but the pointers aren't correctly initialized (i.e. they are `null`), a
runtime check will discover the fact and fail the application gracefully.

The bit of inserted code also contains platform-specific logging function in
place of the calls to `fprintf` used by mkbundle previously. For Xamarin.Android
it means that all the error messages will end up in logcat.

Xamarin.Android's version of the structure and the error logging function are
found in the `mkbundle-api.h` C header which is used *both* when building
libmonodroid (to ensure that when initializing the generated bundle libmonodroid
is ABI-compatible with it) and is copied to the framework directory to be used
when generating the application bundle.

Sample of XA-generated bundle code:
https://gist.github.com/grendello/465ecec96b3bee801e6bbc5a28019833

grendello added a commit that referenced this issue Jun 28, 2018

Better mkbundle interop
Xamarin.Android side of the Mono's mkbundle update which makes it easier (and
safer) to introduce changes in the template generated by mkbundle when creating
the application bundle.

Code generated by mkbundle calls into the Mono runtime and in order for this to
work on Xamarin.Android, libmonodroid needs to properly initialize the bundle or
the application won't work (`dlsym` won't be able to find the Mono runtime
symbols). This has been done by patching the generated code using plain
search-and-replace which is very fragile even to the slightest changes and it
doesn't guarantee that any new additions won't break the Xamarin.Android
bundle (see #1651)

Mono's mkbundle has been updated (see
mono/mono@170e944)
to support third parties, like Xamarin.Android, which need to use special
methods to initialize the bundle. The third party provides a small bit of source
code with a dispatch structure (mkbundle has its own default version of the
structure) which contains pointers to all the Mono runtime methods required by
the initialization code. The provided code is inserted by mkbundle in place of
its default version. If there are any incompatibilities between the two
structures (such as missing pointers, invalid signature, additional pointers
etc) then the bundle will not build, thus allowing the third party to notice the
problem early and update its version of the structure. Likewise, if the build
succeeds but the pointers aren't correctly initialized (i.e. they are `null`), a
runtime check will discover the fact and fail the application gracefully.

The bit of inserted code also contains platform-specific logging function in
place of the calls to `fprintf` used by mkbundle previously. For Xamarin.Android
it means that all the error messages will end up in logcat.

Xamarin.Android's version of the structure and the error logging function are
found in the `mkbundle-api.h` C header which is used *both* when building
libmonodroid (to ensure that when initializing the generated bundle libmonodroid
is ABI-compatible with it) and is copied to the framework directory to be used
when generating the application bundle.

Sample of XA-generated bundle code:
https://gist.github.com/grendello/465ecec96b3bee801e6bbc5a28019833
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment