Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support full aot for Android #1090

Closed
yinyue200 opened this issue Dec 9, 2017 · 7 comments
Closed

Support full aot for Android #1090

yinyue200 opened this issue Dec 9, 2017 · 7 comments

Comments

@yinyue200
Copy link

Hope to implement "full aot" to remove libmonodroid_bundle_app.so.
It can reduce the size of the software when we do not need to dynamically generate the code.

Steps to Reproduce

Expected Behavior

Actual Behavior

Version Information

Log File

@jonpryor
Copy link
Member

Hope to implement "full aot" to remove libmonodroid_bundle_app.so.

That doesn't make any sense at all, as it intermixes two separate things.

(Aside: supporting "Full AOT", while possible, is explicitly not a goal, because it will very likely cause all existing apps to break, as it doesn't support any JIT use. We support "Hybrid AOT", which supports using the Full AOT backend+native libraries, while continuing to support a JIT, for System.Reflection.Emit/etc. use. Plus, Xamarin.Android itself still requires a JIT. Even if Xamarin.Android didn't require it, we have no way of knowing what existing apps require runtime codegen. Removing the JIT is Not A Thing we are considering.)

libmonodroid_bundle_app.so only contains assemblies, zip-compressed. It's a way to avoid the assemblies/ directory within the .apk.

"Full AOT"/Hybrid AOT generates a native library (.so) for each assembly. The assembly is still required to exist, somewhere, so that System.Reflection can work. (The assemblies can be IL-stripped with Hybrid AOT, but they must still exist!)

Finally, Hybrid AOT results in huge native libraries. You almost certainly will not get smaller apps from it. You will get faster apps from it. (The classic size/speed tradeoff.)

You are able to use both mkbundle ("Bundle assemblies into native code"/libmonodroid_bundle_app.so) and Hybrid AOT together and separately. They do not interact with each other in any meaningful way.

@Belorus
Copy link

Belorus commented Jan 11, 2018

@jonpryor U've mentioned that assemblies can be IL-stripped. From security perspective it is a great thing. Right now it is matter of few hours for a smart guy from the street that knows what deflate is and has .APK, to get real IL.

Is there possibility that you guys make that stripping part of XA build pipeline ? Will be happy to become your beta tester.

@jonpryor
Copy link
Member

Is there possibility that you guys make that stripping part of XA build pipeline?

@Belorus: That should already be possible:

  • Set $(AndroidAotMode)=Hybrid
  • Set $(AotAssemblies)=True

This should cause cil-strip.exe to be run, which will remove method body IL from the assemblies.

As with iOS, assemblies are still needed, but the IL method bodies are not.

@bulente
Copy link

bulente commented Jan 12, 2018

I tried to build my project with AndroidAotMode=Hybrid and get this error when archiving:

Could not strip IL of assembly: obj\Release\android\assets\System.Core.dll

[cil-strip stdout] Error: System.MissingMethodException: Method not found: 'Void System.Array.Reverse(!!0[], Int32, Int32)'.
  [cil-strip stdout]    at Mono.Cecil.AssemblyNameReference.get_PublicKeyToken()
  [cil-strip stdout]    at Mono.Cecil.AssemblyNameReference.Parse(String fullName)
  [cil-strip stdout]    at Mono.Cecil.Signatures.SignatureReader.CreateEnumTypeReference(String enumName)
  [cil-strip stdout]    at Mono.Cecil.Signatures.SignatureReader.ReadTypeReference(Byte[] data, BinaryReader br, ElementType& elemType)
  [cil-strip stdout]    at Mono.Cecil.Signatures.SignatureReader.ReadNamedArg(Byte[] data, BinaryReader br, Boolean& read, Boolean resolve)
  [cil-strip stdout]    at Mono.Cecil.SecurityDeclarationReader.CreateSecurityAttribute(SecurityAction action, BinaryReader br, Byte[] permset, Int32 pos, Int32& start, Boolean resolve)
  [cil-strip stdout]    at Mono.Cecil.SecurityDeclarationReader.FromByteArray(SecurityAction action, Byte[] declaration, Boolean resolve)
  [cil-strip stdout]    at Mono.Cecil.AggressiveReflectionReader.ReadSecurityDeclarations()
  [cil-strip stdout]    at Mono.Cecil.AggressiveReflectionReader.VisitTypeDefinitionCollection(TypeDefinitionCollection types)
  [cil-strip stdout]    at Mono.Cecil.StructureReader.TerminateAssemblyDefinition(AssemblyDefinition asm)
  [cil-strip stdout]    at Mono.Cecil.AssemblyFactory.GetAssembly(ImageReader irv, Boolean manifestOnly)
  [cil-strip stdout]    at Mono.CilStripper.Program.Main(String[] args)

@yinyue200
Copy link
Author

yinyue200 commented Jan 13, 2018

@bulente I found enable linker can solve the problem. But it will fail in runtime.
Unhandled Exception:


System.InvalidProgramException: Invalid IL code in Android.Support.V4.App.FragmentActivity:GetOnCreate_Landroid_os_Bundle_Handler (): IL_0000: ret       



01-13 06:54:14.165 E/mono    ( 4530): 
01-13 06:54:14.165 E/mono    ( 4530): Unhandled Exception:
01-13 06:54:14.165 E/mono    ( 4530): System.InvalidProgramException: Invalid IL code in Android.Support.V4.App.FragmentActivity:GetOnCreate_Landroid_os_Bundle_Handler (): IL_0000: ret       
01-13 06:54:14.165 E/mono    ( 4530): 
01-13 06:54:14.165 E/mono    ( 4530): 
01-13 06:54:14.165 E/mono    ( 4530):   at Android.Runtime.JNIEnv.RegisterJniNatives (System.IntPtr typeName_ptr, System.Int32 typeName_len, System.IntPtr jniClass, System.IntPtr methods_ptr, System.Int32 methods_len) [0x00217] in <daa44a0def9344c8889faf81a180b741>:0 
01-13 06:54:14.165 E/mono-rt ( 4530): [ERROR] FATAL UNHANDLED EXCEPTION: System.InvalidProgramException: Invalid IL code in Android.Support.V4.App.FragmentActivity:GetOnCreate_Landroid_os_Bundle_Handler (): IL_0000: ret       
01-13 06:54:14.165 E/mono-rt ( 4530): 
01-13 06:54:14.165 E/mono-rt ( 4530): 
01-13 06:54:14.165 E/mono-rt ( 4530):   at Android.Runtime.JNIEnv.RegisterJniNatives (System.IntPtr typeName_ptr, System.Int32 typeName_len, System.IntPtr jniClass, System.IntPtr methods_ptr, System.Int32 methods_len) [0x00217] in <daa44a0def9344c8889faf81a180b741>:0 

@jonpryor

@bulente
Copy link

bulente commented Jan 13, 2018

I found enable linker can solve the problem. But it will fail in runtime.

@jonpryor, thanks for the feedback but I'm using some 3rd party libraries that doesn't allow me to link system.core file which results that kind of runtime error.

@bulente
Copy link

bulente commented Jan 13, 2018

I wrote a custom msbuild task that removes unwanted .so files and cil-strip .dll files if their counterpart is in the lib folder as .so file before the "_Sign" step. All I want is a partial AOT. I don't need system.core to be striped because I don't want this file to be native as .so. When I run the final apk on the device, it gives "Could not load assembly AAAA during startup registration" error although the file is there in the assemblies folder as striped and there is a libaot- counterpart in the lib folder. Aot mode is normal. Is there a way to do what I want?

jonpryor pushed a commit that referenced this issue Oct 12, 2020
Fixes: #4996
Fixes: #5009
Fixes: #5147

Changes: xamarin/monodroid@1ac5333...767f647

  * xamarin/monodroid@767f64715: [msbuild] Fast Deployment v2.0 (#1090)
  * xamarin/monodroid@0f04ba56d: Merge pull request #1115 from xamarin/remove-xreitem
  * xamarin/monodroid@d75341fc3: Remove provisionator file completely
  * xamarin/monodroid@b62e8c693: Replace XreItem with supported Xcode and JavaJDK syntax

The Fast Deployment system used for debugging Xamarin.Android apps has
been completely re-written.  This is mostly due to changes in Android
which means we can no longer use the external storage directory to
store assemblies.

Fast Deployment works by not including files which change often,
like assemblies, in the actual apk.  This means the `.apk` will mostly
not need to be re-installed during a debugging/development session.
Instead the assemblies are "Fast Deployed" to a special directory where
a debug version of our runtime knows where to find them.

Historically this was on the external storage directory such as

	/storage/emulated/0/Android/data/com.some.package
	/mnt/shell/emulated/0/Android/data/com.some.package
	/storage/sdcard/Android/data/com.some.package

With Android 11, these directories are no longer accessible.  Instead,
we need to deploy the assemblies into the app's internal `files`
directory.  This is usually located in `/data/data/@PACKAGE_NAME@`.
This is not a global writable folder, so we need to use the `run-as`
tool to run all the commands to copy the files into that directory.

The `run-as` tool does not always work on older devices.  From this
point on Fast Deployment v2 will only be available on API-21+ devices.
If a certain device does not support the `run-as` tool, then you can
always fall back to debugging without Fast Deployment.  While this is
slower, it should still work on most devices.

[`$(AndroidFastDeploymentType)`][0] is still supported.  This will
deploy both assemblies, native libraries, typemaps, and `.dex` files to
the `files` directory.  Support for Fast Deploying Android resources
and assets was removed in commit f0d565f, as it required the use of
deprecated API's to work.

The Shared Runtime has also be removed in this new system.  Previously,
we used to deploy the BCL and API specific assemblies via separate
`.apk` files.  This new system removes the need for that.  All the BCL
and API specific assemblies will be deployed to the `files` directory
like all the other assemblies.

The new system is on par with the existing system when it comes to
speed.  More improvements are planned in future releases which should
make it much quicker.

Using the `samples\HelloWorld` project these are the performance
differences using `HelloWorld.csproj /restore /t:Install /v:n`:

  * Deploy "from Clean"
    * v1: 00:00:11.42
    * v2: 00:00:11.78 [3% longer]
  * Incrementally deploy C#-based change
    * v1: 00:00:02.58
    * v2: 00:00:02.43 [6% faster]

[0]: https://docs.microsoft.com/en-us/xamarin/android/deploy-test/building-apps/build-properties#androidfastdeploymenttype
@xamarin xamarin locked as resolved and limited conversation to collaborators Jun 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants