Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After Upgrade from Visual Studio 2019 16.6 to 16.7 APK Size Increases significantly from 24 MB to 64 MB #4986

Closed
inforithmics opened this issue Aug 6, 2020 · 19 comments · Fixed by #5021
Assignees
Labels
Area: App+Library Build Issues when building Library projects or Application projects. question

Comments

@inforithmics
Copy link

inforithmics commented Aug 6, 2020

Steps to Reproduce

  1. Open Attached Xamarin.Android.zip
  2. Change To Release Mode
  3. Execute Deploy (The generated Xamarin.Android.apk is 64 MB)

Xamarin.Android.zip

Expected Behavior

Size is 24MB

Actual Behavior

Size is 64MB

Version Information

Visual Studio 2019 16.7

Work around

Changing following line in the AndroidManifest.xml
from
<uses-sdk android:minSdkVersion="23" android:targetSdkVersion="29" />
to
<uses-sdk android:minSdkVersion="22" android:targetSdkVersion="29" />
reduces the apk size to 24MB

The reason why the apk with minsdkversion 23 is so much bigger is that that the libs aren't anymore compressed
image

Here the apk with MinSdkVersion="22"

image

With Visual Studio Version 2019 16.6 The size was 24MB even with minSDKVersion:="23"

Currently only x86 is selected as compiled target the app doesn't work is only used as example how the apk size is increased

@inforithmics inforithmics added the Area: App+Library Build Issues when building Library projects or Application projects. label Aug 6, 2020
@jpobst jpobst added this to the Under Consideration milestone Aug 6, 2020
@jonathanpeppers
Copy link
Member

jonathanpeppers commented Aug 6, 2020

@inforithmics the difference in the bigger .apk file is this value in the AndroidManifest.xml:

<application android:extractNativeLibs="false"

See: https://developer.android.com/guide/topics/manifest/application-element

I think this value is emitted by default for minSdkVersion="23" and higher. It is something the Android toolchain is doing automatically. Google claims this really cuts down on install size, as the native libraries are loading directly from the .apk file. Otherwise on install, they get extracted to a directory and take up more space.

I could change the manifest to this to get the smaller .apk file:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" android:versionCode="1" android:versionName="1.0" package="com.companyname.app52">
    <uses-sdk android:minSdkVersion="23" android:targetSdkVersion="29" />
    <application android:label="App52.Android" android:theme="@style/MainTheme" android:extractNativeLibs="true"></application>
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />	
</manifest>

This problem really shows up when you are using AOT, because there are large native libraries. It's up to you as the developer which setting is better for your app.

I'm not sure what Xamarin.Android needs to do here, if anything. If android:extractNativeLibs="false" is always bad when AotAssemblies=true, maybe we could default this value to true when AOT is used?

@inforithmics
Copy link
Author

inforithmics commented Aug 6, 2020

  • The problem is that the native libraries are really big and the resulting apk would be bigger than 100 MB, so I wouldn't be able to upload it to the Store.
  • By Upgrading for Visual Studio 2019 16.7 I although upgraded the Android Sdks and Tools that's maybe the reason it got changed.

@jonathanpeppers
Copy link
Member

I think in your case, android:extractNativeLibs="true" would need to be specified to stay under 100MB.

Updating to VS 16.7 or updating Android SDKs might have cause the behavior to change for your app. We updated the version of aapt2 that is bundled with Xamarin.Android in 16.7:

https://docs.microsoft.com/xamarin/android/release-notes/11/11.0#aapt2-version-update-to-400

@inforithmics
Copy link
Author

inforithmics commented Aug 6, 2020

Thanks for the Tipp
setting android:extractNativeLibs="true" worked and the apk is now under 100MB
intrestingly setting the bundle format to aab, the aab file stays under 100MB too (Because in aab the libs are compressed)

@jonathanpeppers
Copy link
Member

I'm going to leave this open for a bit to see if a lot of others hit this issue.

We might need to change something, I'm not sure what it would be yet.

@jonathanpeppers
Copy link
Member

We isolated the change that caused this to start happening in 16.7: 86737ca

Specifically updating the manifest-merger from 26.5 to 27.0 caused android:extractNativeLibs="false" to be emitted when minSdkVersion is 23 or higher.

If you are using AndroidX, it enables the new manifest-merger feature:

https://docs.microsoft.com/xamarin/android/release-notes/10/10.2#improved-android-manifest-merging

@inforithmics
Copy link
Author

Thanks, Good to know.
Yes I'm using AndroidX in the app

@developer9969
Copy link

@jonathanpeppers
I am using AndroidX with a min version higher than 23. Will this increase the size of the apk?
do we need to set android:extractNativeLibs="true

I would like to upgrade to 16.7 visual studio but not sure if this will cause more problems like this and also will the azure pipeline still work as before.

I am deploying to a client in a couple of weeks and I m worried that than I cannot revert to previous version of visual studio.

Any input in the matter will be greatly appreciated

@jonathanpeppers
Copy link
Member

Will this increase the size of the apk? do we need to set android:extractNativeLibs="true

@developer9969 you can read about the setting here:

You might just look at your .apk file and see if this even matters for your app. Google prefers extractNativeLibs="false", because it saves on disk space after your app is installed. But this will cause a larger download size.

@brendanzagaeski
Copy link
Contributor

Cross-referencing note for the Xamarin team

Developer Community item that looks like it might be for the same behavior: https://developercommunity.visualstudio.com/content/problem/1144021/vs-1670-compile-and-archive-issues.html

@jonathanpeppers
Copy link
Member

@brendanzagaeski @dellis1972 there is also a bin/obj bug here...

If you deploy an app with android:extractNativeLibs="true" (most of the time the default when omitted).

Then change it to android:extractNativeLibs="false" and deploy again you get a deploy error:

ADB0010: Mono.AndroidTools.InstallFailedException: Failure [INSTALL_FAILED_INVALID_APK: Failed to extract native libraries, res=-2]

I can fix the problem by doing a Clean. This was probably introduced in 1c4727c -- it doesn't look like compression is taken into account for updating files during the <BuildApk/> MSBuild task.

I will look into this.

@developer9969
Copy link

@jonathanpeppers thank you for replying I will read into the links and noticed that other people have experienced the same behavior (App growth). I will monitor this issue and hopefully this will be fixed in the next version of Visual studio.

@brendanzagaeski
Copy link
Contributor

there is also a bin/obj bug here...

@jonathanpeppers, for the eventual fix for that issue, if you like, it looks like #4990 is about that part of the behavior and so can be included as a Fixes: line in the commit message for that, for thorough bookkeeping.

@jonathanpeppers
Copy link
Member

Merged a fix for the bin/obj issue: #5001

We should discuss about emitting android:extractNativeLibs="true" by default if not specified by Xamarin.Android apps. It seems like Google is changing this value on us, and it is not desired.

jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Aug 17, 2020
… default

Fixes: dotnet#4986
Context: dotnet@86737ca
Context: https://android.googlesource.com/platform/tools/base/+/fd2ac00ab76dfdececce5a25cb7ad23b2b6f5d29%5E%21

When we bumped `manifest-merger` from 26.5.0 to 27.0.0, it started
emitting this in `AndroidManifest.xml` when `minSdkVersion` is 23 or
higher:

    <application android:extractNativeLibs="false">

When *omitted* this value defaults to `true`, meaning that native
libraries are compressed in `.apk` files and extracted to disk on
installation.

When `extractNativeLibs` is `false`, native libraries are
*uncompressed* in `.apk` files and loaded directly from the `.apk`.
This saves on install size at the cost of larger `.apk` size. This
behavior is only supported on API 23 devices and higher.

The Android docs say:

> The default value is "true". However, when building your app using
> Android Gradle plugin 3.6.0 or higher, this property is set to
> "false" by default.

Ignoring the related bugs caused by this behavior:

* dotnet@bf657e8
* dotnet@8828fef

The new `manifest-merger` will implicitly make `.apk` files larger. In
the case of a customer using AOT, an app grew from 24MB to 60MB!

To solve this:

* If not specified in apps' `Properties\AndroidManifest.xml`, we will
  emit `android:extractNativeLibs="true"` by default.

This way Xamarin.Android applications will have consistent behavior.
Developers can set `extractNativeLibs="false"` if they prefer that
behavior.

This behavior also works fine on an API 19 emulator, since
`extractNativeLibs="true"` is already the default and seems to be
ignored.
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Aug 17, 2020
… default

Fixes: dotnet#4986
Context: dotnet@86737ca
Context: https://android.googlesource.com/platform/tools/base/+/fd2ac00ab76dfdececce5a25cb7ad23b2b6f5d29%5E%21

When we bumped `manifest-merger` from 26.5.0 to 27.0.0, it started
emitting this in `AndroidManifest.xml` when `minSdkVersion` is 23 or
higher:

    <application android:extractNativeLibs="false">

When *omitted* this value defaults to `true`, meaning that native
libraries are compressed in `.apk` files and extracted to disk on
installation.

When `extractNativeLibs` is `false`, native libraries are
*uncompressed* in `.apk` files and loaded directly from the `.apk`.
This saves on install size at the cost of larger `.apk` size. This
behavior is only supported on API 23 devices and higher.

The Android docs say:

> The default value is "true". However, when building your app using
> Android Gradle plugin 3.6.0 or higher, this property is set to
> "false" by default.

Ignoring the related bugs caused by this behavior:

* dotnet@bf657e8
* dotnet@8828fef

The new `manifest-merger` will implicitly make `.apk` files larger. In
the case of a customer using AOT, an app grew from 24MB to 60MB!

To solve this:

* If not specified in apps' `Properties\AndroidManifest.xml`, we will
  emit `android:extractNativeLibs="true"` by default.

This way Xamarin.Android applications will have consistent behavior.
Developers can set `extractNativeLibs="false"` if they prefer that
behavior.

This behavior also works fine on an API 19 emulator, since
`extractNativeLibs="true"` is already the default and seems to be
ignored.
jonpryor pushed a commit that referenced this issue Aug 19, 2020
)

Fixes: #4986

The [`//application/@android:extractNativeLibs`][0] attribute within
`AndroidManifest.xml` has had a large-scale impact on `.apk`s:

When `extractNativeLibs` is `true`, then (1) native libraries within
the `.apk` are *compressed*, and (2) the native libraries within the
`.apk` are extracted during installation time.  This allows the `.apk`
to be smaller, while resulting in a *larger* installation footprint,
as two copies of the native libs -- one compressed, one uncompressed --
are present on the device.

When `extractNativeLibs` is `false`, then (1) native libraries within
the `.apk` are stored *uncompressed*, and (2) the native libraries
within the `.apk` are *not* extracted during installation time, but
*only* on API-23 and later devices.  This makes for a larger `.apk`,
but a *smaller* installation footprint.

When *omitted* this value defaults to `true`.

Commit 86737ca bumped `manifest-merger` from 26.5.0 to 27.0.0, which
[altered `manifest-merger.jar` behavior][1] so that it started setting
`//application/@android:extractNativeLibs`=false when `minSdkVersion`
is 23 or higher, "as if" `Properties\AndroidManifest.xml` contained:

	<application android:extractNativeLibs="false">

In the case of Issue #4986, which is an application which uses
Xamarin.Forms and sets `$(AotAssemblies)`=True, resulting in native
libraries being produced and included for *every assembly*, this
"minor" change to `extractNativeLibs` caused a 24MB `.apk` to balloon
in size to 64MB, which was unexpected.

This unexpected size increase was not caught by unit tests, as our
tests which check for `.apk` size regressions don't use Xamarin.Forms,
and it's AndroidX/Android Support -- used by Xamarin.Forms -- which
causes `manifest-merger.jar` to be used.  No `manifest-merger.jar`,
no unexpected size increase.

This `manifest-merger.jar` change is *also* the underlying cause of
bf657e8 and 8828fef.

To solve this, if not explicitly specified within apps'
`Properties\AndroidManifest.xml`, we will now insert
`android:extractNativeLibs="true"` by default.

This way Xamarin.Android applications will have consistent behavior.
Developers can set `extractNativeLibs="false"` if they prefer that
behavior.

This behavior also works fine on an API-19 emulator, since
`extractNativeLibs="true"` is already the default and seems to be
ignored.

[0]: https://developer.android.com/guide/topics/manifest/application-element#extractNativeLibs
[1]: https://android.googlesource.com/platform/tools/base/+/fd2ac00ab76dfdececce5a25cb7ad23b2b6f5d29%5E%21
@developer9969
Copy link

@brendanzagaeski hi there is this fix in vs 16.7.2? many thanks

@brendanzagaeski
Copy link
Contributor

@developer9969, thanks for the question. The fix for the issue is not included in Visual Studio 2019 version 16.7.2.

If everything goes as planned, this fix will be included in future versions of Visual Studio 2019 version 16.8 Preview and Visual Studio 2019 for Mac version 8.8 Preview. I will be adding an "In Preview: " milestone to this issue along with a comment as soon as the fix is published in a Preview version.

Here's an example of how the comment will look: #4409 (comment).

Note also that the fix for this issue doesn't do anything other than set android:extractNativeLibs="true" by default when no value is provided by the application in the Properties\AndroidManifest.xml file. Even when that fix is avaiable, project authors might wish to continue to set android:extractNativeLibs explicitly in AndroidManifest.xml anyway to make it easy to remember which setting their apps are using. That attribute can be set to either "true" to keep the same behavior from Visual Studio 2019 version 16.6 or "false" to accept the tradeoff of a larger APK and download size in exchange for a smaller total install size for the application on device. See also the draft release note for the fix: #5021 (comment).

jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Aug 20, 2020
…tnet#5021)

Fixes: dotnet#4986

The [`//application/@android:extractNativeLibs`][0] attribute within
`AndroidManifest.xml` has had a large-scale impact on `.apk`s:

When `extractNativeLibs` is `true`, then (1) native libraries within
the `.apk` are *compressed*, and (2) the native libraries within the
`.apk` are extracted during installation time.  This allows the `.apk`
to be smaller, while resulting in a *larger* installation footprint,
as two copies of the native libs -- one compressed, one uncompressed --
are present on the device.

When `extractNativeLibs` is `false`, then (1) native libraries within
the `.apk` are stored *uncompressed*, and (2) the native libraries
within the `.apk` are *not* extracted during installation time, but
*only* on API-23 and later devices.  This makes for a larger `.apk`,
but a *smaller* installation footprint.

When *omitted* this value defaults to `true`.

Commit 86737ca bumped `manifest-merger` from 26.5.0 to 27.0.0, which
[altered `manifest-merger.jar` behavior][1] so that it started setting
`//application/@android:extractNativeLibs`=false when `minSdkVersion`
is 23 or higher, "as if" `Properties\AndroidManifest.xml` contained:

	<application android:extractNativeLibs="false">

In the case of Issue dotnet#4986, which is an application which uses
Xamarin.Forms and sets `$(AotAssemblies)`=True, resulting in native
libraries being produced and included for *every assembly*, this
"minor" change to `extractNativeLibs` caused a 24MB `.apk` to balloon
in size to 64MB, which was unexpected.

This unexpected size increase was not caught by unit tests, as our
tests which check for `.apk` size regressions don't use Xamarin.Forms,
and it's AndroidX/Android Support -- used by Xamarin.Forms -- which
causes `manifest-merger.jar` to be used.  No `manifest-merger.jar`,
no unexpected size increase.

This `manifest-merger.jar` change is *also* the underlying cause of
bf657e8 and 8828fef.

To solve this, if not explicitly specified within apps'
`Properties\AndroidManifest.xml`, we will now insert
`android:extractNativeLibs="true"` by default.

This way Xamarin.Android applications will have consistent behavior.
Developers can set `extractNativeLibs="false"` if they prefer that
behavior.

This behavior also works fine on an API-19 emulator, since
`extractNativeLibs="true"` is already the default and seems to be
ignored.

[0]: https://developer.android.com/guide/topics/manifest/application-element#extractNativeLibs
[1]: https://android.googlesource.com/platform/tools/base/+/fd2ac00ab76dfdececce5a25cb7ad23b2b6f5d29%5E%21
@developer9969
Copy link

@brendanzagaeski thank you for the very detailed answer, appreciate it . It helps to explain to management

jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Aug 20, 2020
…tnet#5021)

Fixes: dotnet#4986

The [`//application/@android:extractNativeLibs`][0] attribute within
`AndroidManifest.xml` has had a large-scale impact on `.apk`s:

When `extractNativeLibs` is `true`, then (1) native libraries within
the `.apk` are *compressed*, and (2) the native libraries within the
`.apk` are extracted during installation time.  This allows the `.apk`
to be smaller, while resulting in a *larger* installation footprint,
as two copies of the native libs -- one compressed, one uncompressed --
are present on the device.

When `extractNativeLibs` is `false`, then (1) native libraries within
the `.apk` are stored *uncompressed*, and (2) the native libraries
within the `.apk` are *not* extracted during installation time, but
*only* on API-23 and later devices.  This makes for a larger `.apk`,
but a *smaller* installation footprint.

When *omitted* this value defaults to `true`.

Commit 86737ca bumped `manifest-merger` from 26.5.0 to 27.0.0, which
[altered `manifest-merger.jar` behavior][1] so that it started setting
`//application/@android:extractNativeLibs`=false when `minSdkVersion`
is 23 or higher, "as if" `Properties\AndroidManifest.xml` contained:

	<application android:extractNativeLibs="false">

In the case of Issue dotnet#4986, which is an application which uses
Xamarin.Forms and sets `$(AotAssemblies)`=True, resulting in native
libraries being produced and included for *every assembly*, this
"minor" change to `extractNativeLibs` caused a 24MB `.apk` to balloon
in size to 64MB, which was unexpected.

This unexpected size increase was not caught by unit tests, as our
tests which check for `.apk` size regressions don't use Xamarin.Forms,
and it's AndroidX/Android Support -- used by Xamarin.Forms -- which
causes `manifest-merger.jar` to be used.  No `manifest-merger.jar`,
no unexpected size increase.

This `manifest-merger.jar` change is *also* the underlying cause of
bf657e8 and 8828fef.

To solve this, if not explicitly specified within apps'
`Properties\AndroidManifest.xml`, we will now insert
`android:extractNativeLibs="true"` by default.

This way Xamarin.Android applications will have consistent behavior.
Developers can set `extractNativeLibs="false"` if they prefer that
behavior.

This behavior also works fine on an API-19 emulator, since
`extractNativeLibs="true"` is already the default and seems to be
ignored.

[0]: https://developer.android.com/guide/topics/manifest/application-element#extractNativeLibs
[1]: https://android.googlesource.com/platform/tools/base/+/fd2ac00ab76dfdececce5a25cb7ad23b2b6f5d29%5E%21
jonpryor pushed a commit that referenced this issue Aug 25, 2020
)

Fixes: #4986

The [`//application/@android:extractNativeLibs`][0] attribute within
`AndroidManifest.xml` has had a large-scale impact on `.apk`s:

When `extractNativeLibs` is `true`, then (1) native libraries within
the `.apk` are *compressed*, and (2) the native libraries within the
`.apk` are extracted during installation time.  This allows the `.apk`
to be smaller, while resulting in a *larger* installation footprint,
as two copies of the native libs -- one compressed, one uncompressed --
are present on the device.

When `extractNativeLibs` is `false`, then (1) native libraries within
the `.apk` are stored *uncompressed*, and (2) the native libraries
within the `.apk` are *not* extracted during installation time, but
*only* on API-23 and later devices.  This makes for a larger `.apk`,
but a *smaller* installation footprint.

When *omitted* this value defaults to `true`.

Commit 86737ca bumped `manifest-merger` from 26.5.0 to 27.0.0, which
[altered `manifest-merger.jar` behavior][1] so that it started setting
`//application/@android:extractNativeLibs`=false when `minSdkVersion`
is 23 or higher, "as if" `Properties\AndroidManifest.xml` contained:

	<application android:extractNativeLibs="false">

In the case of Issue #4986, which is an application which uses
Xamarin.Forms and sets `$(AotAssemblies)`=True, resulting in native
libraries being produced and included for *every assembly*, this
"minor" change to `extractNativeLibs` caused a 24MB `.apk` to balloon
in size to 64MB, which was unexpected.

This unexpected size increase was not caught by unit tests, as our
tests which check for `.apk` size regressions don't use Xamarin.Forms,
and it's AndroidX/Android Support -- used by Xamarin.Forms -- which
causes `manifest-merger.jar` to be used.  No `manifest-merger.jar`,
no unexpected size increase.

This `manifest-merger.jar` change is *also* the underlying cause of
bf657e8 and 8828fef.

To solve this, if not explicitly specified within apps'
`Properties\AndroidManifest.xml`, we will now insert
`android:extractNativeLibs="true"` by default.

This way Xamarin.Android applications will have consistent behavior.
Developers can set `extractNativeLibs="false"` if they prefer that
behavior.

This behavior also works fine on an API-19 emulator, since
`extractNativeLibs="true"` is already the default and seems to be
ignored.

[0]: https://developer.android.com/guide/topics/manifest/application-element#extractNativeLibs
[1]: https://android.googlesource.com/platform/tools/base/+/fd2ac00ab76dfdececce5a25cb7ad23b2b6f5d29%5E%21
@brendanzagaeski
Copy link
Contributor

Release status update

A new Preview version of Xamarin.Android has now been published that includes the fix for this item. The fix is not yet included in a Release version. I will update this item again when a Release version is available that includes the fix.

Fix included in Xamarin.Android SDK version 11.1.0.3.

Fix included on Windows in Visual Studio 2019 version 16.8 Preview 3. To try the Preview version that includes the fix, check for the latest updates in Visual Studio Preview.

Fix included on macOS in Visual Studio 2019 for Mac version 8.8 Preview 3. To try the Preview version that includes the fix, check for the latest updates on the Preview updater channel.

@brendanzagaeski
Copy link
Contributor

Release status update

A new Release version of Xamarin.Android has now been published that includes the fix for this item.

Fix included in Xamarin.Android SDK version 11.1.0.17.

Fix included on Windows in Visual Studio 2019 version 16.8. To get the new version that includes the fix, check for the latest updates or install the most recent release from https://visualstudio.microsoft.com/downloads/.

Fix included on macOS in Visual Studio 2019 for Mac version 8.8. To get the new version that includes the fix, check for the latest updates on the Stable updater channel.

@ghost ghost locked as resolved and limited conversation to collaborators Jun 4, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Area: App+Library Build Issues when building Library projects or Application projects. question
Projects
None yet
5 participants