[One .NET] select defaults for App Bundles (#6087)

Fixes: #6059

Context: https://android-developers.googleblog.com/2020/11/new-android-app-bundle-and-target-api.html
Context: https://blogs.windows.com/windowsexperience/2021/06/24/building-a-new-open-microsoft-store-on-windows-11/
Context: https://developer.amazon.com/blogs/appstore/post/50b1ca0f-bbec-48ec-9eea-3c395efb8f9f/amazon-appstore-to-support-android-app-bundle

The Google Play Store requires Android App Bundles (`.aab` files) for
all new apps as of August 2021, and for all apps in November 2021.
Meanwhile, Microsoft has announced that Windows 11 will support
running Android apps and installing them via the Amazon Appstore,
which currently only supports `.apk` file uploads.

While a Xamarin.Android user could build their app twice, changing
the [`$(AndroidPackageFormat)`][0] between each build, we feel it
would be "better" -- faster, more convenient -- if a single
`SignAndroidPackage` target invocation could produce *both* `.aab`
and `.apk` outputs.

[`bundletool.jar`][1] has the ability to create a universal `.apk`
from the `.aab` file.

Introduce support for a new `$(AndroidPackageFormats)` (plural)
MSBuild property, which is a `;`-delimited sequence of
`$(AndroidPackageFormat)` values to produce as build outputs.
For example, if `$(AndroidPackageFormats)`=`aab;apk`, then *both*
`.aab` and `.apk` outputs will be produced.

	<PropertyGroup>
	  <AndroidPackageFormats>aab;apk</AndroidPackageFormats>
	</PropertyGroup>

The `.apk` output will be generated from the `.aab` file, ensuring
consistency.

In .NET 6 Debug configuration builds, `$(AndroidPackageFormats)` will
default to `.apk`, as this behaves better with Fast Deployment and
developer productivity.

In .NET 6 Release configuration builds, `$(AndroidPackageFormats)`
will default to `aab;apk`, i.e. both `.aab` and `.apk` files will be
produced by default.

In Legacy Xamarin.Android, `$(AndroidPackageFormats)` will not be
set, and `$(AndroidPackageFormat)` will continue to default to `.apk`,
meaning that only `.apk` files will be produced (this week…).

[0]: https://docs.microsoft.com/en-us/xamarin/android/deploy-test/building-apps/build-properties#androidpackageformat
[1]: https://developer.android.com/studio/command-line/bundletool

Documentation/guides/building-apps/build-properties.md

@@ -172,25 +172,25 @@ Added in Xamarin.Android 10.2.
 ## AndroidBoundInterfacesContainConstants
 
 A boolean property that
-determines whether binding constants on interfaces will be supported, 
-or the workaround of creating an `IMyInterfaceConsts` class 
+determines whether binding constants on interfaces will be supported,
+or the workaround of creating an `IMyInterfaceConsts` class
 will be used.
 
 Defaults to `True` in .NET 6 and `False` for legacy.
 
 ## AndroidBoundInterfacesContainStaticAndDefaultInterfaceMethods
 
 A boolean property that
-whether default and static members on interfaces will be supported, 
-or  old workaround of creating a sibling class containing static 
+whether default and static members on interfaces will be supported,
+or  old workaround of creating a sibling class containing static
 members like `abstract class MyInterface`.
 
 Defaults to `True` in .NET 6 and `False` for legacy.
 
 ## AndroidBoundInterfacesContainTypes
 
 A boolean property that
-whether types nested in interfaces will be supported, or the workaround 
+whether types nested in interfaces will be supported, or the workaround
 of creating a non-nested type like `IMyInterfaceMyNestedClass`.
 
 Defaults to `True` in .NET 6 and `False` for legacy.
@@ -914,6 +914,43 @@ properties are set, which are required for Android App Bundles:
 [apk]: https://en.wikipedia.org/wiki/Android_application_package
 [bundle]: https://developer.android.com/platform/technology/app-bundle
 
+This property will be deprecated for .net 6. Users should switch over to
+the newer [`AndroidPackageFormats`](~/android/deploy-test/building-apps/build-properties.md#androidpackageformats).
+
+## AndroidPackageFormats
+
+A semi-colon delimited property with valid values of `apk` and `aab`.
+This indicates if you want to package the Android application as
+an [APK file][apk] or [Android App Bundle][bundle]. App Bundles
+are a new format for `Release` builds that are intended for
+submission on Google Play.
+
+When building a Release build you might want to generate both
+and `aab` and an `apk` for distribution to various stores.
+
+Setting `AndroidPackageFormats` to `aab;apk` will result in both
+being generated. Setting `AndroidPackageFormats` to either `aab`
+or `apk` will generate only one file.
+
+For .net 6 `AndroidPackageFormats` will be set to `aab;apk` for
+`Release` builds only. It is recommended that you continue to use
+just `apk` for debugging.
+
+For Legacy Xamarin.Android this value currently defaults to `""`.
+As a result Legacy Xamarin.Android will NOT by default produce
+both as part of a release build. If a user wants to produce both
+outputs they will need to define the following in their `Release`
+configuration.
+
+```
+<AndroidPackageFormats>aab;apk</AndroidPackageFormats>
+```
+
+You will also need to remove the existing `AndroidPackageFormat` for
+that configuration if you have it.
+
+Added in Xamarin.Android 11.5.
+
 ## AndroidPackageNamingPolicy
 
 An enum-style property for

build-tools/automation/azure-pipelines.yaml

@@ -771,6 +771,7 @@ stages:
         testName: Mono.Android.NET_Tests
         project: tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj
         testResultsFiles: TestResult-Mono.Android.NET_Tests-$(XA.Build.Configuration).xml
+        extraBuildArgs: /p:AndroidPackageFormat=apk
         artifactSource: bin/Test$(XA.Build.Configuration)/net6.0-android/Mono.Android.NET_Tests-Signed.apk
         artifactFolder: net6-Default
         useDotNet: true
@@ -793,7 +794,7 @@ stages:
         testName: Mono.Android.NET_Tests-Interpreter
         project: tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj
         testResultsFiles: TestResult-Mono.Android.NET_Tests-$(XA.Build.Configuration)Interpreter.xml
-        extraBuildArgs: /p:TestsFlavor=Interpreter /p:UseInterpreter=True
+        extraBuildArgs: /p:TestsFlavor=Interpreter /p:UseInterpreter=True /p:AndroidPackageFormat=apk
         artifactSource: bin/Test$(XA.Build.Configuration)/net6.0-android/Mono.Android.NET_Tests-Signed.apk
         artifactFolder: net6-Interpreter
         useDotNet: true
@@ -804,7 +805,7 @@ stages:
         testName: Mono.Android.NET_Tests-Aot
         project: tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj
         testResultsFiles: TestResult-Mono.Android.NET_Tests-$(XA.Build.Configuration)Aot.xml
-        extraBuildArgs: /p:TestsFlavor=Aot /p:RunAOTCompilation=true
+        extraBuildArgs: /p:TestsFlavor=Aot /p:RunAOTCompilation=true /p:AndroidPackageFormat=apk
         artifactSource: bin/Test$(XA.Build.Configuration)/net6.0-android/Mono.Android.NET_Tests-Signed.apk
         artifactFolder: net6-aot
         useDotNet: true
@@ -815,7 +816,7 @@ stages:
         testName: Mono.Android.NET_Tests-AotLlvm
         project: tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj
         testResultsFiles: TestResult-Mono.Android.NET_Tests-$(XA.Build.Configuration)AotLlvm.xml
-        extraBuildArgs: /p:TestsFlavor=AotLlvm /p:RunAOTCompilation=true /p:EnableLlvm=true
+        extraBuildArgs: /p:TestsFlavor=AotLlvm /p:RunAOTCompilation=true /p:EnableLlvm=true /p:AndroidPackageFormat=apk
         artifactSource: bin/Test$(XA.Build.Configuration)/net6.0-android/Mono.Android.NET_Tests-Signed.apk
         artifactFolder: net6-aotllvm
         useDotNet: true

src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.BuildOrder.targets

@@ -37,6 +37,7 @@ projects, these properties are set in Xamarin.Android.Legacy.targets.
       $(BuildDependsOn);
       _CopyPackage;
       _Sign;
+      _CreateUniversalApkFromBundle;
     </BuildDependsOn>
     <IncrementalCleanDependsOn>
       _PrepareAssemblies;

src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.DefaultProperties.targets

@@ -18,7 +18,7 @@
     <!-- mono-symbolicate is not supported -->
     <MonoSymbolArchive>false</MonoSymbolArchive>
     <!--
-      Disable @(Content) from referenced projects 
+      Disable @(Content) from referenced projects
       See: https://github.com/dotnet/sdk/blob/955c0fc7b06e2fa34bacd076ed39f61e4fb61716/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Publish.targets#L16
     -->
     <_GetChildProjectCopyToPublishDirectoryItems>false</_GetChildProjectCopyToPublishDirectoryItems>
@@ -46,6 +46,7 @@
   <PropertyGroup Condition=" '$(Configuration)' == 'Release' ">
     <EmbedAssembliesIntoApk Condition=" '$(EmbedAssembliesIntoApk)' == '' ">true</EmbedAssembliesIntoApk>
     <AndroidManagedSymbols Condition=" '$(AndroidManagedSymbols)' == '' ">true</AndroidManagedSymbols>
+    <AndroidPackageFormats Condition=" '$(AndroidPackageFormats)' == '' " >aab;apk</AndroidPackageFormats>
   </PropertyGroup>
 
   <!-- Application project settings -->

src/Xamarin.Android.Build.Tasks/Microsoft.NET.Sdk.Android/WorkloadManifest.targets

@@ -2,7 +2,7 @@
   <Import Project="Sdk.targets" Sdk="Microsoft.Android.Sdk"
       Condition=" '$(TargetPlatformIdentifier)' == 'android' " />
   <Import Project="Sdk.targets" Sdk="Microsoft.Android.Sdk.BundleTool"
-      Condition=" '$(AndroidPackageFormat)' == 'aab' " />
+      Condition=" '$(TargetPlatformIdentifier)' == 'android' " />
 
   <ItemGroup Condition=" '$(TargetFrameworkIdentifier)' == '.NETCoreApp' and $([MSBuild]::VersionGreaterThanOrEquals($(TargetFrameworkVersion), '6.0')) ">
     <SdkSupportedTargetPlatformIdentifier Include="android" DisplayName="Android" />

src/Xamarin.Android.Build.Tasks/Tasks/AndroidApkSigner.cs

@@ -54,13 +54,16 @@ public class AndroidApkSigner : JavaToolTask
 
 		void AddStorePass (CommandLineBuilder cmd, string cmdLineSwitch, string value)
 		{
+			string pass = value.Replace ("env:", string.Empty)
+				.Replace ("file:", string.Empty)
+				.Replace ("pass:", string.Empty);
 			if (value.StartsWith ("env:", StringComparison.Ordinal)) {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} ", value);
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} env:", pass);
 			}
 			else if (value.StartsWith ("file:", StringComparison.Ordinal)) {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} file:", value.Replace ("file:", string.Empty));
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} file:", pass);
 			} else {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", value);
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", pass);
 			}
 		}
 

src/Xamarin.Android.Build.Tasks/Tasks/AndroidSignPackage.cs

@@ -71,7 +71,9 @@ public class AndroidSignPackage : AndroidRunToolTask
 
 		void AddStorePass (CommandLineBuilder cmd, string cmdLineSwitch, string value)
 		{
-			string pass = value.Replace ("env:", string.Empty).Replace ("file:", string.Empty);
+			string pass = value.Replace ("env:", string.Empty)
+				.Replace ("file:", string.Empty)
+				.Replace ("pass:", string.Empty);
 			if (value.StartsWith ("env:", StringComparison.Ordinal)) {
 				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch}:env ", pass);
 			}

src/Xamarin.Android.Build.Tasks/Tasks/BuildApk.cs

@@ -244,6 +244,10 @@ void ExecuteWithAbi (string [] supportedAbis, string apkInputPath, string apkOut
 								Log.LogDebugMessage ($"Skipping {path} as the archive file is up to date.");
 								continue;
 							}
+							if (string.Compare (Path.GetFileName (name), "AndroidManifest.xml", StringComparison.OrdinalIgnoreCase) == 0) {
+								Log.LogDebugMessage ("Ignoring jar entry {0} from {1}: the same file already exists in the apk", name, Path.GetFileName (jarFile));
+								continue;
+							}
 							if (apk.Archive.Any (e => e.FullName == path)) {
 								Log.LogDebugMessage ("Failed to add jar entry {0} from {1}: the same file already exists in the apk", name, Path.GetFileName (jarFile));
 								continue;
@@ -253,7 +257,7 @@ void ExecuteWithAbi (string [] supportedAbis, string apkInputPath, string apkOut
 								jarItem.Extract (d);
 								data = d.ToArray ();
 							}
-							Log.LogDebugMessage ($"Adding {path} as the archive file is out of date.");
+							Log.LogDebugMessage ($"Adding {path} from {jarFile} as the archive file is out of date.");
 							apk.Archive.AddEntry (data, path);
 						}
 					}

src/Xamarin.Android.Build.Tasks/Tasks/BuildApkSet.cs

@@ -42,6 +42,8 @@ public class BuildApkSet : BundleToolAdbTask
 
 		public string ExtraArgs { get; set; }
 
+		public bool GenerateUniversalApkSet { get; set; } = false;
+
 		public override bool RunTask ()
 		{
 			//NOTE: bundletool will not overwrite
@@ -55,12 +57,15 @@ public override bool RunTask ()
 
 		void AddStorePass (CommandLineBuilder cmd, string cmdLineSwitch, string value)
 		{
+			string pass = value.Replace ("env:", string.Empty)
+				.Replace ("file:", string.Empty)
+				.Replace ("pass:", string.Empty);
 			if (value.StartsWith ("env:", StringComparison.Ordinal)) {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} ", value);
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", Environment.GetEnvironmentVariable (pass));
 			} else if (value.StartsWith ("file:", StringComparison.Ordinal)) {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} file:", value.Replace ("file:", string.Empty));
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} file:", pass);
 			} else {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", value);
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", pass);
 			}
 		}
 
@@ -69,11 +74,15 @@ internal override CommandLineBuilder GetCommandLineBuilder ()
 			var aapt2 = string.IsNullOrEmpty (Aapt2ToolExe) ? Aapt2ToolName : Aapt2ToolExe;
 			var cmd   = base.GetCommandLineBuilder ();
 			cmd.AppendSwitch ("build-apks");
-			cmd.AppendSwitch ("--connected-device");
+			if (GenerateUniversalApkSet) {
+				cmd.AppendSwitchIfNotNull ("--mode ", "universal");
+			} else {
+				cmd.AppendSwitch ("--connected-device");
+				cmd.AppendSwitchIfNotNull ("--mode ", "default");
+				AppendAdbOptions (cmd);
+			}
 			cmd.AppendSwitchIfNotNull ("--bundle ", AppBundle);
 			cmd.AppendSwitchIfNotNull ("--output ", Output);
-			cmd.AppendSwitchIfNotNull ("--mode ", "default");
-			AppendAdbOptions (cmd);
 			cmd.AppendSwitchIfNotNull ("--aapt2 ", Path.Combine (Aapt2ToolPath, aapt2));
 			cmd.AppendSwitchIfNotNull ("--ks ", KeyStore);
 			cmd.AppendSwitchIfNotNull ("--ks-key-alias ", KeyAlias);

src/Xamarin.Android.Build.Tasks/Tasks/Unzip.cs

@@ -14,14 +14,29 @@ public class Unzip : AndroidTask
 
 		public ITaskItem [] Sources { get; set; }
 		public ITaskItem [] DestinationDirectories { get; set; }
+		public ITaskItem [] Files { get; set; }
 
 		public override bool RunTask ()
 		{
 			foreach (var pair in Sources.Zip (DestinationDirectories, (s, d) => new { Source = s, Destination = d })) {
 				if (!Directory.Exists (pair.Destination.ItemSpec))
 					Directory.CreateDirectory (pair.Destination.ItemSpec);
-				using (var z = ZipArchive.Open (pair.Source.ItemSpec, FileMode.Open))
-					z.ExtractAll (pair.Destination.ItemSpec);
+				using (var z = ZipArchive.Open (pair.Source.ItemSpec, FileMode.Open)) {
+					if (Files == null || Files.Length == 0) {
+						z.ExtractAll (pair.Destination.ItemSpec);
+					} else {
+						foreach (var file in Files) {
+							ZipEntry entry = z.ReadEntry (file.ItemSpec);
+							if (entry == null) {
+								Log.LogDebugMessage ($"Skipping not existant file {file.ItemSpec}");
+								continue;
+							}
+							string destinationFileName = file.GetMetadata ("DestinationFileName");
+							Log.LogDebugMessage ($"Extracting {file.ItemSpec} to {destinationFileName ?? file.ItemSpec}");
+							entry.Extract (pair.Destination.ItemSpec, destinationFileName ?? file.ItemSpec);
+						}
+					}
+				}
 			}
 
 			return true;

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AotTests.cs

@@ -210,14 +210,14 @@ public void BuildAotApplicationAndÜmläüts (string supportedAbis, bool enableL
 						"aot", abi, "libaot-UnnamedProject.dll.so");
 					Assert.IsTrue (File.Exists (assemblies), "{0} libaot-UnnamedProject.dll.so does not exist", abi);
 					var apk = Path.Combine (Root, b.ProjectDirectory,
-						proj.IntermediateOutputPath, "android", "bin", $"{proj.PackageName}.apk");
+						proj.OutputPath, $"{proj.PackageName}-Signed.apk");
 					using (var zipFile = ZipHelper.OpenZip (apk)) {
 						Assert.IsNotNull (ZipHelper.ReadFileFromZip (zipFile,
 							string.Format ("lib/{0}/libaot-UnnamedProject.dll.so", abi)),
-							$"lib/{0}/libaot-UnnamedProject.dll.so should be in the {proj.PackageName}.apk", abi);
+							$"lib/{0}/libaot-UnnamedProject.dll.so should be in the {proj.PackageName}-Signed.apk", abi);
 						Assert.IsNotNull (ZipHelper.ReadFileFromZip (zipFile,
 							"assemblies/UnnamedProject.dll"),
-							$"UnnamedProject.dll should be in the {proj.PackageName}.apk");
+							$"UnnamedProject.dll should be in the {proj.PackageName}-Signed.apk");
 					}
 				}
 				Assert.AreEqual (expectedResult, b.Build (proj), "Second Build should have {0}.", expectedResult ? "succeeded" : "failed");
@@ -263,14 +263,14 @@ public void BuildAotApplicationAndBundleAndÜmläüts (string supportedAbis, boo
 						"aot", abi, "libaot-UnnamedProject.dll.so");
 					Assert.IsTrue (File.Exists (assemblies), "{0} libaot-UnnamedProject.dll.so does not exist", abi);
 					var apk = Path.Combine (Root, b.ProjectDirectory,
-						proj.IntermediateOutputPath, "android", "bin", $"{proj.PackageName}.apk");
+						proj.OutputPath, $"{proj.PackageName}-Signed.apk");
 					using (var zipFile = ZipHelper.OpenZip (apk)) {
 						Assert.IsNotNull (ZipHelper.ReadFileFromZip (zipFile,
 							string.Format ("lib/{0}/libaot-UnnamedProject.dll.so", abi)),
-							$"lib/{0}/libaot-UnnamedProject.dll.so should be in the {proj.PackageName}.apk", abi);
+							$"lib/{0}/libaot-UnnamedProject.dll.so should be in the {proj.PackageName}-Signed.apk", abi);
 						Assert.IsNull (ZipHelper.ReadFileFromZip (zipFile,
 							"assemblies/UnnamedProject.dll"),
-							$"UnnamedProject.dll should not be in the {proj.PackageName}.apk");
+							$"UnnamedProject.dll should not be in the {proj.PackageName}-Signed.apk");
 					}
 				}
 				Assert.AreEqual (expectedResult, b.Build (proj), "Second Build should have {0}.", expectedResult ? "succeeded" : "failed");
@@ -410,7 +410,7 @@ public void HybridAOT ([Values ("armeabi-v7a;arm64-v8a", "armeabi-v7a", "arm64-v
 
 				b.Build (proj);
 
-				var apk = Path.Combine (Root, b.ProjectDirectory, proj.OutputPath, $"{proj.PackageName}.apk");
+				var apk = Path.Combine (Root, b.ProjectDirectory, proj.OutputPath, $"{proj.PackageName}-Signed.apk");
 				FileAssert.Exists (apk);
 				using (var zip = ZipHelper.OpenZip (apk)) {
 					var entry = zip.ReadEntry ($"assemblies/{proj.ProjectName}.dll");

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildAssetsTest.cs

@@ -113,7 +113,7 @@ public void CheckAssetsAreIncludedInAPK ([Values (true, false)] bool useAapt2)
 				Assert.IsTrue (libb.Build (libproj), "{0} should have built successfully.", libproj.ProjectName);
 				using (var b = CreateApkBuilder (Path.Combine (projectPath, proj.ProjectName))) {
 					Assert.IsTrue (b.Build (proj), "{0} should have built successfully.", proj.ProjectName);
-					using (var apk = ZipHelper.OpenZip (Path.Combine (Root, b.ProjectDirectory, proj.IntermediateOutputPath, "android", "bin", $"{proj.PackageName}.apk"))) {
+					using (var apk = ZipHelper.OpenZip (Path.Combine (Root, b.ProjectDirectory, proj.OutputPath, $"{proj.PackageName}-Signed.apk"))) {
 						foreach (var a in libproj.OtherBuildItems.Where (x => x is AndroidItem.AndroidAsset)) {
 							var item = a.Include ().ToLower ().Replace ("\\", "/");
 							if (item.EndsWith ("/"))