[One .NET] select defaults for App Bundles

Fixes #6059

Users will probably want to target more than one App Store. Google is now requiring the `aab` format for
Google Play Store uploads. Unfortunately this package format is not compatible with other stores.
Users can build their app twice producing an `aab` for one build and `apk` for another, but we should
try to make this a bit easier.

`bundle-tool` has the ability to create a universal apk from the `aab` file. So lets make use of that to
generate one along side the `aab`. We are introducing a new property `AndroidPackageFormats`.
Under .net 6 this will have a value of `aab;apk` by default for Release builds, for Legacy it will be
empty.

If `AndroidPackageFormats` is specified and `AndroidPackageFormat` is empty we will use the
values in `AndroidPackageFormats` to populate `AndroidPackageFormat`. If none of these values
are provided the `AndroidPackageFormat` will still default to `apk` as it has always done.

The following setting will produce both an `aab` and an `apk`.
```
<PropertyGroup>
  <AndroidPackageFormats>aab;apk</AndroidPackageFormats>
</PropertyGroup>
```

This will produce just an `aab`

```
<PropertyGroup>
  <AndroidPackageFormats>aab</AndroidPackageFormats>
</PropertyGroup>
```

and this will produce just an `apk`

```
<PropertyGroup>
  <AndroidPackageFormats>apk</AndroidPackageFormats>
</PropertyGroup>
```

For .net 6 users this will be enabled by default for Release builds. For legacy users they will need
to define the property manually.

Documentation/guides/building-apps/build-properties.md

@@ -172,25 +172,25 @@ Added in Xamarin.Android 10.2.
 ## AndroidBoundInterfacesContainConstants
 
 A boolean property that
-determines whether binding constants on interfaces will be supported, 
-or the workaround of creating an `IMyInterfaceConsts` class 
+determines whether binding constants on interfaces will be supported,
+or the workaround of creating an `IMyInterfaceConsts` class
 will be used.
 
 Defaults to `True` in .NET 6 and `False` for legacy.
 
 ## AndroidBoundInterfacesContainStaticAndDefaultInterfaceMethods
 
 A boolean property that
-whether default and static members on interfaces will be supported, 
-or  old workaround of creating a sibling class containing static 
+whether default and static members on interfaces will be supported,
+or  old workaround of creating a sibling class containing static
 members like `abstract class MyInterface`.
 
 Defaults to `True` in .NET 6 and `False` for legacy.
 
 ## AndroidBoundInterfacesContainTypes
 
 A boolean property that
-whether types nested in interfaces will be supported, or the workaround 
+whether types nested in interfaces will be supported, or the workaround
 of creating a non-nested type like `IMyInterfaceMyNestedClass`.
 
 Defaults to `True` in .NET 6 and `False` for legacy.
@@ -908,6 +908,43 @@ properties are set, which are required for Android App Bundles:
 [apk]: https://en.wikipedia.org/wiki/Android_application_package
 [bundle]: https://developer.android.com/platform/technology/app-bundle
 
+This property will be deprecated for .net 6. Users should switch over to
+the newer [`AndroidPackageFormats`](~/android/deploy-test/building-apps/build-properties.md#androidpackageformats).
+
+## AndroidPackageFormats
+
+A semi-colon delimited property with valid values of `apk` and `aab`.
+This indicates if you want to package the Android application as
+an [APK file][apk] or [Android App Bundle][bundle]. App Bundles
+are a new format for `Release` builds that are intended for
+submission on Google Play.
+
+When building a Release build you might want to generate both
+and `aab` and an `apk` for distribution to various stores.
+
+Setting `AndroidPackageFormats` to `aab;apk` will result in both
+being generated. Setting `AndroidPackageFormats` to either `aab`
+or `apk` will generate only one file.
+
+For .net 6 `AndroidPackageFormats` will be set to `aab;apk` for
+`Release` builds only. It is recommended that you continue to use
+just `apk` for debugging.
+
+For Legacy Xamarin.Android this value currently defaults to `""`.
+As a result Legacy Xamarin.Android will NOT by default produce
+both as part of a release build. If a user wants to produce both
+outputs they will need to define the following in their `Release`
+configuration.
+
+```
+<AndroidPackageFormats>aab;apk</AndroidPackageFormats>
+```
+
+You will also need to remove the existing `AndroidPackageFormat` for
+that configuration if you have it.
+
+Added in Xamarin.Android 11.5.
+
 ## AndroidPackageNamingPolicy
 
 An enum-style property for

build-tools/automation/azure-pipelines.yaml

@@ -771,6 +771,7 @@ stages:
         testName: Mono.Android.NET_Tests
         project: tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj
         testResultsFiles: TestResult-Mono.Android.NET_Tests-$(XA.Build.Configuration).xml
+        extraBuildArgs: /p:AndroidPackageFormat=apk
         artifactSource: bin/Test$(XA.Build.Configuration)/net6.0-android/Mono.Android.NET_Tests-Signed.apk
         artifactFolder: net6-Default
         useDotNet: true
@@ -793,7 +794,7 @@ stages:
         testName: Mono.Android.NET_Tests-Interpreter
         project: tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj
         testResultsFiles: TestResult-Mono.Android.NET_Tests-$(XA.Build.Configuration)Interpreter.xml
-        extraBuildArgs: /p:TestsFlavor=Interpreter /p:UseInterpreter=True
+        extraBuildArgs: /p:TestsFlavor=Interpreter /p:UseInterpreter=True /p:AndroidPackageFormat=apk
         artifactSource: bin/Test$(XA.Build.Configuration)/net6.0-android/Mono.Android.NET_Tests-Signed.apk
         artifactFolder: net6-Interpreter
         useDotNet: true
@@ -804,7 +805,7 @@ stages:
         testName: Mono.Android.NET_Tests-Aot
         project: tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj
         testResultsFiles: TestResult-Mono.Android.NET_Tests-$(XA.Build.Configuration)Aot.xml
-        extraBuildArgs: /p:TestsFlavor=Aot /p:RunAOTCompilation=true
+        extraBuildArgs: /p:TestsFlavor=Aot /p:RunAOTCompilation=true /p:AndroidPackageFormat=apk
         artifactSource: bin/Test$(XA.Build.Configuration)/net6.0-android/Mono.Android.NET_Tests-Signed.apk
         artifactFolder: net6-aot
         useDotNet: true

build-tools/automation/yaml-templates/apk-instrumentation.yaml

@@ -19,7 +19,7 @@ steps:
       solution: ${{ parameters.project }}
       configuration: ${{ parameters.configuration }}
       msbuildArguments: >-
-        /restore
+        /restore /v:diag
         /t:AcquireAndroidTarget,SignAndroidPackage,DeployTest${{ parameters.packageType }}s,CheckAndRecordApkSizes,RunTestApks,UndeployTestApks,RenameApkTestCases,ReportComponentFailures
         /bl:$(System.DefaultWorkingDirectory)/bin/Test${{ parameters.configuration }}/run-${{ parameters.testName }}.binlog
         ${{ parameters.extraBuildArgs }}
@@ -34,7 +34,7 @@ steps:
       arguments: >-
         -t:AcquireAndroidTarget,SignAndroidPackage,DeployTest${{ parameters.packageType }}s,CheckAndRecordApkSizes,RunTestApks,UndeployTestApks,RenameApkTestCases,ReportComponentFailures
         -bl:$(System.DefaultWorkingDirectory)/bin/Test${{ parameters.configuration }}/run-${{ parameters.testName }}.binlog
-        -v:n -c ${{ parameters.configuration }} ${{ parameters.extraBuildArgs }}
+        -v:d -c ${{ parameters.configuration }} ${{ parameters.extraBuildArgs }}
       condition: ${{ parameters.condition }}
 
 - script: >

src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.BuildOrder.targets

@@ -37,6 +37,7 @@ projects, these properties are set in Xamarin.Android.Legacy.targets.
       $(BuildDependsOn);
       _CopyPackage;
       _Sign;
+      _CreateUniversalApkFromBundle;
     </BuildDependsOn>
     <IncrementalCleanDependsOn>
       _PrepareAssemblies;

src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.DefaultProperties.targets

@@ -18,7 +18,7 @@
     <!-- mono-symbolicate is not supported -->
     <MonoSymbolArchive>false</MonoSymbolArchive>
     <!--
-      Disable @(Content) from referenced projects 
+      Disable @(Content) from referenced projects
       See: https://github.com/dotnet/sdk/blob/955c0fc7b06e2fa34bacd076ed39f61e4fb61716/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Publish.targets#L16
     -->
     <_GetChildProjectCopyToPublishDirectoryItems>false</_GetChildProjectCopyToPublishDirectoryItems>
@@ -46,6 +46,7 @@
   <PropertyGroup Condition=" '$(Configuration)' == 'Release' ">
     <EmbedAssembliesIntoApk Condition=" '$(EmbedAssembliesIntoApk)' == '' ">true</EmbedAssembliesIntoApk>
     <AndroidManagedSymbols Condition=" '$(AndroidManagedSymbols)' == '' ">true</AndroidManagedSymbols>
+    <AndroidPackageFormats Condition=" '$(AndroidPackageFormats)' == '' " >aab;apk</AndroidPackageFormats>
   </PropertyGroup>
 
   <!-- Application project settings -->

src/Xamarin.Android.Build.Tasks/Microsoft.NET.Sdk.Android/WorkloadManifest.targets

@@ -2,7 +2,7 @@
   <Import Project="Sdk.targets" Sdk="Microsoft.Android.Sdk"
       Condition=" '$(TargetPlatformIdentifier)' == 'android' " />
   <Import Project="Sdk.targets" Sdk="Microsoft.Android.Sdk.BundleTool"
-      Condition=" '$(AndroidPackageFormat)' == 'aab' " />
+      Condition=" '$(TargetPlatformIdentifier)' == 'android' " />
 
   <ItemGroup Condition=" '$(TargetFrameworkIdentifier)' == '.NETCoreApp' and $([MSBuild]::VersionGreaterThanOrEquals($(TargetFrameworkVersion), '6.0')) ">
     <SdkSupportedTargetPlatformIdentifier Include="android" DisplayName="Android" />

src/Xamarin.Android.Build.Tasks/Tasks/AndroidApkSigner.cs

@@ -54,13 +54,16 @@ public class AndroidApkSigner : JavaToolTask
 
 		void AddStorePass (CommandLineBuilder cmd, string cmdLineSwitch, string value)
 		{
+			string pass = value.Replace ("env:", string.Empty)
+				.Replace ("file:", string.Empty)
+				.Replace ("pass:", string.Empty);
 			if (value.StartsWith ("env:", StringComparison.Ordinal)) {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} ", value);
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} env:", pass);
 			}
 			else if (value.StartsWith ("file:", StringComparison.Ordinal)) {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} file:", value.Replace ("file:", string.Empty));
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} file:", pass);
 			} else {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", value);
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", pass);
 			}
 		}
 

src/Xamarin.Android.Build.Tasks/Tasks/AndroidSignPackage.cs

@@ -71,7 +71,9 @@ public class AndroidSignPackage : AndroidRunToolTask
 
 		void AddStorePass (CommandLineBuilder cmd, string cmdLineSwitch, string value)
 		{
-			string pass = value.Replace ("env:", string.Empty).Replace ("file:", string.Empty);
+			string pass = value.Replace ("env:", string.Empty)
+				.Replace ("file:", string.Empty)
+				.Replace ("pass:", string.Empty);
 			if (value.StartsWith ("env:", StringComparison.Ordinal)) {
 				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch}:env ", pass);
 			}

src/Xamarin.Android.Build.Tasks/Tasks/BuildApk.cs

@@ -244,6 +244,10 @@ void ExecuteWithAbi (string [] supportedAbis, string apkInputPath, string apkOut
 								Log.LogDebugMessage ($"Skipping {path} as the archive file is up to date.");
 								continue;
 							}
+							if (string.Compare (Path.GetFileName (name), "AndroidManifest.xml", StringComparison.OrdinalIgnoreCase) == 0) {
+								Log.LogDebugMessage ("Ignoring jar entry {0} from {1}: the same file already exists in the apk", name, Path.GetFileName (jarFile));
+								continue;
+							}
 							if (apk.Archive.Any (e => e.FullName == path)) {
 								Log.LogDebugMessage ("Failed to add jar entry {0} from {1}: the same file already exists in the apk", name, Path.GetFileName (jarFile));
 								continue;
@@ -253,7 +257,7 @@ void ExecuteWithAbi (string [] supportedAbis, string apkInputPath, string apkOut
 								jarItem.Extract (d);
 								data = d.ToArray ();
 							}
-							Log.LogDebugMessage ($"Adding {path} as the archive file is out of date.");
+							Log.LogDebugMessage ($"Adding {path} from {jarFile} as the archive file is out of date.");
 							apk.Archive.AddEntry (data, path);
 						}
 					}

src/Xamarin.Android.Build.Tasks/Tasks/BuildApkSet.cs

@@ -42,6 +42,8 @@ public class BuildApkSet : BundleToolAdbTask
 
 		public string ExtraArgs { get; set; }
 
+		public bool GenerateUniversalApkSet { get; set; } = false;
+
 		public override bool RunTask ()
 		{
 			//NOTE: bundletool will not overwrite
@@ -55,12 +57,15 @@ public override bool RunTask ()
 
 		void AddStorePass (CommandLineBuilder cmd, string cmdLineSwitch, string value)
 		{
+			string pass = value.Replace ("env:", string.Empty)
+				.Replace ("file:", string.Empty)
+				.Replace ("pass:", string.Empty);
 			if (value.StartsWith ("env:", StringComparison.Ordinal)) {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} ", value);
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", Environment.GetEnvironmentVariable (pass));
 			} else if (value.StartsWith ("file:", StringComparison.Ordinal)) {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} file:", value.Replace ("file:", string.Empty));
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} file:", pass);
 			} else {
-				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", value);
+				cmd.AppendSwitchIfNotNull ($"{cmdLineSwitch} pass:", pass);
 			}
 		}
 
@@ -69,11 +74,15 @@ internal override CommandLineBuilder GetCommandLineBuilder ()
 			var aapt2 = string.IsNullOrEmpty (Aapt2ToolExe) ? Aapt2ToolName : Aapt2ToolExe;
 			var cmd   = base.GetCommandLineBuilder ();
 			cmd.AppendSwitch ("build-apks");
-			cmd.AppendSwitch ("--connected-device");
+			if (GenerateUniversalApkSet) {
+				cmd.AppendSwitchIfNotNull ("--mode ", "universal");
+			} else {
+				cmd.AppendSwitch ("--connected-device");
+				cmd.AppendSwitchIfNotNull ("--mode ", "default");
+				AppendAdbOptions (cmd);
+			}
 			cmd.AppendSwitchIfNotNull ("--bundle ", AppBundle);
 			cmd.AppendSwitchIfNotNull ("--output ", Output);
-			cmd.AppendSwitchIfNotNull ("--mode ", "default");
-			AppendAdbOptions (cmd);
 			cmd.AppendSwitchIfNotNull ("--aapt2 ", Path.Combine (Aapt2ToolPath, aapt2));
 			cmd.AppendSwitchIfNotNull ("--ks ", KeyStore);
 			cmd.AppendSwitchIfNotNull ("--ks-key-alias ", KeyAlias);

src/Xamarin.Android.Build.Tasks/Tasks/Unzip.cs

@@ -14,14 +14,29 @@ public class Unzip : AndroidTask
 
 		public ITaskItem [] Sources { get; set; }
 		public ITaskItem [] DestinationDirectories { get; set; }
+		public ITaskItem [] Files { get; set; }
 
 		public override bool RunTask ()
 		{
 			foreach (var pair in Sources.Zip (DestinationDirectories, (s, d) => new { Source = s, Destination = d })) {
 				if (!Directory.Exists (pair.Destination.ItemSpec))
 					Directory.CreateDirectory (pair.Destination.ItemSpec);
-				using (var z = ZipArchive.Open (pair.Source.ItemSpec, FileMode.Open))
-					z.ExtractAll (pair.Destination.ItemSpec);
+				using (var z = ZipArchive.Open (pair.Source.ItemSpec, FileMode.Open)) {
+					if (Files == null || Files.Length == 0) {
+						z.ExtractAll (pair.Destination.ItemSpec);
+					} else {
+						foreach (var file in Files) {
+							ZipEntry entry = z.ReadEntry (file.ItemSpec);
+							if (entry == null) {
+								Log.LogDebugMessage ($"Skipping not existant file {file.ItemSpec}");
+								continue;
+							}
+							string destinationFileName = file.GetMetadata ("DestinationFileName");
+							Log.LogDebugMessage ($"Extracting {file.ItemSpec} to {destinationFileName ?? file.ItemSpec}");
+							entry.Extract (pair.Destination.ItemSpec, destinationFileName ?? file.ItemSpec);
+						}
+					}
+				}
 			}
 
 			return true;

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AotTests.cs

@@ -204,14 +204,14 @@ public void BuildAotApplicationAndÜmläüts (string supportedAbis, bool enableL
 						"aot", abi, "libaot-UnnamedProject.dll.so");
 					Assert.IsTrue (File.Exists (assemblies), "{0} libaot-UnnamedProject.dll.so does not exist", abi);
 					var apk = Path.Combine (Root, b.ProjectDirectory,
-						proj.IntermediateOutputPath, "android", "bin", $"{proj.PackageName}.apk");
+						proj.OutputPath, $"{proj.PackageName}-Signed.apk");
 					using (var zipFile = ZipHelper.OpenZip (apk)) {
 						Assert.IsNotNull (ZipHelper.ReadFileFromZip (zipFile,
 							string.Format ("lib/{0}/libaot-UnnamedProject.dll.so", abi)),
-							$"lib/{0}/libaot-UnnamedProject.dll.so should be in the {proj.PackageName}.apk", abi);
+							$"lib/{0}/libaot-UnnamedProject.dll.so should be in the {proj.PackageName}-Signed.apk", abi);
 						Assert.IsNotNull (ZipHelper.ReadFileFromZip (zipFile,
 							"assemblies/UnnamedProject.dll"),
-							$"UnnamedProject.dll should be in the {proj.PackageName}.apk");
+							$"UnnamedProject.dll should be in the {proj.PackageName}-Signed.apk");
 					}
 				}
 				Assert.AreEqual (expectedResult, b.Build (proj), "Second Build should have {0}.", expectedResult ? "succeeded" : "failed");
@@ -259,14 +259,14 @@ public void BuildAotApplicationAndBundleAndÜmläüts (string supportedAbis, boo
 						"aot", abi, "libaot-UnnamedProject.dll.so");
 					Assert.IsTrue (File.Exists (assemblies), "{0} libaot-UnnamedProject.dll.so does not exist", abi);
 					var apk = Path.Combine (Root, b.ProjectDirectory,
-						proj.IntermediateOutputPath, "android", "bin", $"{proj.PackageName}.apk");
+						proj.OutputPath, $"{proj.PackageName}-Signed.apk");
 					using (var zipFile = ZipHelper.OpenZip (apk)) {
 						Assert.IsNotNull (ZipHelper.ReadFileFromZip (zipFile,
 							string.Format ("lib/{0}/libaot-UnnamedProject.dll.so", abi)),
-							$"lib/{0}/libaot-UnnamedProject.dll.so should be in the {proj.PackageName}.apk", abi);
+							$"lib/{0}/libaot-UnnamedProject.dll.so should be in the {proj.PackageName}-Signed.apk", abi);
 						Assert.IsNull (ZipHelper.ReadFileFromZip (zipFile,
 							"assemblies/UnnamedProject.dll"),
-							$"UnnamedProject.dll should not be in the {proj.PackageName}.apk");
+							$"UnnamedProject.dll should not be in the {proj.PackageName}-Signed.apk");
 					}
 				}
 				Assert.AreEqual (expectedResult, b.Build (proj), "Second Build should have {0}.", expectedResult ? "succeeded" : "failed");
@@ -406,7 +406,7 @@ public void HybridAOT ([Values ("armeabi-v7a;arm64-v8a", "armeabi-v7a", "arm64-v
 
 				b.Build (proj);
 
-				var apk = Path.Combine (Root, b.ProjectDirectory, proj.OutputPath, $"{proj.PackageName}.apk");
+				var apk = Path.Combine (Root, b.ProjectDirectory, proj.OutputPath, $"{proj.PackageName}-Signed.apk");
 				FileAssert.Exists (apk);
 				using (var zip = ZipHelper.OpenZip (apk)) {
 					var entry = zip.ReadEntry ($"assemblies/{proj.ProjectName}.dll");

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildAssetsTest.cs

@@ -113,7 +113,7 @@ public void CheckAssetsAreIncludedInAPK ([Values (true, false)] bool useAapt2)
 				Assert.IsTrue (libb.Build (libproj), "{0} should have built successfully.", libproj.ProjectName);
 				using (var b = CreateApkBuilder (Path.Combine (projectPath, proj.ProjectName))) {
 					Assert.IsTrue (b.Build (proj), "{0} should have built successfully.", proj.ProjectName);
-					using (var apk = ZipHelper.OpenZip (Path.Combine (Root, b.ProjectDirectory, proj.IntermediateOutputPath, "android", "bin", $"{proj.PackageName}.apk"))) {
+					using (var apk = ZipHelper.OpenZip (Path.Combine (Root, b.ProjectDirectory, proj.OutputPath, $"{proj.PackageName}-Signed.apk"))) {
 						foreach (var a in libproj.OtherBuildItems.Where (x => x is AndroidItem.AndroidAsset)) {
 							var item = a.Include ().ToLower ().Replace ("\\", "/");
 							if (item.EndsWith ("/"))