[Xamarin.Android.Build.Tasks] generate R.java like Android Studio

Context: https://android.googlesource.com/platform/tools/base/+/refs/heads/master/build-system/builder/
Fixes: xamarin#2680
Fixes: xamarin#2836

The current behavior in the `_GenerateJavaDesignerForComponent`
MSBuild target does the following:

* For each library that has Android resources... (in parallel)
* Run an instance of aapt/aapt2 to generate the `R.java` file for each
  library.
* This actually creates an `R.java` file that contains *every*
  resource id for *every* library. These libraries are not using most
  of these ids.

This has a few problems:

* xamarin#2680 notes a problem where a file is locked on Windows during
  `_GenerateJavaDesignerForComponent`.

    Xamarin.Android.Common.targets(1541,2): The process cannot access the file 'C:\repos\msalnet\tests\devapps\XForms\XForms.Android\obj\Debug\90\lp\26\jl\manifest\AndroidManifest.xml' because it is being used by another process.
        at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
        at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
        at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
        at System.IO.StreamWriter.CreateFile(String path, Boolean append, Boolean checkHost)
        at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize, Boolean checkHost)
        at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding)
        at Xamarin.Android.Tasks.ManifestDocument.Save(String filename)
        at Xamarin.Android.Tasks.Aapt.GenerateCommandLineCommands(String ManifestFile, String currentAbi, String currentResourceOutputFile)
        at Xamarin.Android.Tasks.Aapt.ProcessManifest(ITaskItem manifestFile)
        at System.Threading.Tasks.Parallel.<>c__DisplayClass30_0`2.<ForEachWorker>b__0(Int32 i)
        at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0`1.<ForWorker>b__1()
        at System.Threading.Tasks.Task.InnerInvoke()
        at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
        at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object ) [C:\repos\msalnet\tests\devapps\XForms\XForms.Android\XForms.Android.csproj]

* We are hugely contributing to the dex limit for fields. Apps contain
  exponentially more fields for each library with resources.

An example from @PureWeen:

    1>  trouble writing output: Too many field references to fit in one dex file: 70468; max is 65536.

* Quite a few instances of aapt/aapt2 startup on developer's machines:
  this pegs the CPU. We have had a few general complaints about it.

Reviewing the source code for the Android gradle plugin, here is what
they do:

* Build the main app's "full" `R.txt` file.
* For each library, load its `R.txt` file.
* Map each resource in the library's `R.txt` back to the main app
* Write a small `R.java` file for each library: containing *only* the
  lines from the `R.txt` and updated integer values from the main app
  `R.txt` file.

Looking into this, we can do the exact same thing? We have the `R.txt`
file one directory above where we extract resources for each library.
We already had code parsing `R.txt` files I could repurpose, the only
thing *new* is a `R.java` writer: a pretty simple port from java.

The results are great!

    Before:
    3173 ms  _GenerateJavaDesignerForComponentAapt2     1 calls
    After:
      20 ms  GenerateLibraryResources                   1 calls

`_GenerateJavaDesignerForComponent` is now completely gone. This is a
total savings of ~3 seconds on first build and incremental builds
with library changes.

To compare APKs, I used:

    $ ~/android-toolchain/sdk/tools/bin/apkanalyzer dex packages Xamarin.Forms_Performance_Integration-Before.apk | grep ^F

Which omits a line for each field such as:

    F d 0	0	16	xamarin.forms_performance_integration.R$color int abc_background_cache_hint_selector_material_dark

So then, before these changes:

    $ ~/android-toolchain/sdk/tools/bin/apkanalyzer dex packages Xamarin.Forms_Performance_Integration-Before.apk | grep ^F | wc -l
    29681

After:

    $ ~/android-toolchain/sdk/tools/bin/apkanalyzer dex packages Xamarin.Forms_Performance_Integration-After.apk | grep ^F | wc -l
    17210

12K less fields in a "Hello World" Xamarin.Forms app!

Comparing file sizes seems good, too:

    $ zipinfo Xamarin.Forms_Performance_Integration-Before.apk | grep classes.dex
    -rw-rw-r--  6.3 unx  3657872 b- defX 19-Mar-28 16:37 classes.dex
    $ zipinfo Xamarin.Forms_Performance_Integration-After.apk | grep classes.dex
    -rw-rw-r--  6.3 unx  3533120 b- defX 19-Mar-28 16:20 classes.dex

Dex file in the APK is ~120KB smaller.

src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Aapt2.targets

@@ -138,32 +138,4 @@ Copyright (C) 2011-2012 Xamarin. All rights reserved.
   </ItemGroup>
 </Target>
 
-<Target Name="_GenerateJavaDesignerForComponentAapt2"
-    Condition=" '$(_AndroidUseAapt2)' == 'True' "
-    Inputs="@(_AdditonalAndroidResourceCacheFiles);@(_LibraryResourceDirectoryStamps);$(_AndroidResgenFlagFile)"
-    Outputs="$(_AndroidComponentResgenFlagFile)">
-  <!-- Run aapt to generate R.java for additional Android resources-->
-  <Aapt2Link
-      Condition=" '$(_AndroidUseAapt2)' == 'True' "
-      ContinueOnError="$(DesignTimeBuild)"
-      AdditionalAndroidResourcePaths="@(_LibraryResourceHashDirectories)"
-      AdditionalResourceArchives="@(_LibraryResourceHashDirectories->'$(_AndroidLibraryFlatArchivesDirectory)%(Hash).flata')"
-      ApplicationName="$(_AndroidPackage)"
-      AssemblyIdentityMapFile="$(_AndroidLibrayProjectAssemblyMapFile)"
-      CompiledResourceFlatArchive="$(_AndroidLibraryFlatArchivesDirectory)\compiled.flata"
-      ExtraArgs="$(AndroidAapt2LinkExtraArgs)"
-      ImportsDirectory="$(_LibraryProjectImportsDirectoryName)"
-      JavaDesignerOutputDirectory="$(IntermediateOutputPath)android\src"
-      JavaPlatformJarPath="$(JavaPlatformJarPath)"
-      ManifestFiles="@(_AdditonalAndroidResourceCachePaths->'%(Identity)\AndroidManifest.xml');@(LibraryResourceDirectories->'%(Identity)\..\AndroidManifest.xml')"
-      OutputImportDirectory="$(_AndroidLibrayProjectIntermediatePath)"
-      ResourceNameCaseMap="$(_AndroidResourceNameCaseMap)"
-      ResourceDirectories="$(MonoAndroidResDirIntermediate)"
-      ToolPath="$(Aapt2ToolPath)"
-      ToolExe="$(Aapt2ToolExe)"
-      UseShortFileNames="$(UseShortFileNames)"
-      YieldDuringToolExecution="$(YieldDuringToolExecution)"
-  />
-  <Touch Files="$(_AndroidComponentResgenFlagFile)" AlwaysCreate="True" />
-</Target>
 </Project>

src/Xamarin.Android.Build.Tasks/Tasks/GenerateLibraryResources.cs

@@ -0,0 +1,200 @@
+using Microsoft.Build.Framework;
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Text;
+using Xamarin.Android.Tools;
+using Xamarin.Build;
+using ThreadingTasks = System.Threading.Tasks;
+
+namespace Xamarin.Android.Tasks
+{
+	/// <summary>
+	/// We used to invoke aapt/aapt2 per library (many times!), this task does the work to generate R.java for libraries without calling aapt/aapt2.
+	/// </summary>
+	public class GenerateLibraryResources : AsyncTask
+	{
+		/// <summary>
+		/// The main R.txt for the app
+		/// </summary>
+		[Required]
+		public string ResourceSymbolsTextFile { get; set; }
+
+		/// <summary>
+		/// The output directory for Java source code, such as: $(IntermediateOutputPath)android\src
+		/// </summary>
+		[Required]
+		public string OutputDirectory { get; set; }
+
+		/// <summary>
+		/// The list of R.txt files for each library
+		/// </summary>
+		public string [] LibraryTextFiles { get; set; }
+
+		/// <summary>
+		/// The accompanying manifest file for each library
+		/// </summary>
+		public string [] ManifestFiles { get; set; }
+
+		public override bool Execute ()
+		{
+			Yield ();
+			try {
+				var task = ThreadingTasks.Task.Run (() => {
+					DoExecute ();
+				}, CancellationToken);
+
+				task.ContinueWith (Complete);
+
+				base.Execute ();
+			} finally {
+				Reacquire ();
+			}
+
+			return !Log.HasLoggedErrors;
+		}
+
+		string output_directory;
+		Dictionary<Tuple<string, string>, string> main_r_txt;
+
+		void DoExecute ()
+		{
+			if (LibraryTextFiles == null || LibraryTextFiles.Length == 0)
+				return;
+
+			main_r_txt = new Dictionary<Tuple<string, string>, string> ();
+			using (var reader = File.OpenText (ResourceSymbolsTextFile)) {
+				foreach (var line in ParseFile (reader)) {
+					var key = new Tuple<string, string> (line [Index.Class], line [Index.Name]);
+					main_r_txt [key] = line [Index.Value];
+				}
+			}
+
+			Directory.CreateDirectory (OutputDirectory);
+			//So we don't need to consider WorkingDirectory on background threads
+			output_directory = Path.GetFullPath (OutputDirectory);
+
+			ThreadingTasks.ParallelOptions options = new ThreadingTasks.ParallelOptions {
+				CancellationToken = CancellationToken,
+				TaskScheduler = ThreadingTasks.TaskScheduler.Default,
+			};
+
+			ThreadingTasks.Parallel.For (0, LibraryTextFiles.Length, options, ProcessDirectory);
+		}
+
+		void ProcessDirectory (int index)
+		{
+			var r_txt = LibraryTextFiles [index];
+			if (!File.Exists (r_txt)) {
+				LogDebugMessage ($"Skipping, R.txt does not exist: {r_txt}");
+				return;
+			}
+
+			var manifestFile = ManifestFiles [index];
+			if (!File.Exists (manifestFile)) {
+				LogDebugMessage ($"Skipping, AndroidManifest.xml does not exist: {manifestFile}");
+				return;
+			}
+
+			var manifest = AndroidAppManifest.Load (manifestFile, MonoAndroidHelper.SupportedVersions);
+
+			using (var memory = new MemoryStream ())
+			using (var writer = new StreamWriter (memory, Encoding)) {
+				// This code is based on the Android gradle plugin
+				// https://android.googlesource.com/platform/tools/base/+/908b391a9c006af569dfaff08b37f8fdd6c4da89/build-system/builder/src/main/java/com/android/builder/internal/SymbolWriter.java
+
+				writer.WriteLine ("/* AUTO-GENERATED FILE. DO NOT MODIFY.");
+				writer.WriteLine (" *");
+				writer.WriteLine (" * This class was automatically generated by");
+				writer.WriteLine (" * Xamarin.Android from the resource data it found.");
+				writer.WriteLine (" * It should not be modified by hand.");
+				writer.WriteLine (" */");
+
+				writer.Write ("package ");
+				writer.Write (manifest.PackageName);
+				writer.WriteLine (';');
+				writer.WriteLine ();
+				writer.WriteLine ("public final class R {");
+
+				using (var reader = File.OpenText (r_txt)) {
+					string currentClass = null;
+					foreach (var line in ParseFile (reader)) {
+						var type  = line [Index.Type];
+						var clazz = line [Index.Class];
+						var name  = line [Index.Name];
+						var key = new Tuple<string, string> (clazz, name);
+						if (main_r_txt.TryGetValue (key, out string value)) {
+							if (clazz != currentClass) {
+								// If not the first inner class
+								if (currentClass != null) {
+									writer.WriteLine ("\t}");
+								}
+
+								currentClass = clazz;
+								writer.Write ("\tpublic static final class ");
+								writer.Write (currentClass);
+								writer.WriteLine (" {");
+							}
+
+							writer.Write ("\t\tpublic static final ");
+							writer.Write (type);
+							writer.Write (' ');
+							writer.Write (name);
+							writer.Write (" = ");
+							// It may be an int[]
+							if (value.StartsWith ("{", StringComparison.Ordinal)) {
+								writer.Write ("new ");
+								writer.Write (type);
+								writer.Write (' ');
+							}
+							writer.Write (value);
+							writer.WriteLine (';');
+						} else {
+							LogDebugMessage ($"{r_txt}: `{type} {clazz} {name}` value not found");
+						}
+					}
+
+					// If we wrote at least one inner class
+					if (currentClass != null) {
+						writer.WriteLine ("\t}");
+					}
+					writer.WriteLine ('}');
+				}
+
+				writer.Flush ();
+				var r_java = Path.Combine (output_directory, manifest.PackageName.Replace ('.', Path.DirectorySeparatorChar), "R.java");
+				if (MonoAndroidHelper.CopyIfStreamChanged (memory, r_java)) {
+					LogDebugMessage ($"Writing: {r_java}");
+				} else {
+					LogDebugMessage ($"Up to date: {r_java}");
+				}
+			}
+		}
+
+		static readonly Encoding Encoding = new UTF8Encoding (encoderShouldEmitUTF8Identifier: false);
+		static readonly char [] Delimiter = new [] { ' ' };
+
+		class Index
+		{
+			public const int Type  = 0;
+			public const int Class = 1;
+			public const int Name  = 2;
+			public const int Value = 3;
+		}
+
+		/// <summary>
+		/// R.txt is of the format:
+		///    int id icon 0x7f0c000a
+		///    int[] styleable ViewStubCompat { 0x010100d0, 0x010100f2, 0x010100f3 }
+		/// This returns a 4-length string[] of the parts.
+		/// </summary>
+		IEnumerable<string []> ParseFile (StreamReader reader)
+		{
+			while (!reader.EndOfStream) {
+				var line = reader.ReadLine ();
+				var items = line.Split (Delimiter, 4);
+				yield return items;
+			}
+		}
+	}
+}