[Xamarin.Android.Build.Tasks] trailing / causes missing classes.dex

Fixes: dotnet#4478
Fixes: dotnet#4486

We with the release of VS for Mac 8.5 going stable have been getting
some reports of:

    error ADB0010:  Deployment failed
    Mono.AndroidTools.InstallFailedException: Failure [INSTALL_FAILED_INVALID_APK: Package couldn't be installed in /data/app/com.contoso.androidapp1-RTZZFPyLkRI7Bk7VDgGkDg==: Package /data/app/com.contoso.androidapp1-RTZZFPyLkRI7Bk7VDgGkDg==/base.apk code is missing]
        at Mono.AndroidTools.Internal.AdbOutputParsing.CheckInstallSuccess (System.String output, System.String packageName) [0x00159] in <65710797f19f43cc9d8f9e05353b9615>:0
        at Mono.AndroidTools.AndroidDevice+<>c__DisplayClass95_0.<InstallPackage>b__0 (System.Threading.Tasks.Task`1[TResult] t) [0x0001c] in <65710797f19f43cc9d8f9e05353b9615>:0
        at System.Threading.Tasks.ContinuationTaskFromResultTask`1[TAntecedentResult].InnerInvoke () [0x00024] in <f9d1b832704f410aa8ec771f4fe80552>:0
        at System.Threading.Tasks.Task.Execute () [0x00000] in <f9d1b832704f410aa8ec771f4fe80552>:0
        --- End of stack trace from previous location where exception was thrown ---
        at AndroidDeviceExtensions.PushAndInstallPackage (Mono.AndroidTools.AndroidDevice device, System.String apkFile, System.String packageName, System.Boolean reinstall, Mono.AndroidTools.Adb.AdbProgressReporter notifyProgress, System.Threading.CancellationToken token) [0x001d3] in <81f66a65e9434daf9d06602cf6ac0bf9>:0
        at AndroidDeviceExtensions.PushAndInstallPackage (Mono.AndroidTools.AndroidDevice device, System.String apkFile, System.String packageName, System.Boolean reinstall, Mono.AndroidTools.Adb.AdbProgressReporter notifyProgress, System.Threading.CancellationToken token) [0x00402] in <81f66a65e9434daf9d06602cf6ac0bf9>:0
        at Xamarin.AndroidTools.AndroidDeploySession.InstallPackage () [0x003be] in <81f66a65e9434daf9d06602cf6ac0bf9>:0
        at Xamarin.AndroidTools.AndroidDeploySession.RunAsync (System.Threading.CancellationToken token) [0x003f8] in <81f66a65e9434daf9d06602cf6ac0bf9>:0
        at Xamarin.AndroidTools.AndroidDeploySession.RunLoggedAsync (System.Threading.CancellationToken token) [0x00077] in <81f66a65e9434daf9d06602cf6ac0bf9>:0  [/Users/macuser/Projects/AndroidApp1/AndroidApp1/AndroidApp1.csproj]

The error means the `.apk` is missing `classes.dex`.

It seems to affect apps where:

  * `AndroidDexTool=dx`, this would need to be set explicitly or it
    would default to `d8`.
  * The issue appears to occur on macOS-only.

Brendan was able to reproduce the issue:

dotnet#4478 (comment)

Using the zip file, there is an `obj` directory sitting next to the
`.sln` file:

  * `obj`
  * `AndroidApp1`
  * `AndroidApp1/AndroidApp1.csproj`
  * `AndroidApp1/obj`
  * `AndroidApp1.sln`

Note that the actual `$(BaseIntermediateOutputPath)` would be
`AndroidApp1/obj`. Brendan just moved an `obj` directory next to the
`.sln` file while trying to reproduce the issue.

However, it appears the presence of this phantom `obj` triggers the
issue! You can rename it to `obj2` to *solve* the issue. It is a
directory with sub-directories, but no files.

If you look at the build log, the main difference is:

    CompileToDalvik
        Parameters
    -       ClassesOutputDirectory = obj/Release/android/bin/classes/
    +       ClassesOutputDirectory = obj/Release/android/bin/classes

The presence of the trailing `/` breaks this code:

    cmd.AppendSwitchIfNotNull ("--output ", Path.GetDirectoryName (ClassesOutputDirectory));

`classes.dex` is output into `obj/Release/android/bin/classes` instead
of `obj/Release/android/bin` and doesn't make it into the final
`.apk`.

So tracking back to what creates the input value of
`ClassesOutputDirectory`, it is the MSBuild property
`$(_AndroidIntermediateJavaClassDirectory)`, it is merely:

    <_AndroidIntermediateJavaClassDirectory>$(IntermediateOutputPath)android\bin\classes\</_AndroidIntermediateJavaClassDirectory>

But then comparing broken vs working logs:

    Broken:
        _InitialBaseIntermediateOutputPath = obj/
        BaseIntermediateOutputPath = obj/
        IntermediateOutputPath = obj/Release/
        _AndroidIntermediateJavaClassDirectory = obj/Release/android/bin/classes/
    Working:
        _InitialBaseIntermediateOutputPath = obj\
        BaseIntermediateOutputPath = obj\
        IntermediateOutputPath = obj\Release\
        _AndroidIntermediateJavaClassDirectory = obj\Release\android\bin\classes\

It appears the existence of this random `obj` directory influences
MSBuild's path combining behavior?

The trailing `\` works, but `/` does not, due to `TrimEnd`:

    <CompileToDalvik
        ...
        ClassesOutputDirectory="$(_AndroidIntermediateJavaClassDirectory.TrimEnd('\'))"

I have no explanation of to why this started occurring. My guess is
there is some change in the behavior of MSBuild running on Mono?

However, after reviewing `<CompileToDalvik/>`, I think we should just
remove the `Path.GetDirectoryName` call. We should just pass in the
directory name we want to use from MSBuild targets.

I could add a test for this scenario by setting a global property:

    _AndroidIntermediateJavaClassDirectory=obj/Debug/android/bin/classes/

Setting a global property could test the trailing `/` character
regardless of MSBuild behavior.

I also cleaned things up a bit by create a new
`$(_AndroidIntermediateDexOutputDirectory)` property.

src/Xamarin.Android.Build.Tasks/Tasks/CompileToDalvik.cs

@@ -114,7 +114,7 @@ protected override string GenerateCommandLineCommands ()
 					cmd.AppendSwitchIfNotNull ("--main-dex-list=", MultiDexMainDexListFile);
 				}
 			}
-			cmd.AppendSwitchIfNotNull ("--output ", Path.GetDirectoryName (ClassesOutputDirectory));
+			cmd.AppendSwitchIfNotNull ("--output ", ClassesOutputDirectory);
 
 			using (var sw = new StreamWriter (path: inputListFile, append: false,
 					encoding: MonoAndroidHelper.UTF8withoutBOM)) {

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingTest.cs

@@ -677,5 +677,23 @@ public void CheckTheCorrectRuntimeAssemblyIsUsedFromNuget ()
 				Assert.IsTrue (xab.LastBuildOutput.ContainsText (expected), $"Build should be using {expected}");
 			}
 		}
+
+		[Test]
+		public void MissingDexFile ()
+		{
+			//NOTE: The trailing / breaks <CompileToDalvik />
+			var parameters = new [] { "_AndroidIntermediateJavaClassDirectory=obj/Debug/android/bin/classes/" };
+			var proj = new XamarinAndroidApplicationProject {
+				DexTool = "dx"
+			};
+			using (var b = CreateApkBuilder ()) {
+				Assert.IsTrue (b.Build (proj, parameters: parameters), "Build should have succeeded.");
+				var apk = Path.Combine (Root, b.ProjectDirectory,
+					proj.IntermediateOutputPath, "android", "bin", "UnnamedProject.UnnamedProject.apk");
+				using (var zip = ZipHelper.OpenZip (apk)) {
+					Assert.IsTrue (zip.ContainsEntry ("classes.dex"), "Apk should contain classes.dex");
+				}
+			}
+		}
 	}
 }

src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets

@@ -1024,6 +1024,7 @@ because xbuild doesn't support framework reference assemblies.
 	<MonoAndroidCodeBehindDir>$(IntermediateOutputPath)generated</MonoAndroidCodeBehindDir>
 	<MonoAndroidIntermediateResourceCache>$(IntermediateOutputPath)resourcecache</MonoAndroidIntermediateResourceCache>
 	<_AndroidIntermediateJavaSourceDirectory>$(IntermediateOutputPath)android\src\</_AndroidIntermediateJavaSourceDirectory>
+	<_AndroidIntermediateDexOutputDirectory>$(IntermediateOutputPath)android\bin\</_AndroidIntermediateDexOutputDirectory>
 	<_AndroidIntermediateJavaClassDirectory>$(IntermediateOutputPath)android\bin\classes\</_AndroidIntermediateJavaClassDirectory>
 	<_AndroidIntermediateClassesZip>$(IntermediateOutputPath)android\bin\classes.zip</_AndroidIntermediateClassesZip>
 	<_AndroidAotBinDirectory>$(IntermediateOutputPath)aot</_AndroidAotBinDirectory>
@@ -2274,8 +2275,7 @@ because xbuild doesn't support framework reference assemblies.
 <Target Name="_CompileDex"
 		DependsOnTargets="$(_BeforeCompileDex);$(_CompileDexDependsOn)">
 	<ItemGroup>
-		<_DexFile Include="$(IntermediateOutputPath)android\bin\dex\*.dex" />
-		<_DexFile Include="$(IntermediateOutputPath)android\bin\*.dex" />
+		<_DexFile Include="$(_AndroidIntermediateDexOutputDirectory)*.dex" />
 	</ItemGroup>
 </Target>
 

src/Xamarin.Android.Build.Tasks/Xamarin.Android.D8.targets

@@ -36,7 +36,7 @@ Copyright (C) 2018 Xamarin. All rights reserved.
 
     <!-- remove existing dex files that may be previous multidex outputs. -->
     <ItemGroup>
-      <_DexesToDelete Include="$(IntermediateOutputPath)android\bin\*.dex" />
+      <_DexesToDelete Include="$(_AndroidIntermediateDexOutputDirectory)*.dex" />
     </ItemGroup>
     <Delete Files="@(_DexesToDelete)" />
 
@@ -52,7 +52,7 @@ Copyright (C) 2018 Xamarin. All rights reserved.
         JavaOptions="$(JavaOptions)"
         JarPath="$(AndroidR8JarPath)"
         AndroidManifestFile="$(IntermediateOutputPath)android\AndroidManifest.xml"
-        OutputDirectory="$(IntermediateOutputPath)android\bin"
+        OutputDirectory="$(_AndroidIntermediateDexOutputDirectory)"
         Debug="$(AndroidIncludeDebugSymbols)"
         JavaPlatformJarPath="$(JavaPlatformJarPath)"
         ClassesZip="$(_AndroidIntermediateClassesZip)"
@@ -79,7 +79,7 @@ Copyright (C) 2018 Xamarin. All rights reserved.
         JavaOptions="$(JavaOptions)"
         JarPath="$(AndroidR8JarPath)"
         AndroidManifestFile="$(IntermediateOutputPath)android\AndroidManifest.xml"
-        OutputDirectory="$(IntermediateOutputPath)android\bin"
+        OutputDirectory="$(_AndroidIntermediateDexOutputDirectory)"
         Debug="$(AndroidIncludeDebugSymbols)"
         JavaPlatformJarPath="$(JavaPlatformJarPath)"
         ClassesZip="$(_AndroidIntermediateClassesZip)"
@@ -92,7 +92,7 @@ Copyright (C) 2018 Xamarin. All rights reserved.
 
     <Touch Files="$(_AndroidStampDirectory)_CompileToDalvik.stamp" AlwaysCreate="true" />
     <ItemGroup>
-      <FileWrites Include="$(IntermediateOutputPath)android\bin\*.dex" />
+      <FileWrites Include="$(_AndroidIntermediateDexOutputDirectory)*.dex" />
     </ItemGroup>
 
   </Target>

src/Xamarin.Android.Build.Tasks/Xamarin.Android.DX.targets

@@ -71,7 +71,7 @@ Copyright (C) 2018 Xamarin. All rights reserved.
 
     <!-- remove existing dex files that may be previous multidex outputs. -->
     <ItemGroup>
-      <_DexesToDelete Include="$(IntermediateOutputPath)android\bin\*.dex" />
+      <_DexesToDelete Include="$(_AndroidIntermediateDexOutputDirectory)*.dex" />
     </ItemGroup>
     <Delete Files="@(_DexesToDelete)" />
 
@@ -82,7 +82,7 @@ Copyright (C) 2018 Xamarin. All rights reserved.
         JavaToolPath="$(JavaToolPath)"
         JavaMaximumHeapSize="$(JavaMaximumHeapSize)"
         JavaOptions="$(JavaOptions)"
-        ClassesOutputDirectory="$(_AndroidIntermediateJavaClassDirectory.TrimEnd('\'))"
+        ClassesOutputDirectory="$(_AndroidIntermediateDexOutputDirectory)"
         ClassesZip="$(_AndroidIntermediateClassesZip)"
         ToolPath="$(DxToolPath)"
         ToolExe="$(DxToolExe)"
@@ -94,7 +94,7 @@ Copyright (C) 2018 Xamarin. All rights reserved.
     />
     <Touch Files="$(_AndroidStampDirectory)_CompileToDalvik.stamp" AlwaysCreate="true" />
     <ItemGroup>
-      <FileWrites Include="$(IntermediateOutputPath)android\bin\*.dex" />
+      <FileWrites Include="$(_AndroidIntermediateDexOutputDirectory)*.dex" />
     </ItemGroup>
 
   </Target>