JetBrains OpenJDK 11

[grendello]

Moved from: #4562

This PR is a test to see what works and what breaks with JetBrains
OpenJDK 11 as far as Xamarin.Android is concerned.

[jonpryor]

Various fixes required to allow the updated Java.Interop to build are also part of #4569.

PR #4569 should be merged before this.

[jonpryor]

Depends on: https://github.com/xamarin/monodroid/pull/1083

[jonpryor]

Looks like macOS is able to build, but not Windows:

  "C:\Users\dlab14\android-toolchain\sdk\build-tools\29.0.2\dx" --dex --no-strict --output="C:\A\vs2019xam00000H-1\_work\2\s\bin\Release\lib\xamarin.android\xbuild\Xamarin\Android\java_runtime.dex" "C:\A\vs2019xam00000H-1\_work\2\s\bin\Release\lib\xamarin.android\xbuild\Xamarin\Android\java_runtime.jar"
  
##[debug]Processed: ##vso[task.logdetail id=dffb25d4-cb0b-4e58-94d2-4b9a4f215bed;parentid=;name=EXEC;type=Build;starttime=2020-04-19T01:21:10.0332656Z;state=InProgress;]
##[error]EXEC(0,0): Error : No suitable Java found. In order to properly use the Android Developer
##[debug]Processed: ##vso[task.logissue type=Error;sourcepath=EXEC;linenumber=0;columnnumber=0;code=;]No suitable Java found. In order to properly use the Android Developer
##[debug]Processed: ##vso[task.logdetail id=dffb25d4-cb0b-4e58-94d2-4b9a4f215bed;parentid=;type=Build;result=Failed;finishtime=2020-04-19T01:21:10.0488457Z;progress=100;state=Completed;parentid=;name=;]
EXEC : error : No suitable Java found. In order to properly use the Android Developer [C:\A\vs2019xam00000H-1\_work\2\s\src\java-runtime\java-runtime.csproj]
  Tools, you need a suitable version of Java JDK installed on your system.
  We recommend that you install the JDK version of JavaSE, available here:
    http://www.oracle.com/technetwork/java/javase/downloads
  
  If you already have Java installed, you can define the JAVA_HOME environment
  variable in Control Panel / System / Avanced System Settings to point to the
  JDK folder.
  
  You can find the complete Android SDK requirements here:
    http://developer.android.com/sdk/requirements.html
  
##[error]src\java-runtime\java-runtime.targets(59,3): Error MSB3073: The command ""C:\Users\dlab14\android-toolchain\sdk\build-tools\29.0.2\dx" --dex --no-strict --output="C:\A\vs2019xam00000H-1\_work\2\s\bin\Release\lib\xamarin.android\xbuild\Xamarin\Android\java_runtime.dex" "C:\A\vs2019xam00000H-1\_work\2\s\bin\Release\lib\xamarin.android\xbuild\Xamarin\Android\java_runtime.jar"" exited with code -1.

[jonpryor]

Discussed the Windows failure, and the consensus was that we should java-runtime.targets to instead use r8.jar:

https://github.com/xamarin/xamarin-android/blob/ee3a0c582c82c91936020611e8081d644ec2fbaa/src/java-runtime/java-runtime.targets#L59-L62

[jonpryor]

The AllActivityAttributeProperties("manifestmerger.jar") unit test fails:

error AMM0000: java.lang.UnsupportedClassVersionError: com/xamarin/manifestmerger/Main has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0 [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.lang.ClassLoader.defineClass1(Native Method) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.security.AccessController.doPrivileged(Native Method) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: 	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495) [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: Error: A JNI error has occurred, please check your installation and try again [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: Exception in thread "main"  [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]
error AMM0000: java.lang.UnsupportedClassVersionError :  com/xamarin/manifestmerger/Main has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0 [C:\A\vs2019xam00000B-1\_work\2\s\bin\TestRelease\temp\AllActivityAttributePropertiesmanifestmerger.jar\UnnamedProject.csproj]

It appears that if our Gradle-based projects such as src/manifestmerger don't explicitly specify the Java target version, it uses the JDK version -- which make sense! -- and thus implicitly requires JDK 11 on the customer machines.

We'll need to update our Gradle projects accordingly.

[pjcollins]

There are some extra characters in this line that need to be removed.

[jonpryor]

Current build: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=3791298&view=results

[jonpryor]

Squash-and-merge summary:

[ci] Preliminary JDK11 Support (#4567)

Body:

Context: https://issuetracker.google.com/issues/150189789
Context: https://developer.android.com/preview/features#signature-scheme-v4
Context: https://developer.android.com/preview/features#incremental

Preliminary JDK 11 support.

The Android SDK Build-tools r30.0.0-rc4 package currently requires
JDK 11 in order to use `apksigner.jar`:

	Task "AndroidApkSigner"
	  Task Parameter:ApkSignerJar=C:\Users\dlab14\android-toolchain\sdk\build-tools\30.0.0-rc4\lib\apksigner.jar
	  …
	  Task Parameter:ToolPath=C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin
	  Task Parameter:ManifestFile=obj\Release\android\AndroidManifest.xml
	  C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin\java.exe -jar C:\Users\dlab14\android-toolchain\sdk\build-tools\30.0.0-rc4\lib\apksigner.jar sign --ks "C:\Users\dlab14\AppData\Local\Xamarin\Mono for Android\debug.keystore" --ks-pass pass:android --ks-key-alias androiddebugkey --key-pass pass:android --min-sdk-version 21 --max-sdk-version 29  "C:\A\vs2019xam00000Y-1\_work\1\s\bin\TestRelease\temp\BuildAotApplication AndÜmläüts_x86_64_True_True\bin\Release\UnnamedProject.UnnamedProject-Signed.apk"
	  java.lang.UnsupportedClassVersionError: com/android/apksigner/ApkSignerTool has been compiled by a more recent version of the Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes class file versions up to 52.0
	  at java.lang.ClassLoader.defineClass1(Native Method)
	  …
	…\Xamarin.Android.Common.targets(2559,2): error MSB6006: "java.exe" exited with code 1.

This could be triggered by using the Android SDK Manager to install
Build-tools r30.0.0-rc4, then overriding the
`$(AndroidSdkBuildToolsVersion)` MSBuild property in `App.csproj`:

	<PropertyGroup>
	  <AndroidSdkBuildToolsVersion>30.0.0-rc4</AndroidSdkBuildToolsVersion>
	</PropertyGroup>

Many previous commits have added support for JDK 11, e.g. d99facb1,
dcee2c84, 4742d50d, 89f3dc15, and 895b7bc5 (among many others).

Additionally, c8ab4555 updated the build environment so that *both*
JetBrains OpenJDK 1.8 and 11 were installed, though 1.8 was still used.

Now it's time to "flip the build environment," and use JDK 11 to build
the product, installers, and run (most) unit tests:

  * Update `azure-pipelines.yaml` and `azure-pipelines-oss.yaml` so
    that JDK 11 is used to build everything.

  * Update `xaprepare` so that when `$(JavaSdkDirectory)` isn't
    overridden, JDK 11 is used by default.

  * Override the `JI_JAVA_HOME` environment variable when preparing
    `external/Java.Interop` so that it uses the desired JDK.

  * Update the default `$(LatestSupportedJavaVersion)` value to
    11.0.4, the version of the JDK we install.

  * Update the `<ValidateJavaVersions/>` task so that JDK 11 is
    required when `$(AndroidSdkBuildToolsVersion)` is >= r30.
    This will cause an XA0032 error if JDK 1.8 is used to build a
    project when `$(AndroidSdkBuildToolsVersion)` is >= r30.

  * `apksigner` from Build-tools r30 creates a new `.idsig` file
    next to the `.apk`.  This is to enable a new
    `adb install --incremental` feature.  Add the `.idsig` file
    to `@(FileWrites)`.

  * Update various unit test `.csproj` files to consistently
    `<Import/>` the `Configuration.props` file and use properties
    within it for default values.  This in particular allowed building
    e.g. `samples/HelloWorld` to implicitly use JDK 11.

  * Increase "expected" times in
    `tests/msbuild-times-reference/MSBuildDeviceIntegration.csv`
    as JDK 11 appears to be slower than JDK 1.8 (🙁).

  * *Disable* support for `@(JavaSourceJar)` when JDK 11 is used.
    JDK 11 changes the HTML generated by the `javadoc` command, and
    we need to update our HTML parser to handle it.  In the interest
    of expediency, disable this for now.

The Android Designer integration tests continue to use JDK 1.8,
as the Designer doesn't currently build under JDK 11.

TODO:

  * Fix `@(JavaSourceJar)` support
  * Allow Android Designer to work with JDK 11, run Designer
    integration tests using JDK 11.
  * Figure out how to not accidentally bitrot JDK 1.8 support.

Co-authored-by: Jonathan Pryor <jonpryor@vt.edu>
Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Co-authored-by: Jonathan Pobst <jonathan.pobst@microsoft.com>
Co-authored-by: Peter Collins <pecolli@microsoft.com>

[brendanzagaeski]

A question to try to answer briefly in this note is why users would be interested in this change. The answer to that might depend on whether Visual Studio will start installing the JetBrains Runtime OpenJDK 11 by default in the same release that contains this Xamarin.Android change.

Some possibilities:

• If it's not yet known when Visual Studio will start installing OpenJDK 11 by default, then one idea is to include an HTML comment in this draft release note that says something like:

  <!-- This release note might change depending on when Visual Studio starts
  installing OpenJDK 11 by default for the Mobile development with .NET workload.
  -->
  

• If it is already known that Visual Studio will not install OpenJDK 11 by default alongside the first Xamarin.Android release that contains this change, then if possible, it could be good to add a sentence about when users might want to go to the trouble of finding and installing JDK 11 themselves.

  I think the main known reason at this time is for compatibility with Android SDK Build-Tools 30-rc1 and higher, which leads to a question of when users would need to update to that version of Android SDK Build-Tools.

  If the majority of users won't care about using Android SDK Build-Tools 30-rc1 or higher yet, the note could mention that, maybe using wording similar to JetBrains OpenJDK 11 #4567 (comment), and add something like:

  Future versions of the Android SDK might require JDK 11 for more scenarios.
  Project authors can try switching to JDK 11 now to prepare early for those
  potential future requirements.
  

[jonpryor]

Visual Studio will start installing the JetBrains Runtime OpenJDK 11 by default in the same release that contains this Xamarin.Android change.

Almost certainly not, not for d16-7 P3 anyway, and I really doubt the likelihood of d16-7 period. For starters, it's only needed if Build-tools r30 is used, and --at present! -- it's not required to target API-R/Android 11.

(Even if it were required, I rather doubt the number of people who will be using Android 11 in the preview this is released in.)

More importantly, the Android Designer doesn't currently work with JDK 11. We can't change the default, even if we wanted to.

Additionally, I don't know what the rest of the VS "ecosystem" looks like, and whether it can be updated to JDK 11.

Finally, let's take a gander at API-29, shall we? That didn't become the default $(TargetFrameworkVersion) until d16-6, ~9 months after a Xamarin.Android release containing a stable Android 10.0 binding.

I think there is no chance at all of migrating VS to JDK 11 "soon."*

*Definition of "soon" is deliberately undefined.''

If it's not yet known when Visual Studio will start installing OpenJDK 11 by default…

Will do, though I think it'll be "moot." One of the (random) ideas is that we'll delete the Documentation/release-notes contents when a release note has gone stable. My guess is that XA d16-7 will be stable long before VS adds JDK 11, at which point this release note will likely not exist.

If it is already known that Visual Studio will not install OpenJDK 11 by default

99.99% certain of this.

it could be good to add a sentence about when users might want to go to the trouble of finding and installing JDK 11 themselves.

I'm not sure I can write this, not quickly. The only reason I know of to want/need JDK11 support is for Android SDK Build-tools r30+, but why would anybody want to use that, especially considering that it's still a "release candidate"? I don't know why this would be desirable.

I like the blurb you provided.

[brendanzagaeski]

If Visual Studio will not be installing OpenJDK 11 by default yet in the first release with this change, then I think it would be good to include a link to a suggested download location, such as perhaps https://confluence.jetbrains.com/display/JBR/JetBrains+Runtime.

[brendanzagaeski]

Can project authors configure the JDK via the Java Development Kit location setting in Visual Studio options (for example, on Windows: Tools > Options > Xamarin > Android Settings > Java Development Kit Location)? If so, I think it could be good to mention that approach first.

Then, if a different approach might work best on CI systems, such as Azure Pipelines, that could be included as a second option.

[jonpryor]

Looks like these locations have documentation for how to set that value:

• https://docs.microsoft.com/en-us/xamarin/android/troubleshooting/questions/update-jdk?tabs=windows
• https://docs.microsoft.com/en-us/xamarin/android/get-started/installation/windows#configuration

[brendanzagaeski]

It could be handy to create a GitHub issue for this known issue so that it will already have a tracking location, but it's OK to skip that for now too.

[brendanzagaeski]

It looks like there's a word or two missing here unintentionally, probably just due to rearrangements in editing. Maybe this can be something like:

This may result in different parameter names for existing bindings projects.
Authors of bindings projects affected by this change are recommended to continue
to use JDK 8 to build the bindings projects until this issue is resolved.

The additional candidate sentence also helps reinforce the point that this issue is primarily relevant to bindings projects authors and doesn't affect app project authors who are only consuming bindings provided by the Xamarin team.