refactor(android): support for Titanium 9.0.0 and gradle

[jquick-axway]

JIRA:

• https://jira.appcelerator.org/browse/TIMOB-27685
• https://jira.appcelerator.org/browse/TIMOB-27297
• https://jira.appcelerator.org/browse/TIMOB-27298

---

Note:
This PR requires Titanium PR tidev/titanium-sdk#11339 to be merged-in first since it adds gradle support.

---

Summary:

• Updated module version to 5.0.0.
• Now builds with gradle to an AAR library.
• Added x86_64 support.
• Updated dexmaker library from version 1.4 to 2.25.1.
• Update min supported Titanium SDK version to 9.0.0.
  • Depends on Titanium's new gradle build system.
  • Old hyperloop module versions will not work with 9.0.0.
• Removed "ant" build usage. Replaced with gradle and is built as an AAR.
• Added support for optional ./platform/android/build.gradle file in app project.
  • Intended to provide additional dependencies to hyperloop.
  • JAR/AAR libraries in this directory are still supported.
• Modified module to generate a hyperloop.bootstrap.js file.
  • Binds Java class names referenced by require/import to hyperloop generated JS file.
  • Module no longer replaces JS source code's require/import string.
• Added support to new Titanium build hook android.build.requestResourcesDirPaths.
  • Used to add generated JS files directory Resource/hyperloop to app build.
  • Goes through Android "_build.js" script's copyResources() function.
• TIMOB-27298 Fixed bug where you could not access a Java inner class from an inner class.
• TIMOB-27297 Added support for accessing Titanium library's Java classes.

---

Benchmarks:
Performed with hyperloop-example between Titanium 8.3.0 and 9.0.0.

Non-encrypted emulator builds:
- Clean Build: 47s -> 30s  (1.6x faster)
- Incremental: 21s -> 12s  (1.8x faster)

Encrypted device/production builds:
- Clean Build: 98s -> 31s  (3.2x faster)
- Incremental: 41s -> 19s  (2.2x faster)

JS proxy files generated: 1406

---

General Hyperloop Test:

1. Build and run hyperloop-examples for Android with Titanium 9.0.0.
2. Tap through each test in TableView verifying each test still works.
3. Rebuild and re-run app on Android. (Tests incremental build.)
4. Tap through each test in TableView again.

---

Titanium Java Class Access Test:

1. Run test procedure in TIMOB-27297.
2. Verify that ### Is app in foreground: true get logged every 1 second.

---

Inner Java Class Access Test:

1. Run test procedure in TIMOB-27298.
2. Verify that an exception does not occur and EXTERNAL_CONTENT_URI gets logged.

---

Java Package and Gradle File Test:
Tests "build.gradle" dependency support and require/import of Java packages.

1. Create a Class app project.
2. Add "hyperloop" module to "tiapp.xml".
3. Add the below "build.gradle" file to app project's "platform/android" folder.
4. Copy below "app.js" script to project.
5. Build and run on Android.
6. Verify that the following gets logged.

[INFO]  @@@ ExoPlayer MimeTypes.VIDEO_WEBM: video/webm
[INFO]  @@@ ExoPlayer MimeTypes.VIDEO_H264: video/avc
[INFO]  @@@ ExoPlayer MimeTypes.VIDEO_MP4: video/mp4
[INFO]  @@@ ExoPlayer MimeTypes.VIDEO_MPEG: video/mpeg

./platform/android/build.gradle

repositories {
	google()
	jcenter()
}
dependencies {
	implementation 'com.google.android.exoplayer:exoplayer-core:2.8.0'
}

app.js

import { MimeTypes } from "com.google.android.exoplayer2.util.*";
Ti.API.info("@@@ ExoPlayer MimeTypes.VIDEO_WEBM: " + MimeTypes.VIDEO_WEBM); // = "video/webm"

import MimeTypes1 from "com.google.android.exoplayer2.util.MimeTypes";
Ti.API.info("@@@ ExoPlayer MimeTypes.VIDEO_H264: " + MimeTypes1.VIDEO_H264); // = "video/avc"

const MimeTypes2 = require("com.google.android.exoplayer2.util.MimeTypes");
Ti.API.info("@@@ ExoPlayer MimeTypes.VIDEO_MP4: " + MimeTypes2.VIDEO_MP4); // = "video/mp4"

const MimeTypes3 = require("com.google.android.exoplayer2.util.*").MimeTypes;
Ti.API.info("@@@ ExoPlayer MimeTypes.VIDEO_MPEG: " + MimeTypes3.VIDEO_MPEG); // = "video/mpeg"

[infosia]

@jquick-axway Could you merge latest master into your branch so ci can build the PR? Builds were failing because of #330 .

[sgtcoolguy]

So I have not dug super deep into this, but I wanted to confirm what I was seeing here.

Before this PR we would:

• generate JS files for every Java type/package you could reach from any JAR/etc in the metabase.
• Copy them into the app's build directory
• During the parse/transpile of the app JS files we'd look for Java require/imports and rewrite them from the Java type/package to "hyperloop/generated.type.js"

And this PR instead does:

• Same generation, but through some gradle magic also lets hyperloop be aware of all the JARs/AARs/libraries in the SDK and app
• Writes the JS stubs, but doesn't copy them anywhere specific - instead makes the android build aware to include that directory
• Uses the global.binding.redirect('name', 'fun/path/to/file') method (we use in the core common JS SDK code) in a bootstrap file written into the app to achieve the require/import of Java types/packages without having to rewrite the original sources?

If so, that is awesome! It avoids unnecessary I/O modifications so build times should be better; it makes a nice use of something I had sort of hacked into the SDK internals; it still retains most of the same workflow/idea. I like it.

[sgtcoolguy]

I assume we'll need to tweak the Jenkinsfile as well to change how the module is built?

[jquick-axway]

@sgtcoolguy, your before/after analysis is spot on.

The key detail is I'm using Google's getCompileClasspath() method in gradle to fetch all dependency JAR paths that will eventually be passed to the javac build tool. I can do this ahead of time; no build required. It also provides paths to JARs extracted from AARs, which Google automatically extracts/caches under the ~/.gradle/ directory for us.

With this PR, hyperloop has access to dependencies:

• Titanium library and its dependencies.
• JAR/AAR libraries under app project's ./platform/android folder.
• Dependencies defined in optional ./platform/android/build.gradle file.

Breaking-Change:
Hyperloop used to access a module's JAR dependencies too (but not the module JAR itself). The main purpose of this was to avoid adding library version conflicts with putting Google libraries under ./platform/android... and instead access these libraries via a common module such as ti.playservices. I don't think we need this anymore. Instead, you can now set up a ./platform/android/build.gradle file providing exactly the dependencies you need, including newer versions of libraries that Titanium might not be referencing. I know app developers want build.gradle support, so, I think this is an overall positive change.

Benchmarks:
Here is a before/after compared to Titanium 8.3.0's hyperloop using the hyperloop-examples project.

Non-encrypted emulator builds:
- Clean Build: 47s -> 59s  (uh-oh)
- Incremental: 21s -> 13s

Encrypted device/production builds:
- Clean Build: 98s -> 78s
- Incremental: 41s -> 70s  (uh-oh)

JS proxy files generated: 1406

Performance-wise we've gone 1 step forward and 1 step back. Me replacing the require/import code substitution with the bootstrap/binding approach is definitely faster. However, the part that's killing the build times is me feeding the hyperloop generated JS files to the _build.js script's copyResources() function (this is new). This "should" be a good thing since that means the hyperloop JS files are now going through our ProcessJsTask and are encrypted (when applicable) like all of our other JS files. But the problem is that hyperloop typically generates a crazy amount of JS files (in this case 1406 files). The old hyperloop version used to copy the generated JS files "raw" to the APK /assets/Resources/hyperloop folder without any processing.

You could argue that the real issue is that hyperloop generates way more JS files than needed, which is true. Hyperloop generates a JS file for every Java class required-in, the Java classes referenced by those Java classes, and continue on-and-on from there.

Perhaps an alternative solution would be to not generate the JS wrappers at build time and instead set up the proxy objects dynamically via reflection. We could lazy load the Java types as needed. It would add some runtime overhead, but reduce the APK size and build times.

---

I think the build times need to be better.

I need to ponder this some more.
Worst case, I can continue to copy the hyperloop JS files raw like before.

[sgtcoolguy]

So just to spitball here:

• Do we have a breakdown for timings of the clean builds? i.e. Is this the clean, "very first" hyperloop build where it's doing the metabase generation plus the proxy generation? Is this only counting the JS proxy file generation? Do we properly avoid regenerating these on rebuilds (incremental or even a "second-time" clean build)? The goal here is that we should be caching metabase and JS proxies for libraries in a location outside the project if we can so that for example the android API metabase/JS proxies can be re-used across projects and the hit is truly only on the very first build. (or maybe we only do this for known "global" libraries like the core android APIs/Titanium libs and not project-specific AARs/JARs?)
• If we were to exempt the hyperloop-generated proxy JS files from being transpiled, would that "fix" the issue? I think the change in overhead here is that we're passing them through the full build pipeline now whereas before we just copied them? The proxies are intended to be a one-time generated static asset effectively; so that we could just copy them as needed (or maybe even do some symlink magic?)

[jquick-axway]

@sgtcoolguy, I benchmarked "clean" builds by deleting the app project's build directory. This means everything gets regenerated, including the metabase JSON. While the metabase JSON is stored to a separate temp directory, the incremental info is stored under our build/hyperloop folder.

Doing a clean app build the normal way (ex: change deploy type), the hyperloop side will still build incrementally since the build/hyperloop folder is preserved. So, we're okay here.

Hyperloop will only update the metabase JSON if libraries have been added/removed.

Hyperloop only generates/removes proxy JS files as needed incrementally thanks to @janvennemann past changes. So, we're fine here.

The build performance overhead my PR adds is that the generated JS proxy files are now being "processed" via our _build.js script's copyResources() code. We were not doing this before. We used to copy the generated JS files raw. As a quick solution, I think I'll change my PR to copy them raw like before. I just haven't decided exactly how to do this yet. I'll either have the JS files flagged not to be processed (like what we do with JS files referenced via HTML) or add a new build event for module hooks/plugins to add additional assets to the APK.

[jquick-axway]

PR updated:

• Significantly improved build times by not "processing" JS proxy files.
  • Added JS files to Android "builder" object's htmlJsFiles collection whose files are never transpiled, source-mapped, or encrypted. Slight cheat since they're not HTML related, but does the job.
• Added x86_64 architecture support.
• Migrated JS tasks to async/await.

---

New benchmarks between Titanium 8.3.0 and 9.0.0 with hyperloop-examples...

Non-encrypted emulator builds:
- Clean Build: 47s -> 30s  (1.6x faster)
- Incremental: 21s -> 12s  (1.8x faster)

Encrypted device/production builds:
- Clean Build: 98s -> 31s  (3.2x faster)
- Incremental: 41s -> 19s  (2.2x faster)

JS proxy files generated: 1406

[jquick-axway]

Updated PR:

• Removed hyperloop module check in hook. (Not needed.)
• Fixed a few bugs when transition tasks to async/await.
• Updated hook unit tests. (They were failing due to this PR's changes.)

[garymathews]

LGTM! 👍

[lokeshchdhry]

FR Passed.

1. https://jira.appcelerator.org/browse/TIMOB-27685 -- Checked module 5.0.0 with hyperloop examples app. First as well as incremental build & works as expected.
2. https://jira.appcelerator.org/browse/TIMOB-27297 -- ### Is app in foreground: true get logged every 1 second.
3. https://jira.appcelerator.org/browse/TIMOB-27298 -- EXTERNAL_CONTENT_URI gets logged.

SDK : 9.0.0 local build
Module Ver : 5.0.0