android build: Update to AndroidX.

[gnprice]

AndroidX, previously called the Android Support Library, is a collection of libraries developed by Android upstream which are bundled into an app at build time, rather than taken from the system on the device. Some are quite central to a typical Android app.

Upstream has renamed the whole thing to AndroidX, and this goes with a renaming of tons of classes. (Mostly to get them out of android.*, which was always very confusing because that's the same namespace used by the actual system packages.) Docs: https://developer.android.com/jetpack/androidx/migrate

This could have been sticky, because lots of apps and lots of third-party libraries refer to these core libraries, and in a distributed ecosystem it's impossible for everything to update at once. And because these are often so central, a given build of an app must use the same version of them throughout -- including from any third-party libraries it uses. So for example if some third-party library has updated to use AndroidX, then an app cannot switch to that version of that library until it too has updated to use AndroidX.

Fortunately, the planners of this migration recognized this could be a problem, and found a way to make the constraints run only in that one direction. When an app updates to use AndroidX, it can do so even while it's still using third-party libraries that are still built to use the Android Support Library. The key is a feature called "Jetifier": we set the flag android.enableJetifier=true, and then (quoting the doc linked above):

The Android plugin automatically migrates existing third-party libraries to use AndroidX by rewriting their binaries.

So the migration path is, apps go first; then libraries update at their leisure.

---

For us, the most important third-party library (third-party from an Android perspective) is React Native. RN took the AndroidX update in v0.60, so we have to do the same before we can complete that RN upgrade, i.e. #3548.

The Android upstream way to update a given project (as described by the doc linked above) has two prongs:

• (a) The project's own source code can be updated automatically (barring fancy use of classloaders, etc.) by Android Studio, with the feature "Refactor > Migrate to AndroidX...".

• (b) Third-party libraries get updated when they're linked in, by rewriting them as part of this project's build.

Both of those appear in this commit. But in an RN context, there's one other important set of code: third-party RN libraries get pulled in as part of our Gradle build, with build.gradle lines using project, like so:

implementation project(':react-native-vector-icons')

This means they're not covered by (b). This contrasts with RN itself, which is pulled in as a binary artifact:

implementation "com.facebook.react:react-native:+"

So for those third-party RN libraries, we need to edit the source code, like (a), before it gets built. Happily, someone has written a tool to do just that, and we use it:

• (a') Third-party RN libraries get rewritten in place in node_modules/, after yarn installs them and before the Gradle build. This uses a different "jetifier" tool: https://github.com/mikehardy/jetifier

Specifically, the latter "jetifier" will crawl node_modules; find all .java, .kt, and .xml files; and apply a list of string substitutions (about 1900 of them) to rewrite old names to new ones. That sure is kind of heuristic... but it's good enough that it will in fact find any normal import of the affected classes, and on the other hand the patterns being substituted are all long boring fully-qualified Java class names, unlikely to be found in spurious places. For us, I also inspected (below) the exact edits the tool makes.

---

In this commit, we take the upgrade.

• (a) I used "Refactor > Migrate to AndroidX..." in Android Studio. This updated a few spots in our own source code.

• (b) The same auto-refactor added the gradle.properties settings that handle libraries we incorporate as binary artifacts, including RN itself.

  It also updated dependencies in our app/build.gradle. This made the supportLibVersion variable unused, and I deleted it.

• (a') We add jetifier as a dev dependency, and invoke it as a postinstall hook. This means any run of yarn install (or bare yarn) will end by running it.

I inspected manually (by making copies of node_modules, and comparing with git diff --no-index) the effect of jetifier. All the edits look appropriate. Excluding a few that have no effect, the edits appear in the following packages:

• react-native
• react-native-image-picker
• react-native-webview
• react-native-photo-view
• @react-native-community/netinfo

So we can remove jetifier once we've upgraded all of those to versions that use AndroidX in the first place.

I tested the app in both debug and release builds, and exercised each of the dependencies mentioned above: picked and uploaded an image; opened the lightbox; entered and left airplane mode to see the "No Internet connection" banner. Everything I tested works.

[rk-for-zulip]

So we can remove jetifier once we've upgraded all of those to versions that use AndroidX in the first place.

Well... maybe not!

Per the Jetifier docs, Jetifier is automatically installed alongside and run by React Native 0.60, so we should get this for free as part of the React Native 0.60 upgrade. As such, I'm going to hold off on applying this commit until that's been confirmed.

Even if the diff turns out to be unnecessary, though, the identification and testing of the affected modules was definitely needed, and the analysis in the commit message is also helpful. Thanks!

[gnprice]

Per the Jetifier docs, Jetifier is automatically installed alongside and run by React Native 0.60

True. I guess that means we can't easily completely remove jetifier even then. That's a little unfortunate, because it's a hack, but ah well. On the bright side, once we've made those upgrades, it should become the case that it's a no-op -- that it doesn't actually end up editing any files at all.

That description in the docs isn't very precise. What it turns out to mean, as I learned when studying this while preparing this change, is that react-native run-android, when running on RN v0.60, will run jetify first.

Even if the diff turns out to be unnecessary,

The changes in android/ are required in order to make the upgrade to AndroidX, and therefore required in order to upgrade to RN v0.60.

The bits that will get partly substituted for by the automatic call to jetify found in RN v0.60 are the two lines in package.json. That doesn't affect whether we should make this change -- we should in any case make this change, to unblock the RN upgrade, and then after the upgrade we can take those lines out if we like.

I think in fact we'll want to keep those lines there until we've upgraded all of the libraries mentioned above. That's because RN's automatic call to jetify doesn't happen unless you run react-native run-android -- so in particular it doesn't happen if you just run tests, or just make a build (e.g. a release build), without having used react-native run-android since the last time you ran yarn and it happened to update a relevant dependency. (A better place for this step might have been in the Gradle build scripts, but that's not the design they chose.)

[gnprice]

Per the Jetifier docs, Jetifier is

BTW there's a name clash here which is pretty confusing:

• "Jetifier" is the name of a tool from Android upstream. It does all the things you would hope -- it runs as part of the build pipeline, rather than munging source files in place, and it has a bunch of logic to parse the relevant kinds of files, find actual references to classes, and rewrite those. Correspondingly its source weighs in at about 5000 lines of Kotlin.

• The author of the RN-world tool which rewrites source files in node_modules/ with string substitutions chose to use the exact same name for their thing. 😢 With the one difference that they don't seem to capitalize it -- it's "jetifier" throughout the README.

[gnprice]

Pushed an update which emphasizes more strongly the distinction between the two "jetifier" tools, and which takes the point about how one of them won't quite go away even after it stops mattering.

[gnprice]

(P1 because this is part of #3548.)

[gnprice]

Coming back to this as we're now otherwise getting close to completing the RN v0.60 upgrade #3548, because we've successfully migrated to CocoaPods.

Copying from #3548 (comment) my notes on the state of this AndroidX upgrade, what's needed is to rebase, re-test, and merge it, with particular attention to this point from cb87f90:

    In that same comment, a maintainer says that 0.7.0-rc.4 is the
    minimum version with AndroidX compatibility. If this is true, then
    we may have a slight problem, because 0.7.0-rc.4, just like 0.6.0,
    does not work with RN 0.59.10. That would mean we had to do the
    AndroidX upgrade (PR #3852) at the same time as the RN v0.60 upgrade
    (issue #3548). Greg thinks it's likely that the maintainer is
    mistaken that there's a minimum version of unimodules required to
    use AndroidX; the AndroidX engineers designed a smooth migration
    path such that this shouldn't happen (see Greg's comment at
    https://github.com/zulip/zulip-mobile/pull/3987#issuecomment-601918696).

Chris adds:

(As discussed around here, I think that paragraph from my commit message for cb87f90 is correct if you remove "just like 0.6.0". I'm not sure how that made it in, but it's not true; we're happily using 0.6.0 with RN v0.59.10. My apologies.)

[gnprice]

Rebased. The one nontrivial change I made to complete the rebase was that we now already have a postinstall script, so I added the jetify call there instead of as the postinstall script on its own.

In a quick smoketest it seems to work fine, but I haven't yet repeated the testing described at the end of the commit message. Next, I'll

• check up on the unimodules story;
• repeat that original testing;
• then if all's well, merge.

[gnprice]

I've exercised the unimodules machinery by going to the diagnostics screen and looking at the app version there, which we get via the expo-application / ExpoApplication unimodule. All seems well, so I think the unimodules maintainers were just confused about AndroidX in the way that I predicted earlier.

I've also redone the part where I inspected all the changes the jetify tool (the RN-world one as described above) makes in our node_modules/. There are a few more now, mostly in unimodules/Expo-land, but again they all look appropriate.

Next up is to repeat going through different flows in the app to exercise the affected dependencies, and in both debug and release builds. Then I think I'll be ready to merge this.

[gnprice]

Done and merged!

Now 🤞 on what's left of #3548.

[chrisbobbe]

I've exercised the unimodules machinery by going to the diagnostics screen and looking at the app version there, which we get via the expo-application / ExpoApplication unimodule. All seems well, so I think the unimodules maintainers were just confused about AndroidX in the way that I predicted earlier.

I've also redone the part where I inspected all the changes the jetify tool (the RN-world one as described above) makes in our node_modules/. There are a few more now, mostly in unimodules/Expo-land, but again they all look appropriate.

Next up is to repeat going through different flows in the app to exercise the affected dependencies, and in both debug and release builds. Then I think I'll be ready to merge this.

Thanks so much for verifying all this!! The thought hadn't fully formed in my mind that you'd have to do this Unimodules-related investigation yourself, when I said I'd work on #3548 following your merge of #3852. But it sure makes sense in retrospect, and I appreciate you doing it.