33.1.1.444 Android: adt package d8 desugaring errors, exoplayer

[RikAzerion]

Using:

• AIR SDK 33.1.1.444
• Win 10 x64
• Java 1.8.0_191 x64 (tried both JRE/JDK)
• ANE containing com.google.android.exoplayer 2.11.1, specifically exoplayer-core, exoplayer-ui, extension-ima.
• adt package for armv8 run from batch script via FlashDevelop.

Problem Description

When running adt package to package an app, if an ANE is included with com.google.android.exoplayer 2.11.1, it fails to package with the following error:

dx tool failed: Error: Default method desugaring of com.google.android.exoplayer2.upstream.BaseDataSource failed because its super class java.lang.Object is missing
Compilation failed

Many other libraries appear to be throwing similar warrnings, but not errors:

Warning in C:\Users\AirSDK\AppData\Local\Temp\099e8df4-65e2-43e9-b809-85a6a01623dd\admob.jar:com/vungle/warren/analytics/VungleAnalytics$1.class:
Type android.util.Log was not found, it is required for default or static interface methods desugaring of void com.vungle.warren.analytics.VungleAnalytics$1.onResponse(com.vungle.warren.network.Call, com.vungle.warren.network.Response)

I've seen issues related to D8 and desugaring, but they appear to have been resolved, have not been able to find any information other than that. Am I missing something, is this a known problem? Thanks!

[j3k0]

I got the same when trying to include sentry's native android sdk.

In build.gradle dependencies:

implementation 'io.sentry:sentry-android:4.3.0'

When building the app:

Error: Default method desugaring of `io.sentry.config.AbstractPropertiesProvider` failed because its super class `java.lang.Object` is missing

Their support asked those questions I'm not sure how to reply (I think response is in AIR's internals):

AGP version, Gradle version, if they use Proguard, Dexguard or R8, I'd like to know their setup. Did they enable Java 1.8 compatibility on the grade file?

[ajwfrost]

Hi

Can you please edit your SDKPath\lib\adt.cfg file to add a line DebugOut=true and then re-try, it should then generate a file in your c:\users\username folder called adt.log which will contain the actual command used for DX/D8.

FYI we don't currently use gradle for building although given all the forthcoming changes needed around app bundles and asset packs, we are moving to this which should make things a little simpler (and as an interim step it generates the file/folder structure that gradle can work with, which would then make debugging these sorts of issues a lot easier..)

thanks

Andrew

[RikAzerion]

Thanks for the response, I've created a gist for adt.log here: https://gist.github.com/RikAzerion/7aebf73a104073d2583f85e506cd6548

Many other libraries are present, though the error only occurs when exoplayer is included. Though I need those libraries for my app, I could create an ANE with only exoplayer if that would help(?). Thanks!

[RikAzerion]

I see in the log:

AAPT command line: ... --min-sdk-version 21 --target-sdk-version 29

While in my app.xml android manifest I have:

<uses-sdk
android:minSdkVersion="17"
android:targetSdkVersion="30" />

That seems odd.

For each "Generating class file" line, I see: -source 1.5 -target 1.5. Not sure what version that's referring to or if that's incorrect.

[j3k0]

Seeing your logs remind me of one important point. For me it only occurs when building for armv8.

[ajwfrost]

@RikAzerion we would bump up the minimum SDK version if you're targeting 64-bit platforms since they're not available in API level 17. I think the target SDK version may be being ignored :-(

Could you share the full AAPT and DX/D8 command outputs please?

[RikAzerion]

@ajwfrost Thanks for the help. The command output from running adt package appears to be identical to the output in the adt.log (command output here: https://gist.github.com/RikAzerion/15341a1d5fc3b6b6e5afe81080e44f27). I can't find any other log files for AAPT/DX/D8 output, am I misunderstanding?

[ajwfrost]

@RikAzerion if you've got the DebugOut=true set in the config file, then you should be getting an adt.log file generated in your user folder, which also gives the command-lines of these tools rather than just the output. E.g. for mine:

*** ADT Logging at 2021/04/10 07:28:02 ***
Called by: I:\SDKs\33.1.1.406\lib\adt.jar
ADT called with: -package -target apk-captive-runtime -arch armv7 -storetype pkcs12 -keystore I:\Adobe\QualityAssurance\TestMedia\utils\harman.p12 -storepass *** AIRInfo.apk application.xml AIRInfo.swf -C I:\Adobe\QualityAssurance\TestMedia\utils\assets .
Setting architecture to: armv7
AAPT command line: I:\SDKs\33.1.1.406\lib\android\bin\aapt.exe package -z -u -J C:\Users\ukinanfr\AppData\Local\Temp\67ed5049-e330-411b-b6bb-667e0074cd63 -M C:\Users\ukinanfr\AppData\Local\Temp\67ed5049-e330-411b-b6bb-667e0074cd63\AndroidManifest.xml -S C:\Users\ukinanfr\AppData\Local\Temp\67ed5049-e330-411b-b6bb-667e0074cd63\app_entry_res -S C:\Users\ukinanfr\AppData\Local\Temp\67ed5049-e330-411b-b6bb-667e0074cd63\captive_runtime_res --auto-add-overlay -I I:\SDKs\33.1.1.406\lib\android\lib\resources\android-res.jar --no-version-vectors --min-sdk-version 14 --target-sdk-version 29 --version-code 1000 --version-name 0.1 -F C:\Users\ukinanfr\AppData\Local\Temp\67ed5049-e330-411b-b6bb-667e0074cd63\resources.apk
Generating class file:  C:\Program Files\Java\jdk-14.0.2\bin\javac.exe -encoding ascii -J-Xmx1024m -source 1.5 -target 1.5 -Xlint C:\Users\ukinanfr\AppData\Local\Temp\67ed5049-e330-411b-b6bb-667e0074cd63\R.java -d C:\Users\ukinanfr\AppData\Local\Temp\d969be52-f542-482d-ad8e-ff37a115783a
DX tool command line: java -Xmx1024m -jar I:\SDKs\33.1.1.406\lib\android\bin\d8.jar --release --main-dex-list I:\SDKs\33.1.1.406\lib\android\lib\multidex-1.0.2\addToPrimaryDex.keep --min-api 8 --output C:\Users\ukinanfr\AppData\Local\Temp\be66414b-9252-4167-bc9e-798f0d2862be I:\SDKs\33.1.1.406\lib\android\lib\multidex-1.0.2\classes.jar C:\Users\ukinanfr\AppData\Local\Temp\be66414b-9252-4167-bc9e-798f0d2862be/inputJAR.jar C:\Users\ukinanfr\AppData\Local\Temp\67ed5049-e330-411b-b6bb-667e0074cd63\resources.jar I:\SDKs\33.1.1.406\lib\android\lib\runtimeClasses.jar
Baksmali tool command line: java -jar I:\SDKs\33.1.1.406\lib\android\lib\baksmali.jar d -o C:\Users\ukinanfr\AppData\Local\Temp\be66414b-9252-4167-bc9e-798f0d2862be\out C:\Users\ukinanfr\AppData\Local\Temp\be66414b-9252-4167-bc9e-798f0d2862be\classes.dex
Smali tool command line: java -jar I:\SDKs\33.1.1.406\lib\android\lib\smali.jar a -a 8 -o C:\Users\ukinanfr\AppData\Local\Temp\be66414b-9252-4167-bc9e-798f0d2862be\out\classes.dex C:\Users\ukinanfr\AppData\Local\Temp\be66414b-9252-4167-bc9e-798f0d2862be\out
Version in APK = 33.1.1.406, version of ADT = 33.1.1.406

Please ensure you strip the storepass parameter from it if you're providing that on the command-line..
thanks

[RikAzerion]

Hi @ajwfrost , these appear to be included already in the adt.log I posted? https://gist.github.com/RikAzerion/7aebf73a104073d2583f85e506cd6548. There is no other output in adt.log beyond that.

[ajwfrost]

Sorry @RikAzerion, can't believe I didn't spot that!

So the major thing that jumps out at me is that the D8 command line is very long - over 12000 characters - and I am wondering whether this is just getting truncated to 8191 characters by Windows..

All of the dependent jar files are extracted into a temporary folder from where we then compile them, and they're then named on the command line with the full path. Normally I would suggest working around this by changing where your TEMP variable points to, to a shorter path e.g. C:\tmp, but in your case this would still result in a command-line of 9643 characters.

We did do some investigations into this a while ago, I will check with the team tomorrow and see if we can get a better workaround for this. Sadly, D8 doesn't allow wildcards so we can't just tell it to include all jar files in a certain folder..

[RikAzerion]

Hi @ajwfrost thanks for the response. That makes sense, I also have problems when building to ipa for ios from pc/windows, due to the command line character limit being reached when including too many ANE's. Previously I could work around this by including only specific ANE's and moving the project to C:/0/ to shorten the number of characters used internally by adt package, but I'm getting more and more dependencies so am no longer able to build to ipa/ios from pc/windows. So in short, solving any character limit problems when using adt with windows/cmd would be great. Big thanks!

[ajwfrost]

Hi @RikAzerion - took us a little longer than I thought but could you please see whether the attached version will help at all? Based on the .444 build, but the ANEs are just listed as files and it's running AAPT from the folder where they're all sat. I think that would take your command-line down sufficiently.
https://transfer.harman.com/message/6o095qBtCIdZjFyzEsST98

Although having said that, it may be that this is just a red herring and you still get the same error output..! but if you could share the adt.log contents like before, we can check this..

thanks

[RikAzerion]

Hi @ajwfrost thanks for the the fix. I've replaced the adt.jar in the AIRSDK with the provided adt.jar, but the AAPT command appears to be the same length still: https://gist.github.com/RikAzerion/69007c5c2c36169e942e20f5f168294c.

[ajwfrost]

Hi @RikAzerion okay thanks: yes AAPT wasn't changed, that was my mistake in the description! That one was/is 4183 characters long. The one that changed was the D8 compilation, originally over 12000 characters and now reduced to 5226. So we should be good in terms of any Windows command-line restrictions now.

Which means it's failing for some other reason....

Having looked into this a little further: the warnings are ones that we see when doing any successful build, but these are normally swallowed as long as the tool completed successfully. So e.g.

Warning in com.google.android.gms-play-services-ads.jar:com/google/android/gms/internal/ads/zzbyx.class:
  Type `java.lang.Object` was not found, it is required for default or static interface methods desugaring of `void com.google.android.gms.internal.ads.zzbyx.finalize()`

and normally this doesn't appear to actually stop the D8 tool from compiling. But then the same missing definition causes an error at the end of the log file:

Error: Default method desugaring of `com.google.android.exoplayer2.upstream.BaseDataSource` failed because its super class `java.lang.Object` is missing
Compilation failed

The only thing I can suggest in the short term is, if you have an Android SDK installed, you try passing that in to ADT as a platformsdk option. But we will look into this further, it may be that we have to adjust the D8 command line to also pass in some other library or to add an option..

thanks

[ajwfrost]

... also just noticing the --min-api 17 - could you perhaps try updating this to a later version, via the uses-sdk field in the AIR application descriptor file? You're using armv8 so this should be a minimum of 21?

[ajwfrost]

Interesting article on this.. https://jakewharton.com/d8-library-desugaring/

It looks like there are two options here:
a) we can disable the Java 8 language features: the D8 compilation should then work, although if any of those jars do actually require Java 8 features, I'm then not sure what the impact would be. We can try that..
b) we can include the extra library dependency that is missing. This is from the Android SDK so I will have to check on whether we're able to distribute that within our own SDK, but if not we can change things such that we pick it up automatically if you use the ADT platformsdk option.

Will try to get back to you later with some updates to try out here...

thanks

[itlancer]

@ajwfrost, please do not disable Java 8 features. It will break lambda support and something like that #84

[ajwfrost]

@itlancer agreed, I should have mentioned that this would only be done if there was no need to have the Java 8 features! Having got a bit more info on this, though, it looks like there are two possibilities (according to the D8 documentation):

1. if you don't need Java 8 features or later language updates, we disable the sugaring..
2. if you do need Java 8 features or later updates, we will need to link against the Android.jar file from the Android SDK: so for this, you would need to have the Android SDK installed.

There actually seems little point in not going with option 2 here.. particularly because our changes around Android App Bundle and play asset delivery mean that we will also require folk to have an Android SDK installed.

[ajwfrost]

Hi @RikAzerion - am hoping that the below build will fix the problem you're seeing. This build is adjusted a little so that if you pass in the path to the Android SDK, then it will automatically add the android.jar reference onto the D8 command line. Which should then fix the problem you're seeing.
https://transfer.harman.com/message/YVImqAdq7RmsChInBQmdIg

So e.g. call with -platformsdk c:\users\username\appdata\local\android\sdk added to the end of your ADT command line, and this should get picked up. In the adt.log file, you'll see the output where it's now adding --lib and the android.jar file.

Let us know how it goes..

thanks

[RikAzerion]

Hi @ajwfrost , thanks that did something! Only one error left:

ADT exception: dx tool failed:no main manifest attribute, in C:\Users\AirSDK\AppData\Local\Android\Sdk\build-tools\30.0.3\lib\d8.jar

• Have also tried 31.0.0-rc2.

adt.log: https://gist.github.com/RikAzerion/774996f4570586d80267e3646ab0182b

Googling atm, seems like something trivial, fingers crossed.

[ajwfrost]

@RikAzerion I think that’s a later version of the SDK than we’d been using so you may need to copy the d8.jar file from the AIR SDK over the top of the one in your Android SDK (temporarily at least - do back up the Android file first!). Will look at a proper fix for this later..

thanks

[RikAzerion]

Thanks @ajwfrost , adding the d8.jar from AIRSDK 33.1.1.444 to Android\Sdk\build-tools\31.0.0-rc2\lib unfortunately results in similar build errors as before: https://gist.github.com/RikAzerion/1bae6fa25199362f506d7aa90b0206c7.

Should I try with a specific build-tools version?

[ajwfrost]

That log looks like it didn’t have our patched adt.jar used? The extensions are all back with full path names, and the ‘—lib’ argument is missing..

[RikAzerion]

I've re-downloaded https://transfer.harman.com/message/YVImqAdq7RmsChInBQmdIg to be sure but am getting the same results.

After adding -platformsdk, when running adt package, cmd reports:

NOTE: Multiple SDKs exist, chosing version 31.0.0 rc2 for packaging

In adt.log I also see C:\Users\AirSDK\AppData\Local\Android\Sdk\build-tools\31.0.0-rc2\lib\d8.jar used. So -platformsdk does appear to be picked up. I've replaced the d8.jar for platform-tools 31.0.0 rc2 with the AIR SDK's. Am I doing something wrong, or perhaps the wrong adt.jar was uploaded by mistake? Thanks!

Latest adt.log, to be sure: https://gist.github.com/RikAzerion/97502ae7bf52106a483233dda8ea0e1d

[ajwfrost]

Okay sorry, yes I had uploaded the wrong ADT jar file... will need to rebuild this one again..

[ajwfrost]

Please try with the below one...
https://transfer.harman.com/message/LvCMtCSkcyPKaiKuV1emA5
thanks

[RikAzerion]

Hey @ajwfrost unfortunatey still getting the exact same results, platformsdk is picked up but "—lib" parameter is absent and full paths are still used.

[ajwfrost]

@RikAzerion are you able to try this again with the latest SDK release please? You'd need to set up the adt.cfg again to get the debug output, if it's not worked please re-open the issue and post the output here...
thanks