Android tool causing Desugar action to fail when building Android tools from source #11618

nkoroste · 2020-06-19T16:20:30Z

Description of the problem / feature request:

When building Android Tools from the tip of Bazel 3.3.0 branch an error is thrown in Desugar worker log saying: Error parsing command line: Unrecognized option: --persistent_worker

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Build Android tools from source and try to build Android binary target using them

Make sure you're on latest 3.3.0 release branch
bazel build //src:bazel-dev //tools/android/runtime_deps:android_tools
mkdir -p /tmp/bazel_android_tools
tar -xzvf bazel-bin/tools/android/runtime_deps/android_tools.tar.gz -C /tmp/bazel_android_tools
link the local build of Bazel built in step Fix up README #2
build your app and supply a flag to use the locally built android tools ./bazelw build --override_repository=android_tools=/tmp/bazel_android_tools //apps/myApp:app

Also tried doing the same steps but with at SHA f3d1683 from here:
f3d1683#diff-6f653881793ed1c14a8c9d92a49705b1
Also with no luck, I get the same error.

What operating system are you running Bazel on?

Linux/Mac

What's the output of `bazel info release`?

3.3.0

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

ERROR: /Users/sachin/PersonalProjects/samples-bazel-android/app/src/main/java/com/xachin/playground/BUILD.bazel:3:19: Desugaring app/src/main/java/com/xachin/playground/playground-app_kt.jar for Android failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at /private/var/tmp/_bazel_sachin/6af8cdc15b57945be280298fb5d977c4/bazel-workers/worker-2-Desugar.log ---8<---8<---
Error parsing command line: Unrecognized option: --persistent_worker
Try --help.
---8<---8<--- End of log ---8<---8<---
Target //app:app failed to build

The text was updated successfully, but these errors were encountered:

nkoroste · 2020-06-19T16:21:09Z

@jin fyi

jin · 2020-06-19T17:52:21Z

Thanks for the repro @nkoroste - mailed #11620.

nkoroste · 2020-06-19T22:03:24Z

Getting a different issue after this fix, see:

ERROR: /home/nk/Dev/android/foo/platform/BUILD.bazel:195:19: Desugaring platform/core.jar for Android failed (Exit 1): desugar_java8 failed: error executing command 
  (cd /home/nk/.cache/bazel/_bazel_nk/2007fbffc9ead87bb25ae587ecsdcc95/execroot/__main__ && \
  exec env - \
    PATH=/bin:/usr/bin:/usr/local/bin \
  bazel-out/host/bin/external/bazel_tools/tools/android/desugar_java8 @bazel-out/android-armeabi-v7a-fastbuild/bin/platform/core/_dx/core/core.jar_desugared.jar-0.params)
Execution platform: @local_config_platform//:host
java.lang.IllegalArgumentException: resource com/google/devtools/build/android/desugar/runtime/PrimitiveHashcode.class not found.
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:216)
	at com.google.common.io.Resources.getResource(Resources.java:195)
	at com.google.devtools.build.android.desugar.io.ResourceBasedClassFiles.lambda$getContent$0(ResourceBasedClassFiles.java:32)
	at com.google.devtools.build.android.desugar.io.FileContentProvider.get(FileContentProvider.java:102)
	at com.google.devtools.build.android.desugar.io.FileContentProvider.sink(FileContentProvider.java:143)
	at com.google.devtools.build.android.desugar.Desugar.lambda$copyRuntimeClasses$7(Desugar.java:449)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at com.google.devtools.build.android.desugar.Desugar.copyRuntimeClasses(Desugar.java:448)
	at com.google.devtools.build.android.desugar.Desugar.desugarOneInput(Desugar.java:366)
	at com.google.devtools.build.android.desugar.Desugar.desugar(Desugar.java:240)
	at com.google.devtools.build.android.desugar.Desugar.processRequest(Desugar.java:1008)
	at com.google.devtools.build.android.desugar.Desugar.runPersistentWorker(Desugar.java:968)
	at com.google.devtools.build.android.desugar.Desugar.main(Desugar.java:947)
Target //apps/myApp/foo:app failed to build
INFO: Elapsed time: 39.935s (0.67m), Critical Path: 38.49s (0.64m)
INFO: 230 processes: 3 linux-sandbox, 227 worker

Kernald · 2020-06-21T23:30:04Z

I have the same issue when desugaring okhttp with Bazel 3.3 - no problem with Bazel 3.2.

jin · 2020-06-22T06:53:59Z

@Kernald could you try if #11620 works for you and if you're running into the same new problem @nkoroste ran into described in that PR?

Kernald · 2020-06-22T06:57:28Z

I'm gonna try that - I never built Bazel myself so that might take a while.

jin · 2020-06-22T06:59:58Z

Thanks, you can follow the additional instructions in #11618 (comment) of this issue to build the desugar tool because it's unbundled from Bazel.

Kernald · 2020-06-22T07:15:45Z

I tried both with and without the repo override, and in both cases I got the same error that I have with Bazel 3.3.0 - the one just above.

jin · 2020-06-22T07:17:34Z

Gotcha, so there are really two bugs here.

FYI @dx404

jin · 2020-06-22T07:46:41Z

I see the issue. None of our tests cover the primitive type desugaring feature required by that missing desugar resource. Will submit a fix.

nkoroste · 2020-06-23T17:00:01Z

Tested out the latest patch but I now get yet another new error:

ERROR: /home/nk/foo/apk/BUILD.bazel:992:15: Sharding classes for dexing for //foo/apk:Foo failed (Exit 1) mapper failed: error executing command bazel-out/host/bin/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/ziputils/mapper @bazel-out/k8-fastbuild/bin/foo/apk/_dx/Foo/classes.jar-0.params
Use --sandbox_debug to see verbose messages from the sandbox
Error: Could not find or load main class com.google.devtools.build.android.ziputils.DexMapper
Caused by: java.lang.ClassNotFoundException: com.google.devtools.build.android.ziputils.DexMapper
Target //foo/apk:Foo failed to build

I'm not really sure who triggers DexMapper or how your change might of broke it. When compiling the DexMapper target directly with "_deploy.jar" I do see the class in the expected location.

Fixes #11618 Needs to be cherry-picked into 3.3.0. Closes #11620. PiperOrigin-RevId: 318428300

nkoroste · 2020-06-26T16:01:57Z

@jin I see that this is now closed, does your fix account for the latest dexMapper issue I reported above?

jin · 2020-06-26T16:03:42Z

No, it doesn't. Can you file another bug for that with repro steps? The linked commit fixes the worker strategy desugar issue.

nkoroste · 2020-06-26T16:13:33Z

Were you able to build an Android App after your fix? The repro steps was to use your fix and attempt to build an app which failed with this DexMapper issue for me

jin · 2020-06-26T16:15:00Z

Yes, I was able to build the Android apps in rules_jvm_external, and the integration tests on CI are passing. I haven't seen that error while trying to fix the Desugar issue.

katre · 2020-06-26T16:46:22Z

Bazel 3.3.1 RC1 is now available: https://releases.bazel.build/3.3.1/rc1/index.html

Please test this against the errors in this issue.

nkoroste · 2020-06-26T17:49:44Z

@katre happy to test it out but I'm not really sure how to do it with the link you provided. I need to be able to rebase off of Bazel 3.3.1 RC1 branch/tag as we use a custom fork of Bazel.

katre · 2020-06-26T17:56:00Z

Every release candidate is a branch on Github, see here: https://github.com/bazelbuild/bazel/tree/release-3.3.1rc1

nkoroste · 2020-06-26T17:58:27Z

Great I can work with that, thanks!
I just checked https://github.com/bazelbuild/bazel/releases and didn't see it there so I was a bit confused.

katre · 2020-06-26T18:02:28Z

Right, that's only for finalized releases, this is a candidate (which is why it's a branch, not a tag). It'll end up there once it's fully released.

nkoroste · 2020-07-11T02:04:41Z

Still getting the DexMapper issue on 3.3.1 :(

katre · 2020-07-13T13:00:00Z

@jin can you take a look at DexMapper? At this point I think we should aim for a fix in #11687 (3.4.0), rather than another patch to 3.3.

jin · 2020-07-13T13:00:11Z

The DexMapper issue is not related to this issue. This issue has been fixed in 3.3.1.

jin · 2020-07-13T13:01:37Z

@nkoroste if you have a way to repro the DexMapper issue, please create a new issue for us to track it.

nkoroste · 2020-07-13T19:32:11Z

@jin I finally figured out what the issue was - It was related to sandboxing and the --override_repository=android_tools= flag. I was following your instructions from a different PR on how to build using locally built tools:

bazel build //src:bazel-dev //tools/android/runtime_deps:android_tools
mkdir -p /tmp/bazel_android_tools
tar -xzvf bazel-bin/tools/android/runtime_deps/android_tools.tar.gz -C /tmp/bazel_android_tools
./bazelw build --override_repository=android_tools=/tmp/bazel_android_tools //app

which resulted in this:

Error: Could not find or load main class com.google.devtools.build.android.ziputils.DexMapper

When I manually ran the dexmapper subcommand it passed just fine

I can file a seperate bug for this, unless this a user error and there is a more official way to use locally built android_tools?

jin · 2020-07-20T07:23:55Z

Yes, please file a separate bug as it looks like it's from a different failure mode.

nkoroste · 2020-07-20T17:20:32Z

done - #11810

nkoroste mentioned this issue Jun 19, 2020

Desugaring for Android failed: Worker process did not return a WorkResponse #11530

Closed

jin mentioned this issue Jun 19, 2020

Fix Desugar: incorrect assumption of persistent worker conditional and missing runtime desugar lib resource for primitive desugaring #11620

Closed

aiuto added team-Android Issues for Android team untriaged labels Jun 22, 2020

ahumesky mentioned this issue Jun 26, 2020

Add Android data binding support #2694

Open

bazel-io closed this as completed in 3b0439e Jun 26, 2020

jin mentioned this issue Jun 26, 2020

Release 3.3 - June 2020 #11358

Closed

katre pushed a commit that referenced this issue Jun 26, 2020

Fix incorrect assumption of desugar persistent worker conditional

513a5e8

Fixes #11618 Needs to be cherry-picked into 3.3.0. Closes #11620. PiperOrigin-RevId: 318428300

katre reopened this Jul 13, 2020

jin closed this as completed Jul 13, 2020

katre added the release blocker label Jul 13, 2020

nkoroste mentioned this issue Jul 20, 2020

Building using local android_tools causes a DexMapper error #11810

Closed

ahumesky removed the untriaged label Mar 5, 2021

This was referenced May 12, 2021

Desugar opentelemetry-sdk-common for android failed open-telemetry/opentelemetry-java#3231

Closed

Desugar opentelemetry-sdk-common for android failed #13472

Closed

sgowroji mentioned this issue Oct 10, 2022

Wrong build android in windows bazelbuild/intellij#3267

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Android tool causing Desugar action to fail when building Android tools from source #11618

Android tool causing Desugar action to fail when building Android tools from source #11618

nkoroste commented Jun 19, 2020

nkoroste commented Jun 19, 2020

jin commented Jun 19, 2020

nkoroste commented Jun 19, 2020

Kernald commented Jun 21, 2020

jin commented Jun 22, 2020

Kernald commented Jun 22, 2020

jin commented Jun 22, 2020

Kernald commented Jun 22, 2020

jin commented Jun 22, 2020

jin commented Jun 22, 2020

nkoroste commented Jun 23, 2020

nkoroste commented Jun 26, 2020

jin commented Jun 26, 2020 •

edited

nkoroste commented Jun 26, 2020 •

edited

jin commented Jun 26, 2020

katre commented Jun 26, 2020

nkoroste commented Jun 26, 2020

katre commented Jun 26, 2020

nkoroste commented Jun 26, 2020

katre commented Jun 26, 2020

nkoroste commented Jul 11, 2020

katre commented Jul 13, 2020

jin commented Jul 13, 2020

jin commented Jul 13, 2020

nkoroste commented Jul 13, 2020

jin commented Jul 20, 2020

nkoroste commented Jul 20, 2020

Android tool causing Desugar action to fail when building Android tools from source #11618

Android tool causing Desugar action to fail when building Android tools from source #11618

Comments

nkoroste commented Jun 19, 2020

Description of the problem / feature request:

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

What operating system are you running Bazel on?

What's the output of bazel info release?

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

nkoroste commented Jun 19, 2020

jin commented Jun 19, 2020

nkoroste commented Jun 19, 2020

Kernald commented Jun 21, 2020

jin commented Jun 22, 2020

Kernald commented Jun 22, 2020

jin commented Jun 22, 2020

Kernald commented Jun 22, 2020

jin commented Jun 22, 2020

jin commented Jun 22, 2020

nkoroste commented Jun 23, 2020

nkoroste commented Jun 26, 2020

jin commented Jun 26, 2020 • edited

nkoroste commented Jun 26, 2020 • edited

jin commented Jun 26, 2020

katre commented Jun 26, 2020

nkoroste commented Jun 26, 2020

katre commented Jun 26, 2020

nkoroste commented Jun 26, 2020

katre commented Jun 26, 2020

nkoroste commented Jul 11, 2020

katre commented Jul 13, 2020

jin commented Jul 13, 2020

jin commented Jul 13, 2020

nkoroste commented Jul 13, 2020

jin commented Jul 20, 2020

nkoroste commented Jul 20, 2020

What's the output of `bazel info release`?

jin commented Jun 26, 2020 •

edited

nkoroste commented Jun 26, 2020 •

edited