Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bazel CI: Android build is failing on Windows #9102

Closed
meteorcloudy opened this issue Aug 7, 2019 · 4 comments

Comments

@meteorcloudy
Copy link
Member

commented Aug 7, 2019

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1128

(02:09:04) ERROR: D:/b/zqacmrtc/external/maven/BUILD:359:1: Couldn't build file external/maven/androidx_lifecycle_lifecycle_livedata_2_0_0_files/androidx_lifecycle_lifecycle_livedata_2_0_0_resources_aapt2-src.jar: Linking static android resource library for @maven//:androidx_lifecycle_lifecycle_livedata_2_0_0 failed (Exit 1)
--
  | Aug 07, 2019 2:09:04 AM com.google.devtools.build.android.ValidateAndLinkResourcesAction main
  | SEVERE: Error while validating and linking resources
  | com.google.devtools.build.android.aapt2.ResourceLinker$LinkError: java.nio.file.NoSuchFileException: bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_arch_core_core_runtime_2_0_0_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_lifecycle_lifecycle_livedata_core_2_0_0_symbols\symbols.zip

A bisect shows the culprit is b98ad77

/cc @jin

@donaldchai

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2019

It looks like this is being treated as a single filename?

bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_arch_core_core_runtime_2_0_0_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_lifecycle_lifecycle_livedata_core_2_0_0_symbols\symbols.zip

bazel-io pushed a commit that referenced this issue Aug 7, 2019
Automated rollback of commit b98ad77.
*** Reason for rollback ***

Broke Android build on Windows
#9102

*** Original change description ***

Default Bazel to use aapt2 and fix native tests to use aapt2.

This CL sets --incompatible_use_aapt2_by_default=true and --android_aapt=aapt2. It also adds aapt2 to the mock SDK for Android analysis and integration tests.

For the tests that assume and depend on AAPT, we explicitly pin them to use AAPT.

Fixes #6907
Fixes #4103

RELNOTES: Bazel Android builds now use aapt2 by default. To revert to aapt, set `--an...

***

ROLLBACK_OF=261424350

RELNOTES: None
PiperOrigin-RevId: 262099198
@jin

This comment has been minimized.

Copy link
Member

commented Aug 7, 2019

I can reproduce this on Windows. Looking.

PS C:\Users\jingwen\Code\rules_jvm_external\examples\android_instrumentation_test> bazel build //src/test:greeter_test_app --an
droid_aapt=aapt2
INFO: Analyzed target //src/test:greeter_test_app (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
INFO: From Linking external/bazel_tools/src/main/native/windows/windows_jni.dll [for host]:
   Creating library bazel-out/host/bin/external/bazel_tools/src/main/native/windows/windows_jni.dll.if.lib and object bazel-out/host/bin/external/bazel_tools/src/main/native/windows/windows_jni.dll.if.exp
ERROR: C:/users/jingwen/_bazel_jingwen/w7ee76y7/external/maven/BUILD:31:1: Linking static android resource library for @maven//:androidx_test_espresso_espresso_core failed (Exit 1)
Aug 07, 2019 12:24:23 PM com.google.devtools.build.android.ValidateAndLinkResourcesAction main
SEVERE: Error while validating and linking resources
com.google.devtools.build.android.aapt2.ResourceLinker$LinkError: java.nio.file.NoSuchFileException: bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_espresso_espresso_idling_resource_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_runner_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_monitor_symbols\symbols.zip
        at com.google.devtools.build.android.aapt2.ResourceLinker$LinkError.of(ResourceLinker.java:112)
        at com.google.devtools.build.android.aapt2.ResourceLinker.lambda$rethrowLinkError$8(ResourceLinker.java:338)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
        at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at com.google.devtools.build.android.aapt2.ResourceLinker.compiledResourcesToPaths(ResourceLinker.java:301)
        at com.google.devtools.build.android.aapt2.ResourceLinker.linkStatically(ResourceLinker.java:244)
        at com.google.devtools.build.android.ValidateAndLinkResourcesAction.main(ValidateAndLinkResourcesAction.java:195)
        at com.google.devtools.build.android.ResourceProcessorBusyBox$Tool$13.call(ResourceProcessorBusyBox.java:138)
        at com.google.devtools.build.android.ResourceProcessorBusyBox.processRequest(ResourceProcessorBusyBox.java:239)
        at com.google.devtools.build.android.ResourceProcessorBusyBox.main(ResourceProcessorBusyBox.java:203)
Caused by: java.nio.file.NoSuchFileException: bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_espresso_espresso_idling_resource_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_runner_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_monitor_symbols\symbols.zip
        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:116)
        at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
        at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
        at com.google.devtools.build.android.aapt2.ResourceLinker.filterZip(ResourceLinker.java:315)
        at com.google.devtools.build.android.aapt2.ResourceLinker.lambda$compiledResourcesToPaths$5(ResourceLinker.java:297)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

Exception in thread "main" com.google.devtools.build.android.aapt2.ResourceLinker$LinkError: java.nio.file.NoSuchFileException: bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_espresso_espresso_idling_resource_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_runner_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_monitor_symbols\symbols.zip
        at com.google.devtools.build.android.aapt2.ResourceLinker$LinkError.of(ResourceLinker.java:112)
        at com.google.devtools.build.android.aapt2.ResourceLinker.lambda$rethrowLinkError$8(ResourceLinker.java:338)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
        at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at com.google.devtools.build.android.aapt2.ResourceLinker.compiledResourcesToPaths(ResourceLinker.java:301)
        at com.google.devtools.build.android.aapt2.ResourceLinker.linkStatically(ResourceLinker.java:244)
        at com.google.devtools.build.android.ValidateAndLinkResourcesAction.main(ValidateAndLinkResourcesAction.java:195)
        at com.google.devtools.build.android.ResourceProcessorBusyBox$Tool$13.call(ResourceProcessorBusyBox.java:138)
        at com.google.devtools.build.android.ResourceProcessorBusyBox.processRequest(ResourceProcessorBusyBox.java:239)
        at com.google.devtools.build.android.ResourceProcessorBusyBox.main(ResourceProcessorBusyBox.java:203)
Caused by: java.nio.file.NoSuchFileException: bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_espresso_espresso_idling_resource_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_runner_symbols\symbols.zip;bazel-out\android-armeabi-v7a-fastbuild\bin\external\maven\androidx_test_monitor_symbols\symbols.zip
        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:116)
        at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
        at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
        at com.google.devtools.build.android.aapt2.ResourceLinker.filterZip(ResourceLinker.java:315)
        at com.google.devtools.build.android.aapt2.ResourceLinker.lambda$compiledResourcesToPaths$5(ResourceLinker.java:297)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Target //src/test:greeter_test_app failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 11.788s, Critical Path: 9.73s
INFO: 72 processes: 43 local, 29 worker.
FAILED: Build did NOT complete successfully
@jin

This comment has been minimized.

Copy link
Member

commented Aug 7, 2019

Figured out the root cause. The params file contains:

--compiledDep
'bazel-out/android-armeabi-v7a-fastbuild/bin/external/maven/androidx_test_monitor_symbols/symbols.zip;bazel-out/android-armeabi-v7a-fastbuild/bin/external/maven/androidx_test_core_symbols/symbols.zip'

Note the use of ; as a delimiter. --compiledDep is a list of paths converted with PathListConverter

@Option(
name = "compiledDep",
documentationCategory = OptionDocumentationCategory.UNCATEGORIZED,
effectTags = {OptionEffectTag.UNKNOWN},
defaultValue = "",
converter = Converters.PathListConverter.class,
category = "input",
allowMultiple = true,
help = "Compiled resource dependencies to link.")
public List<Path> compiledDeps;

which is delimited with : instead ;

But the compiledDeps execution paths were joined with getHostPathSeparator(), which is ; for Windows only.

return OS.getCurrent() == OS.WINDOWS ? ";" : ":";

@jin

This comment has been minimized.

Copy link
Member

commented Aug 7, 2019

Now I'm running into another error:

ERROR: C:/users/jingwen/code/rules_jvm_external/examples/android_instrumentation_test/src/main/BUILD:1:1: Processing Android resources for //src/main:greeter_app failed (Exit 1)
Aug 07, 2019 4:24:18 PM com.google.devtools.build.android.ResourceProcessorBusyBox processRequest
SEVERE: Error during processing
java.nio.file.InvalidPathException: Illegal character [:] in path at index 4: ///C:/Users/jingwen/AppData/Local/Temp/android_resources_tmp14123356876479215233/linked/bin.-pb.apk
        at java.base/sun.nio.fs.WindowsPathParser.nextSlash(WindowsPathParser.java:212)
        at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:111)
        at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
        at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
        at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
        at java.base/java.nio.file.Path.of(Path.java:147)
        at java.base/java.nio.file.Paths.get(Paths.java:69)
        at com.google.devtools.build.android.aapt2.ProtoApk.asApkPath(ProtoApk.java:229)
        at com.google.devtools.build.android.aapt2.ResourceLinker.link(ResourceLinker.java:565)
        at com.google.devtools.build.android.aapt2.ResourceLinker.link(ResourceLinker.java:546)
        at com.google.devtools.build.android.Aapt2ResourcePackagingAction.main(Aapt2ResourcePackagingAction.java:190)
        at com.google.devtools.build.android.ResourceProcessorBusyBox$Tool$14.call(ResourceProcessorBusyBox.java:144)
        at com.google.devtools.build.android.ResourceProcessorBusyBox.processRequest(ResourceProcessorBusyBox.java:242)
        at com.google.devtools.build.android.ResourceProcessorBusyBox.main(ResourceProcessorBusyBox.java:203)

Exception in thread "main" java.nio.file.InvalidPathException: Illegal character [:] in path at index 4: ///C:/Users/jingwen/AppData/Local/Temp/android_resources_tmp14123356876479215233/linked/bin.-pb.apk
        at java.base/sun.nio.fs.WindowsPathParser.nextSlash(WindowsPathParser.java:212)
        at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:111)
        at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
        at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
        at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
        at java.base/java.nio.file.Path.of(Path.java:147)
        at java.base/java.nio.file.Paths.get(Paths.java:69)
        at com.google.devtools.build.android.aapt2.ProtoApk.asApkPath(ProtoApk.java:229)
        at com.google.devtools.build.android.aapt2.ResourceLinker.link(ResourceLinker.java:565)
        at com.google.devtools.build.android.aapt2.ResourceLinker.link(ResourceLinker.java:546)
        at com.google.devtools.build.android.Aapt2ResourcePackagingAction.main(Aapt2ResourcePackagingAction.java:190)
        at com.google.devtools.build.android.ResourceProcessorBusyBox$Tool$14.call(ResourceProcessorBusyBox.java:144)
        at com.google.devtools.build.android.ResourceProcessorBusyBox.processRequest(ResourceProcessorBusyBox.java:242)
        at com.google.devtools.build.android.ResourceProcessorBusyBox.main(ResourceProcessorBusyBox.java:203)
Target //src/test:greeter_test_app failed to build
Use --verbose_failures to see the command lines of failed build steps.
jin added a commit that referenced this issue Aug 7, 2019
bazel-io pushed a commit that referenced this issue Aug 8, 2019
Fix Android builds with aapt2 on Windows, part 1
Error messages: #9102 (comment)

Closes #9118.

PiperOrigin-RevId: 262258777

@bazel-io bazel-io closed this in 4f5cede Aug 8, 2019

bazel-io pushed a commit that referenced this issue Aug 10, 2019
Automated rollback of commit 5d0dde0.
*** Reason for rollback ***

Rollforward to default Bazel to use aapt2, and fix native tests to use aapt2.

Fixes to Windows build failures have rolled out:

fcfe560
unknown commit
unknown commit
unknown commit (Windows test coverage)

*** Original change description ***

Automated rollback of commit b98ad77.

*** Reason for rollback ***

Broke Android build on Windows
#9102

*** Original change description ***

Default Bazel to use aapt2 and fix native tests to use aapt2.

This CL sets --incompatible_use_aapt2_by_default=true and --android_aapt=aapt2. It also adds aapt2 to the mock SDK for Android analysis and integration tests.

For the tests that assume and depend on AAPT, we explicitly pin them to use AAPT....

***

RELNOTES:
PiperOrigin-RevId: 262672156
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.