Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linting not working in Android Studio using rules_jvm_external #681

Closed
ownbee opened this issue Apr 17, 2019 · 28 comments
Closed

Linting not working in Android Studio using rules_jvm_external #681

ownbee opened this issue Apr 17, 2019 · 28 comments

Comments

@ownbee
Copy link

ownbee commented Apr 17, 2019

Android Studio plugin cannot resolve external libraries imported by bazelbuild/rules_jvm_external. I've tried so many setting and configurations, reinstalling android studio and plugin and clicked on the sync button (all the variants) without any success.

This is what I see when I open any source file (java) in my project:
cannotresolvsymbol

BUILD file:

android_library(
    name = "remote_control_activity",
    srcs = glob([
        "*.java",
        "CustomWidgets/**/*.java",
    ]),
    assets = glob([
        "assets/**",
    ]),
    assets_dir = "assets",
    manifest = "AndroidManifest.xml",
    resource_files = glob([
        "res/**",
        "CustomWidgets/res/**",
    ]),
    deps = [
        artifact("androidx.appcompat:appcompat"),
        artifact("androidx.core:core"),
        artifact("androidx.drawerlayout:drawerlayout"),
        #artifact("androidx.fragment:fragment"),
        "@maven//:androidx_fragment_fragment",
        artifact("com.google.android.material:material"),
    ],
)

Building and installing on emulator have always worked fine in IDE but I can't get the IDE linting to work (only those imported by rules_jvm_external) which make Android Studio pretty useless with Bazel.

Am I doing something wrong?
Is it intended to not work?
Missing support for rules_jvm_external?
Please enlighten me!

OS: Linux (Ubuntu 18.04)
Bazel version: 0.24.0
Andorid Studio: 3.3.0 and 3.4.0-RC3
Bazel plugin: Many different commits on HEAD:master during the past months.
Language: Java (Android)

Best Regards

@chaoren
Copy link
Collaborator

chaoren commented Apr 17, 2019

Missing support for rules_jvm_external?

Probably this. But if rules_jvm_external has the providers that we expect, then it should work.

@jin
Copy link
Member

jin commented Apr 17, 2019

Opened an issue here: bazelbuild/rules_jvm_external#118

@jin jin self-assigned this Apr 17, 2019
@chaoren
Copy link
Collaborator

chaoren commented Apr 17, 2019

I just tried out rules_jvm_external, and it does work for me. The problem is probably with your setup. What version of the bazel plugin are you using?

@chaoren
Copy link
Collaborator

chaoren commented Apr 17, 2019

Also if you can share your .bazelproject file.

@chaoren
Copy link
Collaborator

chaoren commented Apr 17, 2019

Or even your whole project.

@jin
Copy link
Member

jin commented Apr 17, 2019

also, which version of rules_jvm_external is that?

@chaoren
Copy link
Collaborator

chaoren commented Apr 17, 2019

image

$ cat WORKSPACE
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

RULES_JVM_EXTERNAL_TAG = "2.0.1"
RULES_JVM_EXTERNAL_SHA = "55e8d3951647ae3dffde22b4f7f8dee11b3f70f3f89424713debd7076197eaca"

http_archive(
    name = "rules_jvm_external",
    strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
    sha256 = RULES_JVM_EXTERNAL_SHA,
    url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [
        "androidx.appcompat:appcompat:1.0.2",
        "androidx.core:core:1.0.1",
        "androidx.drawerlayout:drawerlayout:1.0.0",
        "androidx.fragment:fragment:1.0.0",
        "com.google.android.material:material:1.0.0",
    ],
    repositories = [
        "https://jcenter.bintray.com/",
        "https://maven.google.com",
        "https://repo1.maven.org/maven2",
    ],
    fetch_sources = True,
)
$ cat BUILD
load("@rules_jvm_external//:defs.bzl", "artifact")

android_library(
    name = "test_deps",
    srcs = ["src/test/Test.java"],
    deps = [
        artifact("androidx.appcompat:appcompat"),
        artifact("androidx.core:core"),
        artifact("androidx.drawerlayout:drawerlayout"),
        artifact("androidx.fragment:fragment"),
        artifact("com.google.android.material:material"),
    ],
)
$ cat .aswb/.bazelproject
directories:
  .

targets:
  //:test_deps

android_sdk_platform: android-stable

@jin jin removed their assignment Apr 17, 2019
@ownbee
Copy link
Author

ownbee commented Apr 17, 2019

Thanks for the fast answers!

That's good news, then I have to figure out what's wrong with my environment.

The latset Bazel plugin I've tried was built locally on commit 219d81d (latest master yesterday) using instructions in README. Since I cant find any bazel plugin inside Android Studio IDE using default repositories.

I'm working on a private repository outside Github. So unfortunately, I can't share the whole project that easily. However, here are some more snippets from files that you asked about.

WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
android_sdk_repository(
    name = "androidsdk",
    api_level = 28,
    build_tools_version = "29.0.0-rc2",
)

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

RULES_JVM_EXTERNAL_TAG = "1.3"

RULES_JVM_EXTERNAL_SHA = "63a9162eb8b530da76453857bd3404db8852080976b93f237577f8000287e73d"

http_archive(
    name = "rules_jvm_external",
    sha256 = RULES_JVM_EXTERNAL_SHA,
    strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
    url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [
        "androidx.appcompat:appcompat:1.0.2",
        "androidx.recyclerview:recyclerview:1.0.0",
        "androidx.fragment:fragment:1.0.0",
        "androidx.annotation:annotation:1.0.2",
        "androidx.drawerlayout:drawerlayout:1.0.0",
        "androidx.preference:preference:1.0.0",
        "androidx.swiperefreshlayout:swiperefreshlayout:1.0.0",
        "androidx.core:core:1.0.0",
        "androidx.viewpager:viewpager:1.0.0",
        "androidx.cardview:cardview:1.0.0",
        "com.google.android.material:material:1.0.0",
        "com.takisoft.preferencex:preferencex:1.0.0",
        "jp.wasabeef:recyclerview-animators:3.0.0",
        "com.github.philjay:mpandroidchart:v3.1.0",
        "io.grpc:grpc-core:1.19.0",
        "io.grpc:grpc-okhttp:1.19.0",
        "io.grpc:grpc-protobuf-lite:1.19.0",
        "io.grpc:grpc-stub:1.19.0",
        "com.google.guava:guava:27.1-android",
        "com.google.protobuf:protobuf-lite:3.0.1",
    ],
    # Fetch srcjars. Defaults to False.
    fetch_sources = True,
    repositories = [
        "https://jcenter.bintray.com/",
        "https://maven.google.com",
        "https://repo1.maven.org/maven2",
        "https://jitpack.io",
    ],
)

.bazelproject:

directories:
  .

targets:
  # Add targets that reach the source code that you want to resolve here
  # By default, we've added all targets in your workspace
  //android-client:all

additional_languages:
  # Uncomment any additional languages you want supported
  # c
  # dart
  # kotlin
  # python

android_sdk_platform: android-28

workspace_type: android

@ownbee
Copy link
Author

ownbee commented Apr 17, 2019

Do I need to configure something inside IDE except installing Bazel plugin?

@chaoren
Copy link
Collaborator

chaoren commented Apr 17, 2019

When you run Bazel -> Sync -> Sync Project with BUILD Files do you get any errors?

@chaoren
Copy link
Collaborator

chaoren commented Apr 17, 2019

If you expand your External Libraries tree, do you see these?

image

@ownbee
Copy link
Author

ownbee commented Apr 17, 2019

When you run Bazel -> Sync -> Sync Project with BUILD Files do you get any errors?

Nope, not a single error.

If you expand your External Libraries tree, do you see these?

I see this. Shouldn't there be any jars beside "res" when expanding? It also seems like there are some missing libraries compared to the list of installed with rules_jvm_external.

external_libraries

@chaoren
Copy link
Collaborator

chaoren commented Apr 17, 2019

The Android SDK is okay. Mine looks like that too.

My maven jars show up as classes_and_libs_merged, and you don't have those, but it looks like you do have them as individual jars.

@ownbee
Copy link
Author

ownbee commented Apr 18, 2019

Alright, do you have any clue why the linting isn't working for me?
Is there something I could try?
Can it be how project structure related? (e.g. how the build files looks like)

@jin
Copy link
Member

jin commented Apr 18, 2019 via email

@ownbee
Copy link
Author

ownbee commented Apr 19, 2019

I've tried it now and I have the same problem.

Imported Bazel project in rules_jvm_external/examples/android_local_test.

Project view:
image

MainActivity.java:
image

Build output:

.bazelproject:

directories:
  # Add the directories you want added as source here
  # By default, we've added your entire workspace ('.')
  .

targets:
  # Add targets that reach the source code that you want to resolve here
  # By default, we've added all targets in your workspace
  //src/main:my_lib

additional_languages:
  # Uncomment any additional languages you want supported
  # c
  # dart
  # python


# Please uncomment an android-SDK platform. Available SDKs are:
android_sdk_platform: android-28

Modified WORKSPACE:

android_sdk_repository(
    name = "androidsdk",
    # Added This line (since env ANDROID_HOME dont take in AS):
    path = "/home/alex/Android/Sdk",
)

When I build project in Android Studio using the "play" button it succeeds:

bazel build --tool_tag=ijwb:AndroidStudio --curses=no --color=yes --experimental_ui=no --progress_in_terminal_title=no -- //src/main:my_lib
INFO: Loading complete.  Analyzing...
INFO: Found 1 target...
INFO: Building...
Target //src/main:my_lib up-to-date:
  bazel-bin/src/main/my_lib.srcjar
  bazel-bin/src/main/my_lib_resources-src.jar
  bazel-bin/src/main/my_lib_resources.jar
  bazel-bin/src/main/libmy_lib.jar
INFO: Elapsed time: 3.167s, Critical Path: 1.55s
INFO: 29 processes: 29 linux-sandbox.

Sync output (no errors or warnings what I can see):

Syncing project: Sync (incremental)...
Updating VCS...
Running Bazel info...
Command: bazel info --tool_tag=ijwb:AndroidStudio --curses=no --color=yes --experimental_ui=no --progress_in_terminal_title=no --

Computing VCS working set...
Sync targets from project view:
  //src/main:my_lib

Building blaze targets...
Command: bazel build --tool_tag=ijwb:AndroidStudio --keep_going --build_event_binary_file=/tmp/intellij-bep-660a0e44-1b13-4e7d-98c9-964858cd4a82 --nobuild_event_binary_file_path_conversion --curses=no --color=yes --experimental_ui=no --progress_in_terminal_title=no --aspects=@intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect --override_repository=intellij_aspect=/home/alex/.AndroidStudio3.4/config/plugins/aswb/aspect --output_groups=intellij-info-android,intellij-info-generic,intellij-info-java,intellij-resolve-android,intellij-resolve-java -- //src/main:my_lib

INFO: Loading complete.  Analyzing...
INFO: Loading package: @androidsdk//
INFO: Found 1 target...
Aspect @intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect of //src/main:my_lib up-to-date:
  bazel-bin/external/androidsdk/sdk-28--907030343.intellij-info.txt
  bazel-bin/external/maven/androidx_interpolator_interpolator_1_0_0--1229704323.intellij-info.txt
  bazel-bin/external/maven/androidx_lifecycle_lifecycle_runtime_2_0_0-846914157.intellij-info.txt
  bazel-bin/external/maven/androidx_localbroadcastmanager_localbroadcastmanager_1_0_0-2128280557.intellij-info.txt
  bazel-bin/external/maven/androidx_versionedparcelable_versionedparcelable_1_0_0-1115242419.intellij-info.txt
  bazel-bin/external/maven/androidx_core_core_1_0_1-1434977918.intellij-info.txt
  bazel-bin/external/maven/androidx_arch_core_core_runtime_2_0_0--1122201894.intellij-info.txt
  bazel-bin/external/maven/androidx_lifecycle_lifecycle_livedata_core_2_0_0-1950011965.intellij-info.txt
  bazel-bin/external/maven/androidx_lifecycle_lifecycle_livedata_2_0_0-963602475.intellij-info.txt
  bazel-bin/external/maven/androidx_print_print_1_0_0--545505959.intellij-info.txt
  bazel-bin/external/maven/androidx_documentfile_documentfile_1_0_0-456963453.intellij-info.txt
  bazel-bin/external/maven/androidx_lifecycle_lifecycle_viewmodel_2_0_0-406575417.intellij-info.txt
  bazel-bin/external/maven/androidx_loader_loader_1_0_0-1105440509.intellij-info.txt
  bazel-bin/external/maven/androidx_legacy_legacy_support_core_utils_1_0_0-2083723243.intellij-info.txt
  bazel-bin/external/maven/androidx_customview_customview_1_0_0-1153636125.intellij-info.txt
  bazel-bin/external/maven/androidx_slidingpanelayout_slidingpanelayout_1_0_0--1366653889.intellij-info.txt
  bazel-bin/external/maven/androidx_cursoradapter_cursoradapter_1_0_0-1061627249.intellij-info.txt
  bazel-bin/external/maven/androidx_coordinatorlayout_coordinatorlayout_1_0_0-664780019.intellij-info.txt
  bazel-bin/external/maven/androidx_viewpager_viewpager_1_0_0-789732539.intellij-info.txt
  bazel-bin/external/maven/androidx_asynclayoutinflater_asynclayoutinflater_1_0_0--1717034911.intellij-info.txt
  bazel-bin/external/maven/androidx_swiperefreshlayout_swiperefreshlayout_1_0_0--1240595459.intellij-info.txt
  bazel-bin/external/maven/androidx_drawerlayout_drawerlayout_1_0_0-425174013.intellij-info.txt
  bazel-bin/external/maven/androidx_legacy_legacy_support_core_ui_1_0_0--119831294.intellij-info.txt
  bazel-bin/external/maven/androidx_fragment_fragment_1_0_0--273609827.intellij-info.txt
  bazel-bin/external/maven/androidx_vectordrawable_vectordrawable_1_0_1--807679682.intellij-info.txt
  bazel-bin/external/maven/androidx_vectordrawable_vectordrawable_animated_1_0_0-536523277.intellij-info.txt
  bazel-bin/external/maven/androidx_appcompat_appcompat_1_0_2--722854841.intellij-info.txt
  bazel-bin/src/main/my_lib--1059706510.intellij-info.txt
  bazel-bin/external/bazel_tools/tools/jdk/remote_toolchain-1176928112.intellij-info.txt
  bazel-bin/external/bazel_tools/tools/jdk/current_java_toolchain--1286701614.intellij-info.txt
  bazel-bin/external/maven/androidx_annotation_annotation_1_0_2--1692957313.intellij-info.txt
  bazel-bin/external/maven/androidx_lifecycle_lifecycle_common_2_0_0--397548608.intellij-info.txt
  bazel-bin/external/maven/androidx_arch_core_core_common_2_0_0--876710477.intellij-info.txt
  bazel-bin/external/maven/androidx_collection_collection_1_0_0-191652381.intellij-info.txt
  bazel-bin/src/main/my_lib.java-manifest
  bazel-bin/external/maven/_aar/androidx_interpolator_interpolator_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_lifecycle_lifecycle_runtime_2_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_localbroadcastmanager_localbroadcastmanager_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_versionedparcelable_versionedparcelable_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_core_core_1_0_1/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_arch_core_core_runtime_2_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_lifecycle_lifecycle_livedata_core_2_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_lifecycle_lifecycle_livedata_2_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_print_print_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_documentfile_documentfile_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_lifecycle_lifecycle_viewmodel_2_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_loader_loader_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_legacy_legacy_support_core_utils_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_customview_customview_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_slidingpanelayout_slidingpanelayout_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_cursoradapter_cursoradapter_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_coordinatorlayout_coordinatorlayout_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_viewpager_viewpager_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_asynclayoutinflater_asynclayoutinflater_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_swiperefreshlayout_swiperefreshlayout_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_drawerlayout_drawerlayout_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_legacy_legacy_support_core_ui_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_fragment_fragment_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_vectordrawable_vectordrawable_1_0_1/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_vectordrawable_vectordrawable_animated_1_0_0/classes_and_libs_merged.jar
  bazel-bin/external/maven/_aar/androidx_appcompat_appcompat_1_0_2/classes_and_libs_merged.jar
  bazel-bin/src/main/my_lib_resources.jar
  bazel-bin/src/main/my_lib_resources-src.jar
  bazel-bin/src/main/libmy_lib.jar
  bazel-bin/src/main/libmy_lib-hjar.jar
  bazel-bin/src/main/libmy_lib-src.jar
  bazel-bin/src/main/libmy_lib.jdeps
INFO: Elapsed time: 0.788s, Critical Path: 0.01s
INFO: 0 processes.
build invocation result: success
Parsing build outputs...
Total rules: 34, new/changed: 0, removed: 0
Reading IDE info result...
Loaded 0 aspect files, total size 0kB
Target map size: 34
Reading jdeps files...
Loaded 0 jdeps files, total size 0kB
Reading package manifests...
Java content entry count: 1
Updating Unpacked AAR libraries...
Updating Jar Cache...
Prefetching files...
Refreshing files
Computing directory structure...
Committing project structure...
Workspace has 30 libraries
Android resource module count: 1, run config modules: 0, order entries: 2, generated resources: 0
Workspace has 3 modules
Updating in-memory state...

Timing summary:
BlazeInvocation: 972ms, Prefetching: 1ms, Other: 70ms

Is there any cache related to Bazel plugin that I could try clear?

@ownbee
Copy link
Author

ownbee commented Apr 19, 2019

After reinstalling Android SDK (by deleting the SDK folder and installed it in AS IDE) and resynced I managed to get three "classes_and_libs_merged" in "External Libraries":

image

And the linting started working on those:
image

However, other libraries (androidx.appcompat in this case) does not show up as "classes_and_libs_merged" and for those it does not work. The interesting part is that androidx.appcompat only has a res folder while "classes_and_libs_merged" has classes as well.

It seem like it's a little bit random whether "classes_and_libs_merged" shows up or not. Can it be due to some conflicts or that the Bazel plugin finds the wrong path to the libraries?

@chaoren
Copy link
Collaborator

chaoren commented Apr 19, 2019

I thought these maven jars were just downloaded off of maven. Why would the Android SDK affect them at all? Unless they also exist in the Android SDK, and they're conflicting somehow?

@jin
Copy link
Member

jin commented Apr 19, 2019

These aar and jar files don't live in the SDK. There are some that do, but they are not these artifacts.

@ownbee
Copy link
Author

ownbee commented Apr 19, 2019

I have no idea what I am doing, I am just trying everything I can and showing the result for you in hope that you can help me figure it out. I have absolutely no experience with intellij plugins or how Android Studio IDE works.

I opened AS after I deleted SDK and then installed the SDK inside IDE, restarted IDE and then when I synced once. After that "classes_and_libs_merged" popped up for androidx.drawerlayout and when I imported it in java file, linting started working for that lib.

It's possible that reinstalling SDK had nothing to do with anything.

@ownbee
Copy link
Author

ownbee commented Apr 20, 2019

After some more digging, it seems like "External Libraries" contains stuff from alot of different directories.This is however expected since they have different origins, some comes from SDK, others from bazel toolchain or rules_jvm_external, right?

Problem is that libraries fetched/extracted by rules_jvm_external exists at multiple directories. At some places the libraries only contain resources of external libraries while others contains both resources and sources/jar/aar.

Here are some examples of the locations of libraries under External Libraries:

Lib Search dir Full Path
androidx.annotation external/maven/v1/https $HOME/.cache/bazel/_bazel_alex/ef4f691ef5f44e4e6b90077b39abc29f/execroot/__main__/external/maven/v1/https/maven.google.com/androidx/annotation/annotation/1.0.2/annotation-1.0.2.jar
andoidx.appcompat .aswb/.blaze/aar_libraries rules_jvm_external/examples/android_local_test/.aswb/.blaze/aar_libraries/appcompat-1.0.2_c6e528b0.aar/res
androidx.drawerlayout bazel-out/k8-fastbuild/bin/external/maven/_aar $HOME/.cache/bazel/_bazel_alex/ef4f691ef5f44e4e6b90077b39abc29f/execroot/__main__/bazel-out/k8-fastbuild/bin/external/maven/_aar/androidx_drawerlayout_drawerlayout_1_0_0/classes_and_libs_merged.jar

bazel-out/k8-fastbuild/bin/external/maven/_aar contains classes_and_libs_merged.jar for all external libraries fetch by rules_jvm_external but it seems that directory is not prioritized over the other.

The devil seems to be .aswb/.blaze/aar_libraries which only contains resources and it seems like this directory is always chosen by Android Studio.

I think both bazel-out/k8-fastbuild/bin/external/maven/_aar and external/maven/v1/https would work since they contains the jar/aar.

Is there any way of changing priority of directories searched?

@ownbee
Copy link
Author

ownbee commented Apr 21, 2019

I found the Android Studio system log (~/.AndroidStudio3.4/system/log/idea.log) and in there I found this:

2019-04-20 16:34:52,297 [xecutor-36]   WARN - android.libraries.UnpackedAars - Failed to extract AAR /home/alex/.cache/bazel/_bazel_alex/ef4f691ef5f44e4e6b90077b39abc29f/execroot/__main__/external/maven/v1/https/maven.google.com/androidx/legacy/legacy-support-core-ui/1.0.0/legacy-support-core-ui-1.0.0.aar to /home/alex/repositories/rules_jvm_external/examples/android_local_test/.aswb/.blaze/aar_libraries/legacy-support-core-ui-1.0.0_91064340.aar 
java.nio.file.NoSuchFileException: /home/alex/repositories/rules_jvm_external/examples/android_local_test/.aswb/.blaze/aar_libraries/legacy-support-core-ui-1.0.0_91064340.aar/jars/classes_and_libs_merged.jar
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
	at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
	at java.nio.file.Files.newOutputStream(Files.java:216)
	at java.nio.file.Files.copy(Files.java:3016)
	at com.google.idea.blaze.android.libraries.UnpackedAars.copyLocally(UnpackedAars.java:434)
	at com.google.idea.blaze.android.libraries.UnpackedAars.lambda$copyLocally$4(UnpackedAars.java:403)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

They appear for once for each rules_jvm_external library when doing a non incrementally sync in IDE.

@ownbee
Copy link
Author

ownbee commented Apr 21, 2019

I finally found a solution to my problem.

intellij/aswb/src/com/google/idea/blaze/android/sync/BlazeAndroidLibrarySource.java has some kind of filter that removes any jars that looks like it's overridden by an aar, called AarJarFilter. By disabling the filter:

//Original
@Override
public boolean test(BlazeLibrary blazeLibrary) {
  if (!(blazeLibrary instanceof BlazeJarLibrary)) {
    return true;
  }
  BlazeJarLibrary jarLibrary = (BlazeJarLibrary) blazeLibrary;
  ArtifactLocation location = jarLibrary.libraryArtifact.jarForIntellijLibrary();
  String configurationLessPath = location.getRelativePath();
  return !aarJarsPaths.contains(configurationLessPath);
}

// Disabled
@Override
public boolean test(BlazeLibrary blazeLibrary) {
   return true;
}

... both jar (classes_and_libs_merged.jar) and aar files shows up in External Libraries and linting works like a charm! Even layout preview started working (never worked for me with Bazel before I started using rules_jvm_external either).

@chaoren
Copy link
Collaborator

chaoren commented Apr 22, 2019

Thanks for the in-depth investigation. My .aswb/.blaze/aar_libraries is empty, and .aswb/.blaze/libraries contains:

$ ls .aswb/.blaze/libraries
annotation-1.0.2_4460b210.jar
classes_and_libs_merged_2df945e6.jar
classes_and_libs_merged_31b432a8.jar
classes_and_libs_merged_7a6998b2.jar
classes_and_libs_merged_a8a9fba9.jar
classes_and_libs_merged_fbd88b48.jar
java_compiler_d43c9198.jar
jdk_compiler_b0e94f2b.jar

Did you carry the project over from a previous version of the bazel plugin? If that's the case, then a freshly imported project with the latest version of the bazel plugin might not have had the same problem. Or just manually deleting .aswb/.blaze/aar_libraries could have fixed it.

Problem is that libraries fetched/extracted by rules_jvm_external exists at multiple directories. At some places the libraries only contain resources of external libraries while others contains both resources and sources/jar/aar.

The stuff under external and bazel-out are handled by bazel with rules_jvm_external rules. aswb/.blaze/aar_libraries is just a cache created by the bazel plugin to avoid jars disappearing from under the IDE. The IDE should only be able to see the jars copied into .aswb/.blaze/aar_libraries and .aswb/.blaze/libraries. What does your .aswb/.blaze/libraries contain?

@ownbee
Copy link
Author

ownbee commented Apr 22, 2019

Did you carry the project over from a previous version of the bazel plugin?

Yes I did at first, but I have tried removed the whole .aswb directory and reimport the projects more than once. But it did not change anything for me...

My .aswb/.blaze/libraries contain:

$ ls -a1 .aswb/.blaze/libraries/
.
..
annotation-1.0.2_75002acf.jar
annotation-1.0.2-sources_49c97a3a-src.jar
classes_and_libs_merged_15dd0009.jar
classes_and_libs_merged_31712fa8.jar
classes_and_libs_merged_34f7b2dc.jar
classes_and_libs_merged_5f96c948.jar
classes_and_libs_merged_6be66f86.jar
classes_and_libs_merged_9189ed62.jar
classes_and_libs_merged_9dd7f246.jar
classes_and_libs_merged_a344a90e.jar
classes_and_libs_merged_ac6aa512.jar
classes_and_libs_merged_ae7e3588.jar
classes_and_libs_merged_b229d588.jar
classes_and_libs_merged_c0127708.jar
classes_and_libs_merged_d73199c4.jar
classes_and_libs_merged_dd51e5a8.jar
grpc-core-1.19.0_5a0f8819.jar
grpc-core-1.19.0-sources_67f3ed84-src.jar
grpc-protobuf-lite-1.19.0_a9c159c5.jar
grpc-protobuf-lite-1.19.0-sources_f33f5330-src.jar
grpc-stub-1.19.0_87357c39.jar
java_compiler_1533de39.jar
jdk_compiler_81756a6a.jar
libsmartome_proto_java-hjar_fb8f9718.jar
libsmartome_proto_java-src_a5f97359-src.jar
lifecycle-common-2.0.0_f1639f88.jar
protobuf-lite-3.0.1_eda66f3d.jar

But this is with the fix I described above. Without the fix, it contain the all the files but classes_and_libs_merged*.jar files.

.aswb/.blaze/aar_libraries/ only contains resources (res) from external libraries. All the external libraries that origins from an aar have their sources (not res) in a classes_and_libs_merged*.jar file under .aswb/.blaze/libraries/.

$ find .aswb/.blaze/aar_libraries/ -type f -name "*jar"
# Nothing
$ find .aswb/.blaze/aar_libraries/ -type f -name "*aar"
# Nothing

EDIT: Sorry if you already read this. Mixed aar_libraries and libraries directory... Now fixed!

@jin
Copy link
Member

jin commented Apr 22, 2019

FWIW, I can reproduce the issue. A fresh import of rules_jvm_external/examples/android_local_test using the internal plugin build also results in #681 (comment)

ss
ss

My .aswb/.blaze/libraries shows:

$ ls .aswb/.blaze/libraries/
android_64662b89.jar                                assertj-core-3.12.1_79801000.jar
android-all-4.1.2_r1-robolectric-r1_f0bd31c7.jar    assertj-core-3.12.1-sources_f7ad9e6b-src.jar
android-all-4.2.2_r1.2-robolectric-r1_fec967b2.jar  classes_and_libs_merged_44991684.jar
android-all-4.3_r2-robolectric-r1_edf5e552.jar      java_compiler_554518eb.jar
android-all-4.4_r1-robolectric-r2_250d4158.jar      jdk_compiler_cdd91178.jar
android-all-5.0.2_r3-robolectric-r0_23f9e941.jar    libandroid-all_9dbd4ba3.jar
android-all-5.1.1_r9-robolectric-r2_9625ed39.jar    libandroid-all-jars_f426255c.jar
android-all-6.0.1_r3-robolectric-r1_82a15ac3.jar    libandroid-all-jars-src_101884f3-src.jar
android-all-7.0.0_r1-robolectric-r1_82ea3747.jar    libandroid-all-src_31952fba-src.jar
android-all-7.1.0_r7-robolectric-r1_36a11c3d.jar    lifecycle-common-2.0.0_b2a8a0ba.jar
android-all-8.0.0_r4-robolectric-r1_6a534803.jar    robolectric-4.1_7ae4af6c.jar
android-all-8.1.0-robolectric-4611349_cd17ea65.jar  robolectric-4.1-sources_62e211d7-src.jar
android-all-9-robolectric-4913185-2_5dbf9e39.jar    Runner_deploy_bd6aca85.jar
annotation-1.0.2_4c8ff4dd.jar                       shadows-framework-4.1_76632684.jar
annotation-1.0.2-sources_3dad5648-src.jar           shadows-framework-4.1-sources_b663f0ef-src.jar

My .aswb/.blaze/aar_libraries has:

$ ls .aswb/.blaze/aar_libraries/
appcompat-1.0.2_7339eca2.aar                 lifecycle-livedata-2.0.0_47050d1.aar
asynclayoutinflater-1.0.0_d93a9108.aar       lifecycle-livedata-core-2.0.0_9ca0a147.aar
coordinatorlayout-1.0.0_f8cbfe03.aar         lifecycle-runtime-2.0.0_f41656cb.aar
core-1.0.1_b9a06be4.aar                      lifecycle-viewmodel-2.0.0_7412da3.aar
core-runtime-2.0.0_80adbb3f.aar              loader-1.0.0_a12c92b8.aar
cursoradapter-1.0.0_171dc820.aar             localbroadcastmanager-1.0.0_6f67e21a.aar
customview-1.0.0_b4206db.aar                 print-1.0.0_454c077c.aar
documentfile-1.0.0_38bab7fc.aar              slidingpanelayout-1.0.0_9665a2b5.aar
drawerlayout-1.0.0_41a96ec0.aar              swiperefreshlayout-1.0.0_46c183f0.aar
fragment-1.0.0_ded0e155.aar                  vectordrawable-1.0.1_56617566.aar
interpolator-1.0.0_64ad25ac.aar              vectordrawable-animated-1.0.0_c90ac452.aar
legacy-support-core-ui-1.0.0_af7f2f0e.aar    versionedparcelable-1.0.0_fd644f43.aar
legacy-support-core-utils-1.0.0_c41e1a6.aar  viewpager-1.0.0_18e2c9af.aar

@chaoren
Copy link
Collaborator

chaoren commented Apr 23, 2019

Okay, I reproduce and figured out the problem. Fix should be available shortly.

@chaoren
Copy link
Collaborator

chaoren commented Apr 24, 2019

#711

@chaoren chaoren closed this as completed Apr 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants