New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On Windows: NullPointerException on fetching jars from repository defined with maven_server #1941

Open
Helcaraxan opened this Issue Oct 13, 2016 · 26 comments

Comments

Projects
None yet
8 participants
@Helcaraxan
Copy link

Helcaraxan commented Oct 13, 2016

Bazel manages to fetch all external jar dependencies defined with maven_jar from the standard repositories. However every single jar that uses a repository defined with maven_server results in a NullPointerException on transfer.

ERROR: C:/Projects/everything/third_party/flagz/BUILD:5:1: no such package '@flagz_core_artifact//jar': Failed to fetch Maven dependency: Could not transfer artifact org.flagz:flagz:jar:2.2.0 from/to flagz_core_artifact (https://<obfuscated-host>/content/repositories/releases/): NullPointerException and referenced by '//third_party/flagz:flagz'.
ERROR: C:/Projects/everything/third_party/flagz/BUILD:15:1: no such package '@flagz_core_artifact//jar': Failed to fetch Maven dependency: Could not transfer artifact org.flagz:flagz:jar:2.2.0 from/to flagz_core_artifact (https://<obfuscated-host>/content/repositories/releases/): NullPointerException and referenced by '//third_party/flagz:flagz_scala'.
ERROR: C:/Projects/everything/third_party/flagz/BUILD:15:1: no such package '@flagz_scala_artifact//jar': Failed to fetch Maven dependency: Could not transfer artifact org.flagz:flagz-scala_2.11:jar:2.2.0 from/to flagz_scala_artifact (https://<obfuscated-host>/content/repositories/releases/): NullPointerException and referenced by '//third_party/flagz:flagz_scala'.
[...]
@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Oct 13, 2016

It's worth mentioning that our maven_server here has authentication enabled, which is present in ~/.m2/settings.xml.

@dslomov

This comment has been minimized.

Copy link
Contributor

dslomov commented Oct 13, 2016

@kchodorow could you help? What is the best way to debug this?

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Oct 18, 2016

@dslomov, @kchodorow: This currently blocks me. Is there anything I can do to help diagnose, etc? Any workarounds?

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Oct 20, 2016

Might be able to work around this via #1960's workaround. I'll try it out.

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Oct 30, 2016

Wasn't able to work around.

@dinowernli provoked a great thought - on Windows, a user's home directory is USERPROFILE, not HOME. By default, Windows doesn't have the HOME environment variable (or understand ~/ come to that). Could that be the problem here...?

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Nov 3, 2016

@dslomov any update on this one?

@kchodorow

This comment has been minimized.

Copy link
Contributor

kchodorow commented Nov 7, 2016

One possible workaround is to use the new Skylark maven_jar rule, instead of the built-in one. This rule doesn't take server= or repository= attributes, but it picks up on your system settings.xml file. You can try it out by specifying:

load('@bazel_tools//tools/build_defs/repo:maven_rules.bzl', 'maven_jar')
# Or continue to use some native, some skylark maven rules by giving the skylark version an alternative name:
load('@bazel_tools//tools/build_defs/repo:maven_rules.bzl', skylark_maven_jar = 'maven_jar')

@dslomov the error message is coming from https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/bazel/repository/MavenDownloader.java#L125. Basically, Aether is throwing an NPE trying to resolve the artifact, and we're wrapping it. I probably set something up wrong about how it resolves custom repositories+auth, but I don't know what it is.

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Nov 9, 2016

Thanks; I'm trying that out now.

@damienmg

This comment has been minimized.

Copy link
Contributor

damienmg commented Dec 5, 2016

Friendly ping?

@Helcaraxan

This comment has been minimized.

Copy link

Helcaraxan commented Dec 5, 2016

We have not been able to make much progress / tests here due to other things but we will be coming back to this soon. Will update as soon as I have more information.

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Dec 6, 2016

I have in my workspace:

git_repository(
  name = "io_bazel",
  remote = "https://github.com/bazelbuild/bazel.git",
  tag = "0.4.1",
)
load('@io_bazel//tools/build_defs/repo:maven_rules.bzl', skylark_maven_jar = 'maven_jar')

I have bazel 0.4.2-rc2 installed.

Initially, in msys2 shell, I got (targets obfuscated)

ERROR: C:/src/s/iw/everything/third_party/some_dep/BUILD:5:1: no such package '@some_dep_artifact//jar': Traceback (most recent call last):
        File "C:/tools/msys64/tmp/_bazel_Pete/fIsNmaZv/external/io_bazel/tools/build_defs/repo/maven_rules.bzl", line 252
                _check_dependencies(ctx)
        File "C:/tools/msys64/tmp/_bazel_Pete/fIsNmaZv/external/io_bazel/tools/build_defs/repo/maven_rules.bzl", line 50, in _check_dependencies
                fail("maven_jar requires %s as a depe...)
maven_jar requires mvn as a dependency. Please check your PATH. and referenced by '//third_party/some_dep:target'.

I installed maven via chocolatey and did export PATH=$PATH:/c/tools/apache-maven-3.3.9/bin.

Then, I get

ERROR: C:/src/s/iw/everything/blah/BUILD:3:1: no such package '@some_dep_artifact//jar': Traceback (most recent call last):
        File "C:/tools/msys64/tmp/_bazel_Pete/fIsNmaZv/external/io_bazel/tools/build_defs/repo/maven_rules.bzl", line 252
                _check_dependencies(ctx)
        File "C:/tools/msys64/tmp/_bazel_Pete/fIsNmaZv/external/io_bazel/tools/build_defs/repo/maven_rules.bzl", line 50, in _check_dependencies
                fail("maven_jar requires %s as a depe...)
maven_jar requires openssl as a dependency. Please check your PATH. and referenced by '//blah'.

However - I appear to have an openssl in my PATH - I believe from msys2's install, /usr/bin/openssl.exe:

MSYS /c/src/s/iw/everything
$ openssl version
OpenSSL 1.0.2j  26 Sep 2016

I also have the chocolatey OpenSSL.Light 1.1.0.20160926 package installed, and git-for-windows also has an openssl in it.

@Helcaraxan

This comment has been minimized.

Copy link

Helcaraxan commented Dec 16, 2016

Any news on this issue with the latest information provided by @petemounce?

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Jan 4, 2017

Friendly ping?

@damienmg

This comment has been minimized.

Copy link
Contributor

damienmg commented Jan 9, 2017

pinging @dslomov and @kchodorow

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Jan 26, 2017

Friendly follow-up ping?

@dslomov dslomov added this to the 0.5 milestone Feb 14, 2017

@meteorcloudy

This comment has been minimized.

Copy link
Member

meteorcloudy commented Feb 28, 2017

@dslomov Ping.

@dslomov

This comment has been minimized.

Copy link
Contributor

dslomov commented Mar 6, 2017

Sorry, just go to this issue. I bet your openssl is in msys's PATH but not in Windows' PATH variable.

@kchodorow, what would be the easiest way to reproduce this?

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Mar 6, 2017

C:\Users\Pete>where openssl
C:\Program Files\OpenVPN\bin\openssl.exe
C:\Program Files\Git\mingw64\bin\openssl.exe
C:\Program Files\Git\usr\bin\openssl.exe
c:\tools\msys64\usr\bin\openssl.exe

C:\Users\Pete>openssl version
OpenSSL 1.0.2i  22 Sep 2016
@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Mar 28, 2017

@dslomov @meteorcloudy @damienmg is there anything else I can provide to help?

@kchodorow

This comment has been minimized.

Copy link
Contributor

kchodorow commented Mar 28, 2017

Sorry about the delay, this got lost in my inbox!

The maven_jar skylark rule is failing at ctx.which('openssl'). I'm not sure what implementation of which Skylark is using on Windows, but I'm guessing that if it can't find openssl, it's wrong.

@petemounce

This comment has been minimized.

Copy link
Contributor

petemounce commented Mar 28, 2017

Ah. which does not exist on Windows out of the box, so I guess if it falls through to a binary then it might be that? cmd gives you where, PowerShell gives you Get-Command.

@damienmg

This comment has been minimized.

Copy link
Contributor

damienmg commented Mar 29, 2017

According @meteorcloudy, repository_ctx.which is implemented the same way on windows that we do on Linux: we iterate over the PATH to find the environment variable.

Could that be a problem with casing of the environment?

@damienmg

This comment has been minimized.

Copy link
Contributor

damienmg commented Mar 29, 2017

@laszlocsomor

This comment has been minimized.

Copy link
Contributor

laszlocsomor commented Mar 29, 2017

About envvars: on Windows the Bazel client will uppercase a few envvars, the rest of them are passed to the server as-is. Even if $PATH is defined as Path, as it is on Windows, we will pass it on as PATH.

EDIT: this bugfix is not in release bazel yet, 0.4.5 still doesn't have it, 0.4.6 / 0.5.0 will, or build Bazel from HEAD.

@laszlocsomor

This comment has been minimized.

Copy link
Contributor

laszlocsomor commented Mar 29, 2017

This error could be related to https://stackoverflow.com/questions/42341374.

@dslomov dslomov modified the milestones: 0.6, 0.5 Apr 4, 2017

@tekumara

This comment has been minimized.

Copy link

tekumara commented Jan 3, 2019

To fix this error I had to make sure maven_server name and server id matched:

maven_server(
  name = "the_cave",
  url = "https://myartefacts.xyz/artifactory/libs",
)

~/.m2/settings.xml

     <servers>
         <server>
             <id>the_cave</id>
             <username>alibaba</username>
             <password>openseasame</password>
         </server>
     </servers>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment