new_http_archive should support specifying a root directory

[shahms]

At the moment, new_http_archive requires all rules in the BUILD file to be relative to the root of the archive. However, standard operating procedure for source code archives is to bundle files beneath a single directory which typically includes the project name and version (e.g. libfoo-1.2.4a.zip would unpack to libfoo-1.2.4a/). In particular, GitHub zip files follow this procedure. Being able to specify an archive root directory would make maintaining external BUILD files for these archives much simpler and seems like it should be relatively straightforward to support.

[kchodorow]

Good idea. I'll add an attribute, maybe something like root_directory.

[kamalmarhubi]

Just adding a +1 here!

[kamalmarhubi]

@kchodorow thanks!! 👍

[kamalmarhubi]

Does this actually work? I tried to use it and it doesn't seem to do the right thing.

My example: https://github.com/kamalmarhubi/sqlite3-bazel/tree/use-strip-prefix, I get errors about missing input.

The test external_integration_test also fails. It has an error in the attribute name, and when fixed I get the same problem of not finding source files:

ERROR: missing input file '@x//:w'.

Full output at: https://gist.github.com/kamalmarhubi/1128b38948a4077e376c

[kamalmarhubi]

That's from running bazel test //src/test/shell/bazel:external_integration_test --test_output=errors after making this patch:

diff --git a/src/test/shell/bazel/external_integration_test.sh b/src/test/shell/bazel/external_integration_test.sh
index b0c709b..638d88a 100755
--- a/src/test/shell/bazel/external_integration_test.sh
+++ b/src/test/shell/bazel/external_integration_test.sh
@@ -512,7 +512,7 @@ new_http_archive(
     name = "x",
     url = "http://localhost:$nc_port/x.tar.gz",
     sha256 = "$sha256",
-    rm_path_prefix = "x/y/z",
+    strip_prefix = "x/y/z",
     build_file = "x.BUILD",
 )
 EOF

[kchodorow]

Arg, sorry about this. I changed the name of the attribute at the 11th hour in code review and the test wasn't run by our CI because it's marked as manual. The actual fix should be in Monday, but changing these to strip_prefix should fix it: https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/bazel/repository/NewHttpArchiveFunction.java#L84-L86

[kamalmarhubi]

@kchodorow this was still buggy for zip archives, crashing out with NPE. fixed and submitted for review: https://bazel-review.googlesource.com/#/c/2084/

:-)

[abergmeier]

This does not seem to work for:

new_http_archive(
  name = "z",
  url = "http://zlib.net/zlib-1.2.8.tar.gz",
  skip_prefix = "zlib-1.2.8",
  sha256 = "2a0dd0894c35b8736ff2bee925aab35b473a6c6b432b25e56442bacb0e72bc3a",

)

I get a similar error as @kamalmarhubi

[abergmeier]

And I ran into #520. Not sure, whether it is a variant of these problems...

[kchodorow]

@abergmeier the option is strip_prefix, not skip_prefix (see http://bazel.io/docs/build-encyclopedia.html#new_http_archive.strip_prefix). Regardless, you should not be getting a stack trace, can you paste your output?

[HUTOYP]

i have a svn path, http://xxx.com/xxx/tag/xxx.1.0.1 for example.
how can i do?