Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

starlark impl of java_common.pack_sources generates invalid srcjar when inputs come from external repository #20882

Closed
rbeasley opened this issue Jan 12, 2024 · 4 comments
Assignees
Labels
P1 I'll work on this now. (Assignee required) team-Rules-Java Issues for Java rules type: bug

Comments

@rbeasley
Copy link
Contributor

rbeasley commented Jan 12, 2024

Description of the bug:

Under Bazel 7, when feeding sources from an external repository to java_common.pack_sources, the sources are mapped to illegal paths beginning with ../. For example, if we pass @foo//:dummy.txt to pack_sources, we'll end up with a jar containing ../foo/dummy.txt.

Which category does this issue belong to?

Java Rules

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

## Set up main repo.
cat >pack_sources.bzl <<"EOF"
def _impl(ctx):
  java_common.pack_sources(
    ctx.actions,
    output_source_jar = ctx.outputs.out,
    sources = ctx.files.srcs,
    source_jars = [
      source_jar
      for dep in ctx.attr.deps
      for source_jar in dep[JavaInfo].source_jars
    ],
    java_toolchain = ctx.toolchains["@bazel_tools//tools/jdk:toolchain_type"].java,
  )

pack_sources = rule(
  _impl,
  attrs = {
    "deps": attr.label_list(providers=[JavaInfo]),
    "srcs": attr.label_list(allow_files = True),
  },
  outputs = {"out": "%{name}.srcjar"},
  toolchains = ["@bazel_tools//tools/jdk:toolchain_type"],
)
EOF

touch WORKSPACE

touch foo bar
cat >BUILD <<"EOF"
package(default_visibility=["//visibility:public"])
filegroup(name="srcs", srcs=["foo", "bar"])
EOF


## Set up subproject
mkdir -p subproject

cat >subproject/WORKSPACE <<"EOF"
local_repository(
  name = "superproject",
  path = "..",
)
EOF

cat >subproject/BUILD.bazel <<"EOF"
load("@superproject//:pack_sources.bzl", "pack_sources")

pack_sources(
  name = "packed",
  srcs = ["@superproject//:srcs"],
)

sh_test(
  name = "test_packed",
  srcs = ["test_packed.sh"],
  data = [":packed"],
)
EOF

cat >subproject/test_packed.sh <<"EOF"
zipinfo -1 $(find -L -name packed.srcjar) | grep -q ^foo
EOF
chmod +x subproject/test_packed.sh

## Build the srcjar and verify that foo is in the toplevel directory.
cd subproject

USE_BAZEL_VERSION=6.0.0 bazelisk test --noenable_bzlmod --java_runtime_version=remotejdk_11 :test_packed

USE_BAZEL_VERSION=7.0.0 bazelisk test --noenable_bzlmod --java_runtime_version=remotejdk_11 :test_packed

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 7.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

Yes. 6da7f21

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

I suspect that the Java and Starlark implementations aren't quite the same:

def _resource_mapper(file):
return "%s:%s" % (
file.path,
semantics.get_default_resource_path(file.short_path, segment_extractor = _java_segments),
)
public void expandToCommandLine(Artifact resource, Consumer<String> args) {
String execPath = resource.getExecPathString();
String resourcePath =
semantics.getDefaultJavaResourcePath(resource.getRootRelativePath()).getPathString();
StringBuilder sb = new StringBuilder(execPath.length() + resourcePath.length() + 1);
sb.append(execPath).append(":").append(resourcePath);
args.accept(sb.toString());
}

Compare resource.getRootRelativePath() to file.short_path.

IIUC, then based on the table at https://github.com/bazelbuild/rules_pkg/blob/main/pkg/mappings.bzl#L166-L177, instead of using file.short_path it should instead be paths.relativize(file.path, paths.normalize(paths.join(file.root.path, file.owner.workspace_root))).

@fmeum
Copy link
Collaborator

fmeum commented Jan 12, 2024

@bazel-io fork 7.0.1

rbeasley added a commit to rbeasley/bazel that referenced this issue Jan 13, 2024
This commit tweaks the Starlark implementation of `java_common.pack_sources`
to behave more like its original Java counterpart.  Specifically, we don't
use `file.short_path`, because when `file` comes from an external repository,
`short_path` will have a `../<externalrepo>` prefix.

Resolves bazelbuild#20882.
@rbeasley rbeasley changed the title starlark impl of java_common.pack_sources generates invalid srcjar when inputs come from separate repository starlark impl of java_common.pack_sources generates invalid srcjar when inputs come from external repository Jan 13, 2024
@meteorcloudy
Copy link
Member

/cc @hvadehra

@hvadehra hvadehra added P1 I'll work on this now. (Assignee required) and removed untriaged labels Jan 15, 2024
@hvadehra hvadehra added the team-Rules-Java Issues for Java rules label Jan 15, 2024
bazel-io pushed a commit to bazel-io/bazel that referenced this issue Jan 15, 2024
Fixes bazelbuild#20882

PiperOrigin-RevId: 598608807
Change-Id: If9230cd1c143eef09beabaf33cb1cb56e4540ae1
meteorcloudy pushed a commit that referenced this issue Jan 16, 2024
)

Fixes #20882

Commit
6ae53e5

PiperOrigin-RevId: 598608807
Change-Id: If9230cd1c143eef09beabaf33cb1cb56e4540ae1

---------

Co-authored-by: Googler <hvd@google.com>
Co-authored-by: hvd <hvadehra@gmail.com>
@iancha1992
Copy link
Member

A fix for this issue has been included in Bazel 7.0.1 RC2. Please test out the release candidate and report any issues as soon as possible. Thanks!

@iancha1992
Copy link
Member

@bazel-io fork 7.1.0

bazel-io pushed a commit to bazel-io/bazel that referenced this issue Jan 22, 2024
Fixes bazelbuild#20882

PiperOrigin-RevId: 598608807
Change-Id: If9230cd1c143eef09beabaf33cb1cb56e4540ae1
github-merge-queue bot pushed a commit that referenced this issue Jan 23, 2024
)

Fixes #20882

Commit
6ae53e5

PiperOrigin-RevId: 598608807
Change-Id: If9230cd1c143eef09beabaf33cb1cb56e4540ae1

---------

Co-authored-by: Googler <hvd@google.com>
Co-authored-by: keertk <keerthanakumar@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 I'll work on this now. (Assignee required) team-Rules-Java Issues for Java rules type: bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants