Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate pseudo-compile jars for bazel java compilation #518

Merged
merged 3 commits into from
Mar 1, 2021

Conversation

shs96c
Copy link
Collaborator

@shs96c shs96c commented Feb 10, 2021

Normally, bazel uses ijar to prepare a jar containing just
ABI-entries for classfiles. These are stable when non-API breaking
changes are made, and so allow bazel to compile code faster, and are
known as "compile jars"

Because of bazelbuild/bazel#4549
rules_jvm_external doesn't use ijar, and instead uses the
downloaded jar as the "compile jar". Normally, this is fine, but Bazel
4.0.0 now sets -Xlint:path for javac invocations, and this means
that if there's a Class-Path manifest entry in a compile jar, the
jars within that will be checked. This can lead to noisy builds:
bazelbuild/bazel#12968

This change generates a "pseudo-compile jar" for jvm_import
targets. All it does is repack the input jar, excluding the
META-INF/MANIFEST.MF file. This means that we should avoid
compilation problems whilst still working well with Kotlin projects.

@cheister
Copy link
Collaborator

I feel like just deleting the entire manifest.mf file will have other follow on problems for the jars that might set things like Multi-Release to true or other jar meta info.

What about just deleting the Class-Path attributed from the manifest.mf file if it exists? Seems like we could even adapt AddJarManifestEntry to delete the entry if it's there?

@shs96c
Copy link
Collaborator Author

shs96c commented Feb 12, 2021

I'd not thought about the case where we'd be dealing with a multi-release jar. Good catch.

I've reworked the PR to use AddJarManifestEntry and made some changes to facilitate that. I've reworked the new test to use the AddJarManifestEntry but left it otherwise unchanged from the original version.

@cheister
Copy link
Collaborator

Looks good. I wonder if we should just combine the part where we add the Target-Label with the part that removes the Class-Path attribute since we'll now be modifying the jar always.

The other condition we check for in Target-Label case is if the the jar is signed then we don't modify the manifest because it will make the signature invalid. We probably want that for the Class-Path attribute as well so seems like we should just combine the two?

@shs96c
Copy link
Collaborator Author

shs96c commented Feb 17, 2021

The compilejar is only used for compilation, and the runtime checks aren't done on it. As long as it contains the right class files in the right place, everything should be fine.

I may modify AddJarManifestEntry to handle the checks for us, so we can unify the logic into one place.

@shs96c shs96c force-pushed the compile-jar branch 2 times, most recently from da73035 to c74550f Compare February 25, 2021 14:47
outputs = [compilejar],
)

compilejar = ctx.actions.declare_file("header_" + injar.basename, sibling = injar)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing this wasn't meant to be duplicated?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooops. Fixed :)

inputs = [injar] + ctx.files._add_jar_manifest_entry,
args = ctx.actions.args()
args.add_all(["--source", injar, "--output", outjar])
args.add_all(["--manifest-entry", "Target-Label:{target_label}".format(target_label = ctx.label)])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be combined with the call below so we only call add_jar_manifest_entry once?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Desirable as that may be, the stamped jar and the compile jar are used for different purposes. The output jar will be supplied as a transitive dependency and is included in the list of runtime jars.

The compile jar is just used by javac for compilation. Normally, bazel uses ijar to generate this, and that strips out the bodies of methods, leaving only the ABI in place. For this case, we could remove everything that wasn't the class files, and reduce the manifest down to indicate whether or not this is a multi-release jar. In previous reviews, you've asked for the manifest to be left intact, which is done, other than stripping the Class-Path, since that's the thing that causes bazel to the bad path element warnings this PR was designed to remove :)

# the lint check will emit a `bad path element` warning. We get quiet and
# correct builds if we remove the `Class-Path` manifest entry entirely.
args.add_all(["--remove-entry", "Class-Path"])
ctx.actions.run(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also add a mnemonic and progress message?

Normally, bazel uses `ijar` to prepare a jar containing just
ABI-entries for classfiles. These are stable when non-API breaking
changes are made, and so allow bazel to compile code faster, and are
known as "compile jars"

Because of bazelbuild/bazel#4549
`rules_jvm_external` doesn't use `ijar`, and instead uses the
downloaded jar as the "compile jar". Normally, this is fine, but Bazel
4.0.0 now sets `-Xlint:path` for javac invocations, and this means
that if there's a `Class-Path` manifest entry in a compile jar, the
jars within _that_ will be checked. This can lead to noisy builds:
bazelbuild/bazel#12968

This change generates a "pseudo-compile jar" for `jvm_import`
targets. All it does is repack the input jar, excluding the
`META-INF/MANIFEST.MF` file. This means that we should avoid
compilation problems whilst still working well with Kotlin projects.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants