Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add embedded jdk support for ppc64le #11436

Closed
wants to merge 5 commits into from

Conversation

jayfurmanek
Copy link
Contributor

This adds support for embedding OpenJDK in bazel for ppc64le.
Azul Systems does not create Zulu/OpenJDK builds for ppc64le, but AdoptOpenjdk.net does.

This PR adds the necessary configs and conditionals and http_file configuration URLs for adoptopenjdk downloads.
The minimize_jdk.sh script needed a slight generalization.

I've been using adoptOpenjdk-embedded bazel for quite a while and can vouch for its stability.

@aiuto aiuto added the team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website label May 19, 2020
@aiuto aiuto requested a review from philwo May 19, 2020 02:17
Copy link
Member

@philwo philwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really cool! Thank you :)

The tests fail, but the fix is simple (there's just a wrong URL in the jdk.WORKSPACE).

@philwo philwo requested a review from meisterT May 19, 2020 12:31
@philwo
Copy link
Member

philwo commented May 19, 2020

FYI @meisterT

@meisterT
Copy link
Member

Nice!

@jayfurmanek
Copy link
Contributor Author

jayfurmanek commented May 19, 2020

I think the remaining fails here are unrelated.

Also, what's the policy on compile.sh changes? In our builds, we add in the below to the bazel build command:

--distinct_host_configuration=false \
--javabase=@bazel_tools//tools/jdk:jdk \

It looks like it doesn't build in the jdk by default. Is the idea that you should do that kind of thing with a bazelrc file?

Thanks for reviewing!

@philwo
Copy link
Member

philwo commented May 19, 2020

--distinct_host_configuration=false
--javabase=@bazel_tools//tools/jdk:jdk \

To be honest, I'm not sure what these flags do. 😬 Why do you need them for your build?

The test failures are indeed related to this change:

ERROR: An error occurred during the fetch of repository 'remotejdk11_linux_ppc64le':
   java.io.IOException: Error downloading [https://mirror.bazel.com/openjdk/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.7%2B10/OpenJDK11U-jdk_ppc64le_linux_hotspot_11.0.7_10.tar.gz, https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.7%2B10/OpenJDK11U-jdk_ppc64le_linux_hotspot_11.0.7_10.tar.gz] to /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/sandbox/linux-sandbox/3385/execroot/io_bazel/_tmp/c5e052945f2150d3644bf8016601089b/root/fc065a228971d6bda964c6ba833756dc/external/remotejdk11_linux_ppc64le/OpenJDK11U-jdk_ppc64le_linux_hotspot_11.0.7_10.tar.gz: Unknown host: github.com

They happen because we run most tests without network access on our CI and instead inject the external repositories (like JDKs) from the outside. It should work if you copy the definition of your remotejdk11_linux_ppc64le repository into the top-level WORKSPACE file and suffix it with _for_testing, similar to this one: https://cs.opensource.google/bazel/bazel/+/master:WORKSPACE;l=674;drc=762c27d2cdc8d66cc4f91a141ec667150e8a5f39

Then add your remotejdk11_linux_ppc64le_for_testing repository to these lists similar to the existing ones:

That should fix it. :)

@jayfurmanek
Copy link
Contributor Author

That worked. Thanks for the assistance @philwo !

@jayfurmanek
Copy link
Contributor Author

Anything else needed here? Thx!!

@philwo
Copy link
Member

philwo commented May 28, 2020

@jayfurmanek I think we're good, thanks! I'll try to merge this now.

@bazel-io bazel-io closed this in 04e073e May 28, 2020
ruixin-bao pushed a commit to linux-on-ibm-z/bazel that referenced this pull request Jul 27, 2020
This patch follows a similar approach used in PR
bazelbuild#11436 to add support
for embedding OpenJDK in bazel for s390x.

Similar to ppc, the openjdk used will be downloadded from adoptopenjdk website.

Another thing this PR does is to also provide the cached version of
allmodules_jdk and minimal_jdk zip files built on s390x. I have
tested by linking this file locally on s390x machines and with these zip files provided,
tests no longer fail with java exec format error on s390x.
ruixin-bao pushed a commit to linux-on-ibm-z/bazel that referenced this pull request Jul 27, 2020
This patch follows a similar approach used in PR
bazelbuild#11436 to add support
for embedding OpenJDK in bazel on s390x.

Similar to ppc, the openjdk used will be downloadded from adoptopenjdk website.

Another thing this PR does is to provide the cached version of
allmodules_jdk and minimal_jdk zip files built on s390x. I have
tested by linking the files locally on s390x and with these zip files provided,
tests no longer fail with java exec format error on s390x.
ruixin-bao pushed a commit to linux-on-ibm-z/bazel that referenced this pull request Jul 27, 2020
This patch follows a similar approach used in PR
bazelbuild#11436 to add support
for embedding OpenJDK in bazel on s390x.

Similar to ppc, the openjdk used will be downloadded from adoptopenjdk website.

Another thing this PR does is to provide the cached version of
allmodules_jdk and minimal_jdk zip files built on s390x. I have
tested by linking the files locally on s390x and with these zip files provided,
tests no longer fail with java exec format error on s390x.
ruixin-bao pushed a commit to linux-on-ibm-z/bazel that referenced this pull request Aug 14, 2020
This patch follows a similar approach used in PR
bazelbuild#11436 to add support
for embedding OpenJDK in bazel on s390x.

Similar to ppc, the openjdk used will be downloadded from adoptopenjdk website.
ruixin-bao pushed a commit to linux-on-ibm-z/bazel that referenced this pull request Aug 14, 2020
This patch follows a similar approach used in PR
bazelbuild#11436 to add support
for embedding OpenJDK in bazel on s390x.

Similar to ppc, the openjdk used will be downloadded from adoptopenjdk website.
ruixin-bao pushed a commit to linux-on-ibm-z/bazel that referenced this pull request Aug 18, 2020
This patch follows a similar approach used in PR
bazelbuild#11436 to add support
for embedding OpenJDK in bazel on s390x.

Similar to ppc, the openjdk used will be downloadded from adoptopenjdk website.
bazel-io pushed a commit that referenced this pull request Aug 21, 2020
This patch follows a similar approach used in PR
#11436 to add support
for embedding OpenJDK in bazel on s390x.

Similar to ppc, the openjdk used will be downloadded from adoptopenjdk website.

/cc @philwo

Closes #11965.

PiperOrigin-RevId: 327798995
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants