Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#674] feat(docker): use JDK11 as the default java version in Dockerfile #683

Merged
merged 1 commit into from
Mar 9, 2023

Conversation

zuston
Copy link
Member

@zuston zuston commented Mar 4, 2023

What changes were proposed in this pull request?

Use JDK11 as the default java version in Dockerfile when deploying shuffle servers on K8s.

Why are the changes needed?

As the GC problems mentioned in #674, I upgraded the JVM version from JDK8 to JDK11 for shuffle server. And after benchmark, I found the effect of the upgrade is remarkable. And it also works well with Hadoop3.1.1

Based on above practice, it's better to use JDK11 as the default java version to improve stability

Does this PR introduce any user-facing change?

Yes

How was this patch tested?

Have tested in production env.

@zuston
Copy link
Member Author

zuston commented Mar 4, 2023

PTAL @advancedxy

@codecov-commenter
Copy link

codecov-commenter commented Mar 4, 2023

Codecov Report

Merging #683 (f0cdc5d) into master (313ddd6) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #683   +/-   ##
=========================================
  Coverage     60.79%   60.79%           
  Complexity     1839     1839           
=========================================
  Files           221      221           
  Lines         12655    12655           
  Branches       1069     1069           
=========================================
  Hits           7694     7694           
  Misses         4553     4553           
  Partials        408      408           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@kaijchen
Copy link
Contributor

kaijchen commented Mar 6, 2023

Does it help when the java version in pom.xml is still 1.8?

@advancedxy
Copy link
Contributor

Does it help when the java version in pom.xml is still 1.8?

GC is part of java runtime, it shouldn't matter whether the code is compiled with Java 1.8 or Java 11.

@zuston
Copy link
Member Author

zuston commented Mar 9, 2023

Could you help review again? JDK11 has been applied in our production env, it works well with hadoop3.1.1 @advancedxy

@advancedxy
Copy link
Contributor

Could you help review again? JDK11 has been applied in our production env, it works well with hadoop3.1.1 @advancedxy

If it could work well with hadoop client. I think this is a good start.

Would you mind create an umbrella issue to tracking all the issues needed to promote using JDK 11 here?
If we are going to promote JDK 11, some of the following work(not exclusive) should be done:

  1. CI workflows to add JDK11 to run all the tests, not just the compile and build only.
  2. related build system/files, this docker file is one for example
  3. Documentation changes.
  4. benchmark result etc.

Copy link
Contributor

@advancedxy advancedxy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code itself LGTM.
However would you mind to modify the RP title to reflect the actual change, since it only changes the dockerfile

Also, would you mind to provide a bit more information about how it is tested.

@zuston zuston changed the title [#674] Use jdk11 as default java version [#674] feat(docker): use JDK11 as the default java version in Dockerfile Mar 9, 2023
@zuston
Copy link
Member Author

zuston commented Mar 9, 2023

The code itself LGTM.
However would you mind to modify the RP title to reflect the actual change, since it only changes the dockerfile

Done.

Also, would you mind to provide a bit more information about how it is tested.

Yes, more detailed info will be exposed in #674

@zuston
Copy link
Member Author

zuston commented Mar 9, 2023

Would you mind create an umbrella issue to tracking all the issues needed to promote using JDK 11 here?
If we are going to promote JDK 11, some of the following work(not exclusive) should be done:

Yes

@zuston zuston merged commit 31e4a3f into apache:master Mar 9, 2023
@zuston zuston deleted the issue-674 branch March 9, 2023 07:35
advancedxy pushed a commit to advancedxy/incubator-uniffle that referenced this pull request Mar 21, 2023
…ockerfile (apache#683)

### What changes were proposed in this pull request?

Use JDK11 as the default java version in Dockerfile when deploying shuffle servers on K8s.

### Why are the changes needed?

As the GC problems mentioned in apache#674, I upgraded the JVM version from JDK8 to JDK11 for shuffle server. And after benchmark, I found the effect of the upgrade is remarkable. And it also works well with Hadoop3.1.1

Based on above practice, it's better to use JDK11 as the default java version to improve stability

### Does this PR introduce _any_ user-facing change?

Yes

### How was this patch tested?

Have tested in production env.

Co-authored-by: zhangjunfan <zhangjunfan@qiyi.com>
xianjingfeng pushed a commit to xianjingfeng/incubator-uniffle that referenced this pull request Apr 5, 2023
…ockerfile (apache#683)

### What changes were proposed in this pull request?

Use JDK11 as the default java version in Dockerfile when deploying shuffle servers on K8s.

### Why are the changes needed?

As the GC problems mentioned in apache#674, I upgraded the JVM version from JDK8 to JDK11 for shuffle server. And after benchmark, I found the effect of the upgrade is remarkable. And it also works well with Hadoop3.1.1

Based on above practice, it's better to use JDK11 as the default java version to improve stability

### Does this PR introduce _any_ user-facing change?

Yes

### How was this patch tested?

Have tested in production env.

Co-authored-by: zhangjunfan <zhangjunfan@qiyi.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants