Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8272124: Cgroup v1 initialization causes NullPointerException when cgroup path contains colon #5127

Closed
wants to merge 4 commits into from

Conversation

hseigel
Copy link
Member

@hseigel hseigel commented Aug 16, 2021

Please review this small fix for JDK-8272124. The fix puts a limit of 3 when splitting self cgroup lines by ':' so that Cgroup paths won't get truncated if they contain embedded ':'s. For example, an entry of "11:memory:/user.sli:ce" in a /proc/self/cgroup file will now result in a Cgroup path of "/user.sli:ce" instead of "/user.sli".

The fix was tested with Mach5 tiers 1 and 2, and Mach5 tiers 3-5 on Linux x64 and Linux aarch64.

Thanks, Harold


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8272124: Cgroup v1 initialization causes NullPointerException when cgroup path contains colon

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/5127/head:pull/5127
$ git checkout pull/5127

Update a local copy of the PR:
$ git checkout pull/5127
$ git pull https://git.openjdk.java.net/jdk pull/5127/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 5127

View PR using the GUI difftool:
$ git pr show -t 5127

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/5127.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Aug 16, 2021

👋 Welcome back hseigel! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Aug 16, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Aug 16, 2021

@hseigel The following labels will be automatically applied to this pull request:

  • core-libs
  • serviceability

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added serviceability core-libs labels Aug 16, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Aug 16, 2021

Webrevs

@hseigel
Copy link
Member Author

@hseigel hseigel commented Aug 16, 2021

/label add runtime

@openjdk
Copy link

@openjdk openjdk bot commented Aug 16, 2021

@hseigel The label runtime is not a valid label. These labels are valid:

  • serviceability
  • hotspot
  • sound
  • hotspot-compiler
  • ide-support
  • kulla
  • i18n
  • shenandoah
  • jdk
  • javadoc
  • security
  • 2d
  • swing
  • hotspot-runtime
  • jmx
  • nio
  • build
  • beans
  • core-libs
  • compiler
  • net
  • hotspot-gc
  • hotspot-jfr
  • awt

@hseigel
Copy link
Member Author

@hseigel hseigel commented Aug 16, 2021

/label add hotspot-runtime

@openjdk openjdk bot added the hotspot-runtime label Aug 16, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Aug 16, 2021

@hseigel
The hotspot-runtime label was successfully added.

Copy link
Member

@mseledts mseledts left a comment

Changes look good to me.

@@ -172,7 +172,7 @@
private String mntInfoCgroupv1MoreCpusetLine = "121 32 0:37 / /cpuset rw,relatime shared:69 - cgroup none rw,cpuset\n";
private String mntInfoCgroupsV1DoubleCpuset = mntInfoHybrid + mntInfoCgroupv1MoreCpusetLine;
private String mntInfoCgroupsV1DoubleCpuset2 = mntInfoCgroupv1MoreCpusetLine + mntInfoHybrid;
private String cgroupv1SelfCgroupContent = "11:memory:/user.slice/user-1000.slice/user@1000.service\n" +
private String cgroupv1SelfCgroupContent = "11:memory:/user.slice/user-1000.slice/user@1000.s:ervice\n" +
Copy link
Contributor

@jerboaa jerboaa Aug 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add a comment explaining that /proc/self/cgroup is supposed to contain three colon-separated fields, hierarchy-ID:controller-list:cgroup-path. Thus, this cgroup-path intentionally contains a colon so as to ensure the correct path is being extracted by the logic in CgroupSubsystemFactory.

Better yet, add a new test with mountinfo and /proc/self/cgroup contents observed in the wild and assert that memoryInfo.getCgroupPath() equals memoryInfo.getMountRoot()

@@ -196,7 +196,7 @@ static CgroupMetrics create() {
if (isCgroupsV2) {
action = (tokens -> setCgroupV2Path(infos, tokens));
}
selfCgroupLines.map(line -> line.split(":"))
selfCgroupLines.map(line -> line.split(":", 3))
.filter(tokens -> (tokens.length >= 3))
Copy link
Contributor

@jerboaa jerboaa Aug 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This filter no longer makes sense as tokens.length == 3 in every case after this patch. Please remove. From the javadoc from String.split(String, limit):

If the limit is positive then the pattern will be applied
at most limit-1 times, the array's length will be
no greater than limit, and the array's last entry will contain
all input beyond the last matched delimiter.

Copy link
Contributor

@jerboaa jerboaa Aug 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, we might want to keep tokens.length == 3, but I don't know if it'll ever be < 3 in reality. It certainly would no longer be > 3.

@hseigel
Copy link
Member Author

@hseigel hseigel commented Aug 17, 2021

Thanks Misha and Severin for looking at this change!

Please review this updated commit that tries to address Severin's comments. A new test case was added to TestCgroupSubsystemFactory.java for the multiple ':'s case and comments were added to CgroupSubsystemFactory.java.

The ".filter(tokens -> (tokens.length >= 3))" code was removed but can be restored if need be.

Thanks, Harold

CgroupTypeResult res = result.get();
CgroupInfo memoryInfo = res.getInfos().get("memory");
assertEquals(memoryInfo.getCgroupPath(), "/system.slice/containerd.service/kubepods-burstable-podf65e797d_d5f9_4604_9773_94f4bb9946a0.slice:cri-containerd:86ac6260f9f8a9c1276748250f330ae9c2fcefe5ae809364ad1e45f3edf7e08a");
assertEquals(memoryInfo.getMountRoot(), memoryInfo.getMountRoot());
Copy link
Contributor

@jerboaa jerboaa Aug 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant here is to also change the mountinfo entry for memory and then use

assertEquals(memoryInfo.getMountRoot(), memoryInfo.getCgroupPath());

as that's really what was happening in the wild. mount root - containing colons - would correctly parse, but the cgroup path - containing colons - would not and the assertion would fail (without the change in CgroupSubsystemFactory). Thanks!

Copy link
Contributor

@jerboaa jerboaa Aug 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hseigel it still has assertEquals(memoryInfo.getMountRoot(), memoryInfo.getMountRoot()); - tautology - should be: assertEquals(memoryInfo.getMountRoot(), memoryInfo.getCgroupPath());

@hseigel
Copy link
Member Author

@hseigel hseigel commented Aug 18, 2021

Please review this update that modifies the new test case to have a mountinfo entry that contains multiple ":" characters.
Thanks, Harold

@hseigel hseigel changed the title 8272124: Cgroup v1 initialization causes NullPointerException when path contains colon 8272124: Cgroup v1 initialization causes NullPointerException when cgroup path contains colon Aug 18, 2021
// `/proc/self/cgroup` should contain **three** colon-separated fields,
// `hierarchy-ID:controller-list:cgroup-path`. This cgroup-path intentionally
// contains a colon to ensure that the correct path is being extracted by the
// login in CgroupSubsystemFactory.
Copy link
Contributor

@jerboaa jerboaa Aug 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/login/logic

CgroupTypeResult res = result.get();
CgroupInfo memoryInfo = res.getInfos().get("memory");
assertEquals(memoryInfo.getCgroupPath(), "/system.slice/containerd.service/kubepods-burstable-podf65e797d_d5f9_4604_9773_94f4bb9946a0.slice:cri-containerd:86ac6260f9f8a9c1276748250f330ae9c2fcefe5ae809364ad1e45f3edf7e08a");
assertEquals(memoryInfo.getMountRoot(), memoryInfo.getMountRoot());
Copy link
Contributor

@jerboaa jerboaa Aug 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hseigel it still has assertEquals(memoryInfo.getMountRoot(), memoryInfo.getMountRoot()); - tautology - should be: assertEquals(memoryInfo.getMountRoot(), memoryInfo.getCgroupPath());

@hseigel
Copy link
Member Author

@hseigel hseigel commented Aug 18, 2021

Thanks for finding those issues. Please review the latest iteration of this fix.
Thanks, Harold

Copy link
Contributor

@jerboaa jerboaa left a comment

Looks good. Thank you!

@openjdk
Copy link

@openjdk openjdk bot commented Aug 18, 2021

@hseigel This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8272124: Cgroup v1 initialization causes NullPointerException when cgroup path contains colon

Reviewed-by: mseledtsov, sgehwolf

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 24 new commits pushed to the master branch:

  • 30b0f82: 8272626: Avoid C-style array declarations in java.*
  • e8f1219: 8271276: C2: Wrong JVM state used for receiver null check
  • 79a06df: 8272567: [IR Framework] Make AbstractInfo.getRandom() static
  • 481c1f0: 8269951: [macos] Focus not painted in JButton when setBorderPainted(false) is invoked
  • 5189047: 8272558: IR Test Framework README misses some flags
  • ec63957: 8272398: Update DockerTestUtils.buildJdkDockerImage()
  • 14623cd: 8270835: regression after JDK-8261006
  • fe72197: 8272551: mark hotspot runtime/modules tests which ignore external VM flags
  • 05d64da: 8272291: mark hotspot runtime/logging tests which ignore external VM flags
  • a68b5b9: 8272369: java/io/File/GetXSpace.java failed with "RuntimeException: java.nio.file.NoSuchFileException: /run/user/0"
  • ... and 14 more: https://git.openjdk.java.net/jdk/compare/b2c272d4e21eb4aeb6c8e1c0be0d08943dc36db8...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Aug 18, 2021
@hseigel
Copy link
Member Author

@hseigel hseigel commented Aug 18, 2021

Thanks Severin and Misha for your reviews and improvements for this fix!

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Aug 18, 2021

Going to push as commit 4d6593c.
Since your change was applied there have been 24 commits pushed to the master branch:

  • 30b0f82: 8272626: Avoid C-style array declarations in java.*
  • e8f1219: 8271276: C2: Wrong JVM state used for receiver null check
  • 79a06df: 8272567: [IR Framework] Make AbstractInfo.getRandom() static
  • 481c1f0: 8269951: [macos] Focus not painted in JButton when setBorderPainted(false) is invoked
  • 5189047: 8272558: IR Test Framework README misses some flags
  • ec63957: 8272398: Update DockerTestUtils.buildJdkDockerImage()
  • 14623cd: 8270835: regression after JDK-8261006
  • fe72197: 8272551: mark hotspot runtime/modules tests which ignore external VM flags
  • 05d64da: 8272291: mark hotspot runtime/logging tests which ignore external VM flags
  • a68b5b9: 8272369: java/io/File/GetXSpace.java failed with "RuntimeException: java.nio.file.NoSuchFileException: /run/user/0"
  • ... and 14 more: https://git.openjdk.java.net/jdk/compare/b2c272d4e21eb4aeb6c8e1c0be0d08943dc36db8...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Aug 18, 2021
@openjdk openjdk bot added the integrated label Aug 18, 2021
@openjdk openjdk bot removed ready rfr labels Aug 18, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Aug 18, 2021

@hseigel Pushed as commit 4d6593c.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-libs hotspot-runtime integrated serviceability
3 participants