8292206: TestCgroupMetrics.java fails as getMemoryUsage() is lower than expected

[iklam]

The testMemoryUsage() test scenario queries the total memory usage of all processes of the current Linux user, including other concurrently running jtreg test cases. Even if the current process allocates 256MB of ram, it's possible for other dying processes to release much more than that amount. Therefore, it's not possible to guarantee that Metrics.getMemoryUsage() would return a higher number.

I am removing this test scenario for now as I don't see it providing any actual value.

If we want to have more in-depth functional tests for the Metrics.getMemoryXXX() APIs, we need to do it inside a container in a more controlled setting.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8292206: TestCgroupMetrics.java fails as getMemoryUsage() is lower than expected

Reviewers

• David Holmes (@dholmes-ora - Reviewer) ⚠️ Review applies to 8f27544b
• Severin Gehwolf (@jerboaa - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/11734/head:pull/11734
$ git checkout pull/11734

Update a local copy of the PR:
$ git checkout pull/11734
$ git pull https://git.openjdk.org/jdk pull/11734/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 11734

View PR using the GUI difftool:
$ git pr show -t 11734

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/11734.diff

[bridgekeeper]

👋 Welcome back iklam! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@iklam The following label will be automatically applied to this pull request:

• hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full - Incremental (8c1ca8b4)
• 00: Full (8f27544b)

[iklam]

Note: theoretically other test scenarios can also fail for the same reason, but we've never seen them fail in our CI pipeline

https://github.com/openjdk/jdk/blob/ba942c24e8894f4422870fb53253f5946dc4f0d1/test/lib/jdk/test/lib/containers/cgroup/MetricsTesterCgroupV1.java (see lines L250-L254)

        oldVal = metrics.getKernelMemoryMaxUsage();
        newVal = getLongValueFromFile(Controller.MEMORY, "memory.kmem.max_usage_in_bytes");
        if (!CgroupMetricsTester.compareWithErrorMargin(oldVal, newVal)) {
            fail(Controller.MEMORY, "memory.kmem.max_usage_in_bytes", oldVal, newVal);
        }

compareWithErrorMargin checks that the difference is less than 25%, which probably will never happen between the first two lines in the code quoted above. Other usage of compareWithErrorMargin in the same test may suffer the same problem.

[dholmes-ora]

Removing this seems reasonable. Thanks for the explanation.

[openjdk]

@iklam This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8292206: TestCgroupMetrics.java fails as getMemoryUsage() is lower than expected

Reviewed-by: dholmes, sgehwolf

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 63 new commits pushed to the master branch:

• 5012039: 8298887: On the latest macOS+XCode the Robot API may report wrong colors
• 34cdda5: Merge
• 22007a1: 8298893: Rename option UsePolyIntrinsics to UsePoly1305Intrinsics
• 9adc349: 8298726: (fs) Change PollingWatchService to record last modified time as FileTime rather than milliseconds
• 81933b7: 8298642: ParallelGC -XX:+UseNUMA eden spaces allocated on wrong node
• 92fe304: 8298588: WebSockets: HandshakeUrlEncodingTest unnecessarily depends on a response body
• f7be5b5: 8299156: Broken link in jdk.compiler/module-info.java
• 3d4d9fd: 8298947: compiler/codecache/MHIntrinsicAllocFailureTest.java fails intermittently
• e85d00f: 8299147: Minor accessibility errors in the specs and man index pages
• f80face: 8202931: [macos] java/awt/Choice/ChoicePopupLocation/ChoicePopupLocation.java fails
• ... and 53 more: https://git.openjdk.org/jdk/compare/ba942c24e8894f4422870fb53253f5946dc4f0d1...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[jerboaa]

I am removing this test scenario for now as I don't see it providing any actual value.

I disagree. It seems useful in the docker/container case as there are APIs in the JDK reporting those metrics and it wouldn't be tested otherwise.

Note that this test is also being used by TestSystemMetrics.java. Entrypoint in the TestSystemMetrics case is main() (over entrypoint testAll() via TestCgroupMetrics).

If we want to have more in-depth functional tests for the Metrics.getMemoryXXX() APIs, we need to do it inside a container in a more controlled setting.

That's what the above mentioned TestSystemMetrics.java does. I take it that test is not problematic? If so, we should just not invoke testMemoryUsage() when invoked from TestCgroupMetrics.

[iklam]

I am removing this test scenario for now as I don't see it providing any actual value.

I disagree. It seems useful in the docker/container case as there are APIs in the JDK reporting those metrics and it wouldn't be tested otherwise.

Note that this test is also being used by TestSystemMetrics.java. Entrypoint in the TestSystemMetrics case is main() (over entrypoint testAll() via TestCgroupMetrics).

If we want to have more in-depth functional tests for the Metrics.getMemoryXXX() APIs, we need to do it inside a container in a more controlled setting.

That's what the above mentioned TestSystemMetrics.java does. I take it that test is not problematic? If so, we should just not invoke testMemoryUsage() when invoked from TestCgroupMetrics.

Thanks for your review. I've disabled the testMemorySubsystem() and testMemoryUsage() scenarios only when testing outside of containers (in TestCgroupMetrics.java). These scenarios will be executed by TestSystemMetrics.java, which is executed in a container.

[jerboaa]

Looks good, thanks!

[iklam]

Thanks @jerboaa and @dholmes-ora for the review.

Tested with tier5 container tests in our CI.

/integrate

[openjdk]

Going to push as commit 6ccee83.
Since your change was applied there have been 66 commits pushed to the master branch:

• b378381: 8299199: Avoid redundant split calls in FontConfiguration.initReorderMap implementations
• 62a033e: 8299191: Unnecessarily global friend functions for relocInfo
• a3693cc: 8295087: Manual Test to Automated Test Conversion
• 5012039: 8298887: On the latest macOS+XCode the Robot API may report wrong colors
• 34cdda5: Merge
• 22007a1: 8298893: Rename option UsePolyIntrinsics to UsePoly1305Intrinsics
• 9adc349: 8298726: (fs) Change PollingWatchService to record last modified time as FileTime rather than milliseconds
• 81933b7: 8298642: ParallelGC -XX:+UseNUMA eden spaces allocated on wrong node
• 92fe304: 8298588: WebSockets: HandshakeUrlEncodingTest unnecessarily depends on a response body
• f7be5b5: 8299156: Broken link in jdk.compiler/module-info.java
• ... and 56 more: https://git.openjdk.org/jdk/compare/ba942c24e8894f4422870fb53253f5946dc4f0d1...master

Your commit was automatically rebased without conflicts.

[openjdk]

@iklam Pushed as commit 6ccee83.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.