8292984: Refactor internal container-related interfaces for clarity #27470

caspernorrbin · 2025-09-24T13:55:35Z

Hi everyone,

The current memory-related code paths in Linux are unclear and convoluted, with responsibilities and data flow crossing between os::Linux and various container-related layers.

For example, consider the call sequence for os::available_memory():

os::available_memory()
        |
        v
os::Linux::available_memory()
        |--------------------------------------------
        v                                           v    
OSContainer::memory_limit_in_bytes()       or return host physical memory
        |
        v
CgroupSubsystem::memory_limit_in_bytes()
        |--------------------------------------------
        v                                           v
return os::Linux::physical_memory()        or return cgroup v1/v2 limit

This structure is difficult to follow. Calls move between os::Linux and container subsystems in a confusing manner. Ideally, each component should be responsible only for its relevant functionality:

os::Linux should focus solely on actual machine memory values.
CgroupSubsystem should focus exclusively on cgroup memory limits.
The selection of which value to use should occur at the os layer, based on whether the environment is containerized.

A revised structure separates these responsibilities:

os::available_memory()
        |--------------------------------------------
        v                                           v
OSContainer::memory_limit_in_bytes()       or os::Linux::available_memory()                       
        |--------------------------------------------
        v                                           v
CgroupSubsystem::memory_limit_in_bytes()   os::Linux::physical_memory()
        |
        v
return bounded cgroup v1/v2 limit

With these changes:

os::Linux only retrieves machine values.
CgroupSubsystem works exclusively with cgroup limits.
OSContainer fetches and passes bounds for the cgroup values.
The decision of container or machine value is done in the os layer.

The concrete code changes include:

Moving container selection logic from os::Linux::{available/free}_memory() to os:{available/free}_memory(), so os::Linux now only deals with machine values (was already the case for os::physical_memory()).
Moving os::Linux::available_memory_in_container() to OSContainer instead, removing container-specific logic from os::Linux. Also refactored to use the new bool and reference interface introduced in JDK-8357086.
Moving accessing host values from CgroupSubsystem to OSContainer, and abstracting CgroupSubsystem to use more generic limits (e.g. upper_mem_bound) instead of a direct system reference (e.g. host_mem).

Note: I intentionally kept the julong parameter types unchanged. I believe it's better to update all types simultaneously in JDK-8365606 instead to ensure the change is complete and consistent.

Testing:

Oracle tiers 1-3.
Container tests on cgroup v1 and v2 systems.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

JDK-8292984: Refactor internal container-related interfaces for clarity (Enhancement - P4)

Reviewers

Severin Gehwolf (@jerboaa - Reviewer)
Erik Österlund (@fisk - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/27470/head:pull/27470
$ git checkout pull/27470

Update a local copy of the PR:
$ git checkout pull/27470
$ git pull https://git.openjdk.org/jdk.git pull/27470/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 27470

View PR using the GUI difftool:
$ git pr show -t 27470

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/27470.diff

Using Webrev

Link to Webrev Comment

bridgekeeper · 2025-09-24T13:58:06Z

👋 Welcome back cnorrbin! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2025-09-24T13:59:46Z

@caspernorrbin This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8292984: Refactor internal container-related interfaces for clarity

Reviewed-by: sgehwolf, eosterlund

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 39 new commits pushed to the master branch:

2c7f738: 8368825: Use switch expression for DateTimeFormatterBuilder pattern character lookup
56baf64: 8368520: TLS 1.3 KeyUpdate fails with SunPKCS11 provider
cc563c8: 8368866: compiler/codecache/stress/UnexpectedDeoptimizationTest.java intermittent timed out
... and 36 more: https://git.openjdk.org/jdk/compare/2746c1a555891564963299182b3b0293eaefc901...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2025-09-24T14:00:46Z

@caspernorrbin The following label will be automatically applied to this pull request:

hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2025-09-24T14:04:36Z

Webrevs

01: Full (865c2b4f)
00: Full (6c8d5a09)

jerboaa

Looks good to me, but should probably wait until JDK-8367485 is integrated.

jerboaa · 2025-09-25T12:03:20Z

/reviewers 2

openjdk · 2025-09-25T12:04:34Z

@jerboaa
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

caspernorrbin · 2025-09-30T08:39:50Z

Looks good to me, but should probably wait until JDK-8367485 is integrated.

Thank you for reviewing! I've now merged those changes in.

jerboaa · 2025-09-30T09:46:48Z

src/hotspot/os/linux/cgroupSubsystem_linux.cpp

-  julong phys_mem = static_cast<julong>(os::Linux::physical_memory());
-  log_trace(os, container)("total physical memory: " JULONG_FORMAT, phys_mem);
-  jlong mem_limit = contrl->controller()->read_memory_limit_in_bytes(phys_mem);
+  jlong mem_limit = contrl->controller()->read_memory_limit_in_bytes(upper_bound);


This removes the logging at the trace level for the upper bound. Intentional?

Yes, it was intentional. Both cgroup v1/v2 already log the upper bound in read_memory_limit_in_bytes() at debug level, along with more information. It felt unnecessary logging the same value twice.

jerboaa

Looks fine.

fisk

Looks good.

caspernorrbin · 2025-10-02T13:37:48Z

Thank you for the reviews!

/integrate

openjdk · 2025-10-02T13:38:44Z

Going to push as commit 5252262.
Since your change was applied there have been 39 commits pushed to the master branch:

2c7f738: 8368825: Use switch expression for DateTimeFormatterBuilder pattern character lookup
56baf64: 8368520: TLS 1.3 KeyUpdate fails with SunPKCS11 provider
cc563c8: 8368866: compiler/codecache/stress/UnexpectedDeoptimizationTest.java intermittent timed out
... and 36 more: https://git.openjdk.org/jdk/compare/2746c1a555891564963299182b3b0293eaefc901...master

Your commit was automatically rebased without conflicts.

openjdk · 2025-10-02T13:38:54Z

@caspernorrbin Pushed as commit 5252262.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

caspernorrbin added 2 commits September 24, 2025 13:40

keep os::linux separate

a91b132

keep cgroupsubsystem separate

6c8d5a0

openjdk bot added the hotspot-runtime hotspot-runtime-dev@openjdk.org label Sep 24, 2025

openjdk bot added the rfr Pull request is ready for review label Sep 24, 2025

jerboaa approved these changes Sep 25, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Sep 25, 2025

openjdk bot removed the ready Pull request is ready to be integrated label Sep 25, 2025

Merge branch 'master' into linux-container-mem-restructure

865c2b4

jerboaa reviewed Sep 30, 2025

View reviewed changes

jerboaa approved these changes Sep 30, 2025

View reviewed changes

fisk approved these changes Oct 2, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Oct 2, 2025

openjdk bot added the integrated Pull request has been integrated label Oct 2, 2025

openjdk bot closed this Oct 2, 2025

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 2, 2025

8292984: Refactor internal container-related interfaces for clarity #27470

8292984: Refactor internal container-related interfaces for clarity #27470

Uh oh!

Conversation

caspernorrbin commented Sep 24, 2025 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented Sep 24, 2025

Uh oh!

openjdk bot commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openjdk bot commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mlbridge bot commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

jerboaa left a comment

Choose a reason for hiding this comment

Uh oh!

jerboaa commented Sep 25, 2025

Uh oh!

openjdk bot commented Sep 25, 2025

Uh oh!

caspernorrbin commented Sep 30, 2025

Uh oh!

jerboaa Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

caspernorrbin Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

jerboaa Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

jerboaa left a comment

Choose a reason for hiding this comment

Uh oh!

fisk left a comment

Choose a reason for hiding this comment

Uh oh!

caspernorrbin commented Oct 2, 2025

Uh oh!

openjdk bot commented Oct 2, 2025

Uh oh!

openjdk bot commented Oct 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

caspernorrbin commented Sep 24, 2025 •

edited by openjdk bot

Loading

openjdk bot commented Sep 24, 2025 •

edited

Loading

openjdk bot commented Sep 24, 2025 •

edited

Loading

mlbridge bot commented Sep 24, 2025 •

edited

Loading