Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK-8266222: [aix] In mmap-mode, partial releases with os::release_memory may trash internal bookkeeping #3765

Conversation

@tstuefe
Copy link
Member

@tstuefe tstuefe commented Apr 28, 2021

In os_aix.cpp we keep book about for reserved address ranges since we need to keep information about which API (mmap or shmat) had been used for this range. (code hint: see vmembk_(add|remove|find)).

When releasing memory via os::release_memory, we remove those ranges from the internal bookkeeping.

However, for mmap() the release may be partial. In that case the internal bookkeeping is not updated correctly: we just remove the record for the whole original address range.

After that operation, the remainder address range is still mapped from the point of the VM, but the bookkeeping is lost and subsequent calls to os::release_memory or os::commit/uncommit_memory for this range will run into a guarantee.

Partial releases can happen e.g. when, in the process of running os::reserve_memory_aligned(), the extra-aligned pages are unmapped. This is of more concern now with the new Metaspace, since we now reserve with larger alignments (4G+).

This problem was hidden by the fact that we typically run in 64K paged mode where we use SysV shared memory; but if AME is enabled on AIX, 64K pages are disabled, and the VM falls back to 4K pages and using mmap, and runs into this bug.

Edit: Tested on AIX machines with and without AME activated, solves the problems. All SAP nightlies ran, no problems caused by this patch.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8266222: [aix] In mmap-mode, partial releases with os::release_memory may trash internal bookkeeping

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/3765/head:pull/3765
$ git checkout pull/3765

Update a local copy of the PR:
$ git checkout pull/3765
$ git pull https://git.openjdk.java.net/jdk pull/3765/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 3765

View PR using the GUI difftool:
$ git pr show -t 3765

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/3765.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Apr 28, 2021

👋 Welcome back stuefe! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

Loading

@openjdk
Copy link

@openjdk openjdk bot commented Apr 28, 2021

@tstuefe The following label will be automatically applied to this pull request:

  • hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

Loading

@tstuefe tstuefe marked this pull request as ready for review May 3, 2021
@openjdk openjdk bot added the rfr label May 3, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented May 3, 2021

Webrevs

Loading

@@ -1989,6 +1989,7 @@ bool os::pd_release_memory(char* addr, size_t size) {
// Dynamically do different things for mmap/shmat.
vmembk_t* const vmi = vmembk_find(addr);
guarantee0(vmi);
vmi->assert_is_valid_subrange(addr, size);
Copy link
Member Author

@tstuefe tstuefe May 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer note: this is unrelated to the problem, I just lifted the assert call from both branches and made it independent from debug/release.

Loading

Copy link
Contributor

@TheRealMDoerr TheRealMDoerr left a comment

This looks good.

Loading

@openjdk
Copy link

@openjdk openjdk bot commented May 5, 2021

@tstuefe This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8266222: [aix] In mmap-mode, partial releases with os::release_memory may trash internal bookkeeping

Reviewed-by: mdoerr

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 112 new commits pushed to the master branch:

  • c9873c4: 8260621: (jrtfs) ThreadLocal memory leak in ImageBufferCache when using jrtfs
  • 82768d9: 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][]
  • b71f85a: 8264398: BevelBorderUIResource​(int, Color, Color) and BevelBoder(int, Color, Color) spec should clarify about usage of highlight and shadow color
  • b172555: 8266171: -Warray-bounds happens in imageioJPEG.c
  • 8bcebe7: 8265505: findsym does not work on remote debug server
  • b88785d: 8266038: Move newAddress() to JVMDebugger
  • 2c53654: 8266179: [macos] jpackage should specify architecture for produced pkg files
  • d282799: 8255566: Add size validation when parsing values from VersionProps
  • 61365d5: 8266465: Add wildcard to JTwork/JTreport exclude in jib-profiles.js
  • f00b70e: 8266527: RandomTestCoverage.java failing due to API removal
  • ... and 102 more: https://git.openjdk.java.net/jdk/compare/343a4a76f273078f78897e8fb7e695bc2c111536...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

Loading

@openjdk openjdk bot added the ready label May 5, 2021
@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented May 5, 2021

Danke, Martin!

Loading

@tstuefe
Copy link
Member Author

@tstuefe tstuefe commented May 5, 2021

Note that since we don't have that many AIX reviewers, and this only concerns AIX code, I go ahead and push. Its our port anyway, if something breaks we will have to fix it.

/integrate

Loading

@openjdk openjdk bot closed this May 5, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels May 5, 2021
@openjdk
Copy link

@openjdk openjdk bot commented May 5, 2021

@tstuefe Since your change was applied there have been 114 commits pushed to the master branch:

  • 65ce4d2: 8266491: Remove resolve and obj_equals leftovers from BarrierSetAssembler
  • a8046c9: 8266436: Synthetic constructor trees have non-null return type
  • c9873c4: 8260621: (jrtfs) ThreadLocal memory leak in ImageBufferCache when using jrtfs
  • 82768d9: 8221503: vmTestbase/nsk/jdb/eval/eval001/eval001.java fails with: com.sun.jdi.InvalidTypeException: Can't assign double[][][] to double[][][]
  • b71f85a: 8264398: BevelBorderUIResource​(int, Color, Color) and BevelBoder(int, Color, Color) spec should clarify about usage of highlight and shadow color
  • b172555: 8266171: -Warray-bounds happens in imageioJPEG.c
  • 8bcebe7: 8265505: findsym does not work on remote debug server
  • b88785d: 8266038: Move newAddress() to JVMDebugger
  • 2c53654: 8266179: [macos] jpackage should specify architecture for produced pkg files
  • d282799: 8255566: Add size validation when parsing values from VersionProps
  • ... and 104 more: https://git.openjdk.java.net/jdk/compare/343a4a76f273078f78897e8fb7e695bc2c111536...master

Your commit was automatically rebased without conflicts.

Pushed as commit 250b45a.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Loading

@tstuefe tstuefe deleted the JDK-8266222-aix-4k-mmap-mode-partial-release-bug branch May 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants