JDK-8317661: [REDO] store/load order not preserved when handling memory pool due to weakly ordered memory architecture of aarch64

[dafedafe]

Issue

An intermittent Memory Pool not found error has been noticed when running a few tests (vmTestbase/vm/mlvm/meth/stress/compiler/deoptimize/Test.java, vmTestbase/vm/mlvm/meth/stress/compiler/sequences/Test.java) on macosx_aarch64 (production build) with non-segmented code cache.

Origin

The issue originates from the fact that aarch64 architecture is a weakly ordered memory architecture, i.e. it permits the observation and completion of memory accesses in a different order from the program order.

More precisely: while calling CodeHeapPool::get_memory_usage, the used and committed variables are retrieved

jdk/src/hotspot/share/services/memoryPool.cpp

Lines 181 to 182 in 138542d

	size_t used = used_in_bytes();
	size_t committed = _codeHeap->capacity();

and these are computed based on different variables saved in memory in CodeCache::allocate (during heap->allocate and heap->expand_by to be precise) .

jdk/src/hotspot/share/code/codeCache.cpp

Lines 535 to 537 in 138542d

	cb = (CodeBlob*)heap->allocate(size);
	if (cb != nullptr) break;
	if (!heap->expand_by(CodeCacheExpansionSize)) {

The problem happens when first heap->expand_by gets called (which increases committed) and then heap->allocate gets called in a second loop pass (which increases used). Although stores in CodeCache::allocate happen in the this order, when reading from memory in CodeHeapPool::get_memory_usage it can happen that used has the newly computed value, while committed is still "old" (because of ARM’s weak memory order). This is a problem, since committed must be > than used.

Solution

To avoid this situation we must assure that values used to calculate committed are actually saved before the values used to calculate used and that the opposite be true for reading. To enforce this explicit barriers are introduced: OrderAccess::release in CodeCache::allocate if we need to expand the heap, to ensure heap expansion is visible before we allocate, and OrderAccess::acquire before calculating committed in CodeHeapPool::get_memory_usage to make sure that the cache expansion is seen.

Acquiring a CodeCache_lock has been attempted in #15819 but resulted in deadlocks (that seem to be unavoidable).

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8317661: [REDO] store/load order not preserved when handling memory pool due to weakly ordered memory architecture of aarch64 (Bug - P4)

Reviewers

• David Holmes (@dholmes-ora - Reviewer)
• Andrew Haley (@theRealAph - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/16143/head:pull/16143
$ git checkout pull/16143

Update a local copy of the PR:
$ git checkout pull/16143
$ git pull https://git.openjdk.org/jdk.git pull/16143/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 16143

View PR using the GUI difftool:
$ git pr show -t 16143

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/16143.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back dfenacci! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@dafedafe The following label will be automatically applied to this pull request:

• hotspot-runtime

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full - Incremental (c6fa9674)
• 00: Full (492ffbb9)

[dcubed-ojdk]

What testing has been done on this [REDO]?

[dholmes-ora]

When we do the release here, what previous stores are we trying to order with the reading code below? It is hard to see the complete picture with this code just from the PR changes.

[dafedafe]

@dholmes-ora you’re right, it is quite difficult to get the whole picture: let me try to make it a bit clearer (I hope it makes sense).

The issue originates here:

jdk/src/hotspot/share/services/memoryPool.cpp

Lines 181 to 182 in 138542d

	size_t used = used_in_bytes();
	size_t committed = _codeHeap->capacity();

We want to ensure that the values used to calculate committed are actually saved before the values used to calculate used are used.

The problem starts in CodeCache::allocate. First CodeHeap::allocate is called and if there is not enough space we call CodeHeap::expand_by and retry CodeHeap::allocate (while loop):

jdk/src/hotspot/share/code/codeCache.cpp

Lines 535 to 537 in e510dee

	cb = (CodeBlob*)heap->allocate(size);
	if (cb != nullptr) break;
	if (!heap->expand_by(CodeCacheExpansionSize)) {

During the second call of CodeHeap::allocate we update the values used to calculate the used variable (_next_segment in particular since it is the last to be updated) and we want to make sure that when we read it, all earlier values have been written.

As for the link between used and _next_segment, used is calculated here

jdk/src/hotspot/share/memory/heap.cpp

Line 556 in e510dee

return segments_to_size(_next_segment - _freelist_segments);

from _next_segment and _freelist_segments. _next_segment is the last to be written and it is updated here

jdk/src/hotspot/share/memory/heap.cpp

Line 306 in e510dee

_next_segment += number_of_segments;

which is called from CodeCache::allocate

jdk/src/hotspot/share/code/codeCache.cpp

Line 535 in e510dee

cb = (CodeBlob*)heap->allocate(size);

[dholmes-ora]

Sorry I'm still missing part of this (and I know we discussed it a few weeks ago). What is the variable(s) that is written before _next_segment that we need to see updated when we call _codeHeap->capacity()?

[dafedafe]

Basically it would be the field VirtualSpace::_high. _codeHeap->capacity() calls VirtualSpace::committed_size which reads the _high and _low fields:

jdk/src/hotspot/share/memory/virtualspace.cpp

Lines 768 to 770 in e510dee

	size_t VirtualSpace::committed_size() const {
	return pointer_delta(high(), low(), sizeof(char));
	}

_low doesn't really get updated but _high is increased in VirtualSpace::expand_by, which is called by CodeHeap::expand_by

jdk/src/hotspot/share/memory/heap.cpp

Line 259 in e510dee

if (!_memory.expand_by(dm)) return false;

which in turn is called in CodeCache::allocate:

jdk/src/hotspot/share/code/codeCache.cpp

Line 537 in e510dee

if (!heap->expand_by(CodeCacheExpansionSize)) {

[dafedafe]

@dholmes-ora is there possibly a way to restrict the update that we need to see to that _high variable only?

[dholmes-ora]

So ... as we discussed offline a few weeks ago, it is very hard to clearly express the ordering constraints in this code as the two variables are so far apart in the code. So to reiterate, on the writer side we have:

 while (true) {
    cb = (CodeBlob*)heap->allocate(size);  // updates _next_segment
    if (cb != nullptr) break;
    if (!heap->expand_by(CodeCacheExpansionSize)) {  // updates _high

where the writes to _high in expand_by have to be ordered with the write to _next_segment in allocate that happen on the next loop iteration.

And on the reader side we have:

size_t used      = used_in_bytes();   // reads _next_segment
size_t committed = _codeHeap->capacity();  // reads _high

where the reads must also be ordered.

I can't help but think that the clearest fix here is to use explicit barriers as discussed before the locking solution was proposed. The problem using release_store and load_acquire in the bottom level code is that those semantics are only needed in this higher-level code for the codeHeap, not for every kind of heap the code applies to.

I wonder if perhaps the following is clearer:

 while (true) {
    cb = (CodeBlob*)heap->allocate(size);
    if (cb != nullptr) break;
    if (!heap->expand_by(CodeCacheExpansionSize)) {
      // ...
    } else {
      OrderAccess::release(); // ensure heap expansion is visible before we allocate on next iteration
    }

coupled with:

size_t used      = used_in_bytes(); 
OrderAccess:acquire();
size_t committed = _codeHeap->capacity(); 

Thoughts?

[theRealAph]

I can't help but think that the clearest fix here is to use explicit barriers as discussed before the locking solution was proposed. The problem using release_store and load_acquire in the bottom level code is that those semantics are only needed in this higher-level code for the codeHeap, not for every kind of heap the code applies to.

I wonder if perhaps the following is clearer:

 while (true) {
    cb = (CodeBlob*)heap->allocate(size);
    if (cb != nullptr) break;
    if (!heap->expand_by(CodeCacheExpansionSize)) {
      // ...
    } else {
      OrderAccess::release(); // ensure heap expansion is visible

"visible to an asynchronous observer (e.g. CodeHeapPool::get_memory_usage())"

before we allocate on next iteration
}

coupled with:

size_t used = used_in_bytes();
OrderAccess:acquire();
size_t committed = _codeHeap->capacity();

Thoughts?

As long as it's more thoroughly commented, i guess so.

A floating release is, however, a pretty bad code smell. The only reason we're aren't solving this problem (IMHO) properly is that the code cache lock isn't re-entrant.

[dafedafe]

The problem using release_store and load_acquire in the bottom level code is that those semantics are only needed in this higher-level code for the codeHeap, not for every kind of heap the code applies to.

I wonder if perhaps the following is clearer:

 while (true) {
    cb = (CodeBlob*)heap->allocate(size);
    if (cb != nullptr) break;
    if (!heap->expand_by(CodeCacheExpansionSize)) {
      // ...
    } else {
      OrderAccess::release(); // ensure heap expansion is visible before we allocate on next iteration
    }

This would definitely make sense.

A floating release is, however, a pretty bad code smell. The only reason we're aren't solving this problem (IMHO) properly is that the code cache lock isn't re-entrant.

Indeed, it still looks a bit "messy". I was also wondering if there could be a performance penalty by using explicit barriers (although at least OrderAccess::release() won't be called very often with this solution).

In any case I've modified the code to make use of explicit barriers following @dholmes-ora suggestion.

Thanks a lot!

[dafedafe]

What testing has been done on this [REDO]?

@dcubed-ojdk I ran Tier1-5 + hs-precheckin-comp,hs-comp-stress tests and checked that it fixes the original test
(vmTestbase/vm/mlvm/meth/stress/compiler/deoptimize/Test.java)

[dholmes-ora]

I think this is the best of a bunch of bad options.

Thanks.

[openjdk]

@dafedafe This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8317661: [REDO] store/load order not preserved when handling memory pool due to weakly ordered memory architecture of aarch64

Reviewed-by: dholmes, aph

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 430 new commits pushed to the master branch:

• 141dae8: 8318811: Compiler directives parser swallows a character after line comments
• 667cca9: 8288899: java/util/concurrent/ExecutorService/CloseTest.java failed with "InterruptedException: sleep interrupted"
• b9dcd4b: 8318964: Fix build failures caused by 8315097
• d52a995: 8315097: Rename createJavaProcessBuilder
• 957703b: 8307168: Inconsistent validation and handling of --system flag arguments
• 5b5fd36: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
• abad040: 8313781: Add regression tests for large page logging and user-facing error messages
• 9123961: 8318096: Introduce AsymmetricKey interface with a getParams method
• 4a142c3: 8274122: java/io/File/createTempFile/SpecialTempFile.java fails in Windows 11
• 77fe0fd: 8272215: Add InetAddress methods for parsing IP address literals
• ... and 420 more: https://git.openjdk.org/jdk/compare/e510dee162612d9a706ba54d0ab79a49139e77d8...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[dafedafe]

@dholmes-ora @theRealAph thank you very much for your reviews!

[dafedafe]

/integrate

[openjdk]

Going to push as commit ddd0716.
Since your change was applied there have been 430 commits pushed to the master branch:

• 141dae8: 8318811: Compiler directives parser swallows a character after line comments
• 667cca9: 8288899: java/util/concurrent/ExecutorService/CloseTest.java failed with "InterruptedException: sleep interrupted"
• b9dcd4b: 8318964: Fix build failures caused by 8315097
• d52a995: 8315097: Rename createJavaProcessBuilder
• 957703b: 8307168: Inconsistent validation and handling of --system flag arguments
• 5b5fd36: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
• abad040: 8313781: Add regression tests for large page logging and user-facing error messages
• 9123961: 8318096: Introduce AsymmetricKey interface with a getParams method
• 4a142c3: 8274122: java/io/File/createTempFile/SpecialTempFile.java fails in Windows 11
• 77fe0fd: 8272215: Add InetAddress methods for parsing IP address literals
• ... and 420 more: https://git.openjdk.org/jdk/compare/e510dee162612d9a706ba54d0ab79a49139e77d8...master

Your commit was automatically rebased without conflicts.

[openjdk]

@dafedafe Pushed as commit ddd0716.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.