Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8315082: [REDO] Generational ZGC: Tests crash with assert(index == 0 || is_power_of_2(index)) #15589

Closed
wants to merge 21 commits into from

Conversation

robcasloz
Copy link
Contributor

@robcasloz robcasloz commented Sep 6, 2023

This changeset (REDO of JDK-8312749) ensures that the array copy stub underlying the intrinsic implementation of Object.clone only copies its (double-word aligned) payload, excluding the remaining object alignment padding words, when a non-default ObjectAlignmentInBytes value is used. This prevents the specialized ZGC stubs for Object[] array copy from processing undefined object alignment padding words as valid object pointers. For further details about the specific failure, see initial analysis of JDK-8312749 by Erik Österlund and Stefan Karlsson and comments in test/hotspot/jtreg/compiler/gcbarriers/TestArrayCopyWithLargeObjectAlignment.java.

As a side-benefit, the changeset simplifies the array size computation logic in GraphKit::new_array() by decoupling computation of header size and alignment padding size.

Additional changes compared to JDK-8312749

This changeset proposes the exact same solution as JDK-8312749, that is, identical changes to barrierSetC2.cpp, graphKit.cpp, library_call.cpp, and TestArrayCopyWithLargeObjectAlignment.java. On top of that, it relaxes an assertion in the idealization of ArrayCopy nodes violated by JDK-8312749 and reported in JDK-8315029 (new changes in arraycopynode.cpp, new regression test TestCloneArrayWithDifferentLengthConstness.java). The original, stricter assertion checks that, while idealizing an ArrayCopy node, the "constness" of the array copy's word-length (whether it is known by C2 to be constant or not) is equivalent to that of the array copy's element-length. For cases in which the element-length is within a small, fixed range (e.g. for an int array of length 3..4) so that all element-length values lead to the same number of words (2), the assertion used to hold before this changeset only because of weak type propagation in AndL (preventing the constant word-length to be discovered), see the left graph below:

from-element-to-word-length

With the proposed changes, the array copy word-length is computed in a more straightforward way that enables C2 to infer the precise number of words in the same scenario (see the right graph above). To accommodate this optimization, this changeset relaxes the assertion to check only one direction of the implication (if the element-length is constant, so is the word-length). The optimization does not affect the remaining idealization code, since it only uses element-length in the context in which the optimization is applied.

The changeset includes an additional regression test (TestCloneArrayWithDifferentLengthConstness.java) that exercises different constant/variable combinations of element-length and word-length.

Testing

Functionality
  • tier1-5 (windows-x64, linux-x64, linux-aarch64, macosx-x64, and macosx-aarch64)
  • tier6-7 (linux-x64 only)
  • compiler-focused stress testing
  • failing tests reported in JDK-8315029
  • tier1-3, and a few custom examples, applying JDK-8139457 on top of this changeset
Performance

Tested performance on the following set of OpenJDK micro-benchmarks, on linux-x64 (for both G1 and ZGC, using different ObjectAlignmentInBytes values):

  • openjdk.bench.java.lang.ArrayClone.byteClone
  • openjdk.bench.java.lang.ArrayClone.intClone
  • openjdk.bench.java.lang.ArrayFiddle.simple_clone
  • openjdk.bench.java.lang.Clone.cloneLarge
  • openjdk.bench.java.lang.Clone.cloneThreeDifferent

No significant regression was observed.


Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue

Issue

  • JDK-8315082: [REDO] Generational ZGC: Tests crash with assert(index == 0 || is_power_of_2(index)) (Bug - P3)

Reviewers

Contributors

  • Stefan Karlsson <stefank@openjdk.org>
  • Erik Österlund <eosterlund@openjdk.org>

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15589/head:pull/15589
$ git checkout pull/15589

Update a local copy of the PR:
$ git checkout pull/15589
$ git pull https://git.openjdk.org/jdk.git pull/15589/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15589

View PR using the GUI difftool:
$ git pr show -t 15589

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15589.diff

Webrev

Link to Webrev Comment

@bridgekeeper
Copy link

bridgekeeper bot commented Sep 6, 2023

👋 Welcome back rcastanedalo! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Sep 6, 2023

@robcasloz The following label will be automatically applied to this pull request:

  • hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot hotspot-dev@openjdk.org label Sep 6, 2023
@robcasloz
Copy link
Contributor Author

/contributor add @stefank

@robcasloz
Copy link
Contributor Author

/contributor add @fisk

@robcasloz
Copy link
Contributor Author

/label add hotspot-compiler

@robcasloz
Copy link
Contributor Author

/label add hotspot-gc

@openjdk
Copy link

openjdk bot commented Sep 7, 2023

@robcasloz
Contributor Stefan Karlsson <stefank@openjdk.org> successfully added.

@openjdk
Copy link

openjdk bot commented Sep 7, 2023

@robcasloz
Contributor Erik Österlund <eosterlund@openjdk.org> successfully added.

@openjdk openjdk bot added the hotspot-compiler hotspot-compiler-dev@openjdk.org label Sep 7, 2023
@openjdk
Copy link

openjdk bot commented Sep 7, 2023

@robcasloz
The hotspot-compiler label was successfully added.

@openjdk openjdk bot added the hotspot-gc hotspot-gc-dev@openjdk.org label Sep 7, 2023
@openjdk
Copy link

openjdk bot commented Sep 7, 2023

@robcasloz
The hotspot-gc label was successfully added.

@robcasloz robcasloz marked this pull request as ready for review September 11, 2023 07:02
@openjdk openjdk bot added the rfr Pull request is ready for review label Sep 11, 2023
@mlbridge
Copy link

mlbridge bot commented Sep 11, 2023

Webrevs

@robcasloz
Copy link
Contributor Author

@albertnetymk @vnkozlov @TobiHartmann could you please review this revised version of JDK-8312749? The core changes remain identical, the only additional changes are an assertion relaxation and a second regression test (see description for details). Thanks!

@openjdk
Copy link

openjdk bot commented Sep 11, 2023

@robcasloz This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8315082: [REDO] Generational ZGC: Tests crash with assert(index == 0 || is_power_of_2(index))

Co-authored-by: Stefan Karlsson <stefank@openjdk.org>
Co-authored-by: Erik Österlund <eosterlund@openjdk.org>
Reviewed-by: ayang, thartmann, kvn

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 205 new commits pushed to the master branch:

  • 54028e7: 8316562: serviceability/sa/jmap-hprof/JMapHProfLargeHeapTest.java times out after JDK-8314829
  • 7c991cc: 8296246: Update Unicode Data Files to Version 15.1.0
  • a021dbc: 8316149: Open source several Swing JTree JViewport KeyboardManager tests
  • 455cfae: 8315880: change LockingMode default from LM_LEGACY to LM_LIGHTWEIGHT
  • 9e00949: 8316427: Duplicated code for {obj,type}ArrayKlass::array_klass
  • c43ebd3: 8315981: Opensource five more random Swing tests
  • e30e356: 8316461: Fix: make test outputs TEST SUCCESS after unsuccessful exit
  • 242eeae: 8286757: adlc tries to build with /pathmap but without /experimental:deterministic
  • b275bdd: 8308995: Update Network IO JFR events to be static mirror events
  • e1870d3: 8316411: compiler/compilercontrol/TestConflictInlineCommands.java fails intermittent with force inline by CompileCommand missing
  • ... and 195 more: https://git.openjdk.org/jdk/compare/024133b089d911dcc3ea70dfdaa6b150b14a9eb4...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Sep 11, 2023
@robcasloz
Copy link
Contributor Author

robcasloz commented Sep 11, 2023

Thanks for reviewing, Albert!

Copy link
Member

@TobiHartmann TobiHartmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed analysis and explanation. The fix looks good.

Copy link
Contributor

@vnkozlov vnkozlov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments.

Comment on lines +678 to +684
if (is_array) {
// Ensure the array payload size is rounded up to the next BytesPerLong
// multiple when converting to double-words. This is necessary because array
// size does not include object alignment padding, so it might not be a
// multiple of BytesPerLong for sub-long element types.
payload_size = kit->gvn().transform(new AddXNode(payload_size, kit->MakeConX(BytesPerLong - 1)));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Back to this change. Why rounding is only arrays? Do we have a check that object's alignment >= 8 bytes? If it less you may access beyond array.
Can Add result overflow in 32-bit VM?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why rounding is only arrays?

Class instance (bytewise) size is already rounded to MinObjAlignmentInBytes (i.e. it is a multiple of BytesPerLong, assuming minimum object alignment of 8 bytes), so rounding it again would be a no-op.

Do we have a check that object's alignment >= 8 bytes? If it less you may access beyond array.

This code simply assumes that the minimum object alignment in HotSpot is 8 bytes, as per src/hotspot/share/runtime/globals.hpp:

#ifdef _LP64
  ...
  product(int, ObjectAlignmentInBytes, 8,                                   \
          "Default object alignment in bytes, 8 is minimum")                \
          range(8, 256)                                                     \
          constraint(ObjectAlignmentInBytesConstraintFunc, AtParse)
  ...
#else
  ...
  const int ObjectAlignmentInBytes = 8;
  ...
#endif

Is there any VM configuration where this might not hold? Or do I misunderstand your question?

Can Add result overflow in 32-bit VM?

payload_size + (BytesPerLong - 1) cannot overflow because it is performed after a subtraction of a larger quantity (base_off). Would it make sense to add an assertion like this?:

assert(base_off >= BytesPerLong - 1, "array payload size computation should not overflow");

}
// else if round_mask == 0, the size computation is self-rounding
Node* non_rounded_size = _gvn.transform(new AddXNode(headerx, abody));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here

Copy link
Contributor Author

@robcasloz robcasloz Sep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

headerx + abody cannot overflow because of the array length limit imposed by TypeAryPtr::max_array_length(), which is designed to fit the total bytes of an array header and payload (including external alignment) in 32 bits (on 32-bits platforms). I checked this using the length limits for all basic types. Exceeding this limit causes the array allocation to throw a java.lang.OutOfMemoryError: Requested array size exceeds VM limit exception).

Node* size = non_rounded_size;
if (round_mask != 0) {
Node* mask1 = MakeConX(round_mask);
size = _gvn.transform(new AddXNode(size, mask1));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here again question about overflow in 32-bit VM.
Do we generate compare with FastAllocateSizeLimit before this code is executed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

size + mask1 cannot overflow because, as you suggest, it is only computed in the fast allocation path (i.e. for small values of size restricted by FastAllocateSizeLimit). The only way to bypass this check and indeed trigger the overflow is to set -XX:FastAllocateSizeLimit=2147483647 (debug-only flag), this causes indeed a segmentation fault, reported in JDK-8316512.

In general, the behavior of size computation within GraphKit::new_array w.r.t. overflow is not altered by this changeset, as far as I can see, because it essentially only changes whether round_mask is added to the header size (as done before) or to the total, non-rounded size (as proposed here).

@robcasloz
Copy link
Contributor Author

robcasloz commented Sep 19, 2023

Thanks for the detailed analysis and explanation. The fix looks good.

Thanks for reviewing, Tobias!

@robcasloz
Copy link
Contributor Author

Few comments.

Thanks for reviewing, Vladimir! See my replies inlined for each comment/question.

Copy link
Contributor

@vnkozlov vnkozlov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for answering all my questions.

@robcasloz
Copy link
Contributor Author

/integrate

@openjdk
Copy link

openjdk bot commented Sep 21, 2023

Going to push as commit ceff47b.
Since your change was applied there have been 211 commits pushed to the master branch:

  • df4a25b: 8308762: Metaspace leak with Instrumentation.retransform
  • 8412479: 8316229: Enhance class initialization logging
  • c04c9ea: 8316627: JViewport Test headless failure
  • 5cacf21: 8316156: ByteArrayInputStream.transferTo causes MaxDirectMemorySize overflow
  • 3461c7b: 8316532: Native library copying in BuildMicrobenchmark.gmk cause dups on macOS
  • 3301fb1: 8315869: UseHeavyMonitors not used
  • 54028e7: 8316562: serviceability/sa/jmap-hprof/JMapHProfLargeHeapTest.java times out after JDK-8314829
  • 7c991cc: 8296246: Update Unicode Data Files to Version 15.1.0
  • a021dbc: 8316149: Open source several Swing JTree JViewport KeyboardManager tests
  • 455cfae: 8315880: change LockingMode default from LM_LEGACY to LM_LIGHTWEIGHT
  • ... and 201 more: https://git.openjdk.org/jdk/compare/024133b089d911dcc3ea70dfdaa6b150b14a9eb4...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Sep 21, 2023
@openjdk openjdk bot closed this Sep 21, 2023
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Sep 21, 2023
@openjdk
Copy link

openjdk bot commented Sep 21, 2023

@robcasloz Pushed as commit ceff47b.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot hotspot-dev@openjdk.org hotspot-compiler hotspot-compiler-dev@openjdk.org hotspot-gc hotspot-gc-dev@openjdk.org integrated Pull request has been integrated
4 participants