8318966: Some methods make promises about Java array element alignment that are too strong #16681

JornVernee · 2023-11-15T22:46:03Z

See the JBS issue for an extended problem description.

This patch changes the specification and implementation of MethodHandles::byteArrayViewVarHandle, MethodHandles::byteBufferViewVarHandle, ByteBuffer::alignedSlice, and ByteBuffer::alignmentOffset to weaken the guarantees they make about the alignment of Java array elements, in order to bring them in line with the guarantees made by an arbitrary JVM implementation (which makes no guarantees about array element alignment beyond them being aligned to their natural alignment).

MethodHandles::byteArrayViewVarHandle: we can not guarantee any alignment for the accesses. Which means we can only reliably support plain get and set access modes. The javadoc text explaining which other access modes are supported, or how to compute aligned offsets into the array is dropped, as it is not guaranteed to be correct on all JVM implementations. The implementation of the returned VarHandle is changed to throw an UnsupportedOperationException for the unsupported access modes, as mandated by the spec of VarHandle 1.
MethodHandles::byteBufferViewVarHandle: the implementation/spec is incorrect when accessing a heap buffer (wrapping a byte[]), for the same reasons as byteArrayViewVarHandle. The spec is changed to specify that when accessing a heap buffer, only plain get and set access modes are supported. The implementation of the returned var handle is changed to throw an IllegalStateException when an access is attempted on a heap buffer using an access mode other than plain get or set. Note that we don't throw an outright UnsupportedOperationException for this case, since whether the access modes are supported depends on the byte buffer instance being used.
ByteBuffer::alignedSlice and ByteBuffer::alignmentOffset: The former method depends directly on the latter for all its alignment computations. We change the implementation of the latter method to throw an UnsupportedOperationException for all unit sizes greater than 1, when the buffer is non-direct. This change is largely covered by the existing specification:

     * @throws UnsupportedOperationException
     *         If the native platform does not guarantee stable alignment offset
     *         values for the given unit size when managing the memory regions
     *         of buffers of the same kind as this buffer (direct or
     *         non-direct).  For example, if garbage collection would result
     *         in the moving of a memory region covered by a non-direct buffer
     *         from one location to another and both locations have different
     *         alignment characteristics.

However, the @implNote mentions that an UnsupportedOperationException will be thrown for unit sizes greater than 8. This is updated to say unit sizes greater than 1.

Both byteArrayViewVarHandle and byteBufferViewVarHandle do not support byte[], or boolean[] as the view array class, for which accesses would be aligned. So, it is safe to say that access will always be unaligned when using the var handle returned by these methods.

Note that the testing code for these APIs is tightly coupled, so it is practically convenient to address all these issues together.

Testing: local java/lang/invoke/VarHandles, tier 1-4

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue
Change requires CSR request JDK-8320247 to be approved

Issues

JDK-8318966: Some methods make promises about Java array element alignment that are too strong (Bug - P3)
JDK-8320247: Some methods make promises about Java array element alignment that are too strong (CSR)

Reviewers

Paul Sandoz (@PaulSandoz - Reviewer)
Maurizio Cimadamore (@mcimadamore - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/16681/head:pull/16681
$ git checkout pull/16681

Update a local copy of the PR:
$ git checkout pull/16681
$ git pull https://git.openjdk.org/jdk.git pull/16681/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 16681

View PR using the GUI difftool:
$ git pr show -t 16681

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/16681.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2023-11-15T22:46:56Z

👋 Welcome back jvernee! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2023-11-15T22:47:41Z

@JornVernee The following labels will be automatically applied to this pull request:

core-libs
nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

src/java.base/share/classes/java/nio/X-Buffer.java.template

PaulSandoz · 2023-11-16T17:31:46Z

src/java.base/share/classes/java/lang/invoke/MethodHandles.java

-     * {@link Double#doubleToRawLongBits}, respectively).
+     * Only plain {@linkplain VarHandle.AccessMode#GET get} and {@linkplain VarHandle.AccessMode#SET set}
+     * access modes are supported by the returned var handle. For all other access modes, an
+     * {@link UnsupportedOperationException} will be thrown.


I recommend adding an api note explaining that native memory segments, direct byte buffers, or heap memory segments backed by long[] should be used if support for other access modes are required.

Good idea. Thanks

Should we make these unaligned access modes throw ISE like before, when the given index is unaligned?

You mean get and set? They should never throw, as unaligned access is fine. For other access modes, we can never guarantee that an access is aligned, so UOE is appropriate. (IIRC this is mandated by existing spec. I'll try to find it again)

P.S. See e.g. the javadoc of VarHandle::getVolatile:

@throws UnsupportedOperationException if the access mode is unsupported for this VarHandle.

P.P.S. Also remembering that we can not have any implementation for the access methods, in order for isAccessModeSupported to work correctly. And the logic that handles unsupported methods throws UOE (see VarForm::getMemberName)

Good point, so previous behavior of throwing ISE is out of spec

PaulSandoz · 2023-11-16T17:34:40Z

src/java.base/share/classes/java/lang/invoke/MethodHandles.java

@@ -4641,6 +4606,8 @@ public static VarHandle byteArrayViewVarHandle(Class<?> viewArrayClass,
     * update access modes compare values using their bitwise representation
     * (see {@link Float#floatToRawIntBits} and
     * {@link Double#doubleToRawLongBits}, respectively).
+     * <p>
+     * Access to heap byte buffers is always unaligned.


I recommend merging that sentence into the paragraph on heap byte buffers e.g.:

For direct buffers, access of the bytes at an index is always misaligned. As a result only the plain...

But... surely if I'm just accessing bytes the access cannot possibly be mis-aligned?

Right. I think there is a difference between 'unaligned' and 'misaligned'. Unaligned means there is no effort made to align the access (so it may be misaligned). Misaligned means definitely not aligned. At least, that is my interpretation.

This is what I wrote in the latest version:

* For heap byte buffers, access is always unaligned. As a result, only the plain * {@linkplain VarHandle.AccessMode#GET get} * and {@linkplain VarHandle.AccessMode#SET set} access modes are supported by the * returned var handle. For all other access modes, an {@link IllegalStateException} * will be thrown.

Uhm, I think my assumption was wrong - the byte/boolean carrier is not supported by this method, so there's no case where heap access would be guaranteed to be aligned.

PaulSandoz · 2023-11-16T17:39:35Z

src/java.base/share/classes/java/nio/X-Buffer.java.template

     * @implNote
     * This implementation throws {@code UnsupportedOperationException} for
-     * non-direct buffers when the given unit size is greater then {@code 8}.
+     * non-direct buffers when the given unit size is greater then {@code 1}.


This is no longer an implementation note, its now part of the specified API. So i think we can simplify the text of the @throws UOE ... to just say:

@throws UOE if the buffer is non-direct and the unit size > 1

Same for the other method.

Right. Good idea

…in access

JornVernee · 2023-11-16T19:09:23Z

src/java.base/share/classes/java/lang/invoke/X-VarHandleByteArrayView.java.template

@@ -736,17 +323,10 @@ final class VarHandleByteArrayAs$Type$s extends VarHandleByteArrayBase {
        static boolean compareAndSet(VarHandle ob, Object obb, int index, $type$ expected, $type$ value) {
            ByteBufferHandle handle = (ByteBufferHandle)ob;
            ByteBuffer bb = (ByteBuffer) Objects.requireNonNull(obb);
-#if[Object]


Note that I removed this 'if Object' block (and the one below), as the public API does not support object access any way:

jdk/src/java.base/share/classes/java/lang/invoke/MethodHandles.java

Lines 4591 to 4593 in 9727f4b

* The supported component types (variables types) are {@code short},

* {@code char}, {@code int}, {@code long}, {@code float} and

* {@code double}.

JornVernee · 2023-11-16T19:10:53Z

test/jdk/java/nio/Buffer/BasicChar.java

@@ -527,18 +527,6 @@ private static void checkSlice(CharBuffer b, CharBuffer slice) {



-


These white space changes come from re-generating the test files using the script included in the repo. I've kept the changes so that the next contributor won't have to remove them again.

mlbridge · 2023-11-16T19:23:43Z

Webrevs

JornVernee · 2023-11-27T19:42:13Z

Anything else needed to move this forward? I think I've addressed all the review comments made so far.

bridgekeeper · 2023-12-26T03:05:59Z

@JornVernee This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bridgekeeper · 2024-01-23T09:57:00Z

@JornVernee This pull request has been inactive for more than 8 weeks and will now be automatically closed. If you would like to continue working on this pull request in the future, feel free to reopen it! This can be done using the /open pull request command.

JornVernee · 2024-01-23T10:16:04Z

/open

openjdk · 2024-01-23T10:16:47Z

@JornVernee This pull request is now open

liach · 2024-01-23T14:43:24Z

Also curious, is there any documentation (like JVMS) that allows JVMs to make no offset into byte arrays aligned for larger access? Would be helpful if we can have a reference here.

JornVernee · 2024-01-23T18:34:58Z

Also curious, is there any documentation (like JVMS) that allows JVMs to make no offset into byte arrays aligned for larger access? Would be helpful if we can have a reference here.

There is simply no guarantee in the JVMS that array elements are aligned more than their size. So, the public API may not assume that they are, since it needs to be implementable by an arbitrary JVM that is JVMS compliant.

openjdk · 2024-02-13T03:41:17Z

@JornVernee This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8318966: Some methods make promises about Java array element alignment that are too strong

Reviewed-by: psandoz, mcimadamore

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 2 new commits pushed to the master branch:

7cd25ed: 8322854: Incorrect rematerialization of scalar replaced objects in C2
7ec2bad: 8323520: Drop unnecessary virtual specifier in Space

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

JornVernee · 2024-02-13T14:15:39Z

I'll do one more batch of testing (after merging with the latest master) and then integrate

JornVernee · 2024-02-14T14:30:10Z

/integrate

openjdk · 2024-02-14T14:30:56Z

Going to push as commit 9c852df.
Since your change was applied there have been 20 commits pushed to the master branch:

737b4c5: 8323883: JFR AssertionError: Missing object ID 15101
61f2493: 8325767: Serial: Move transform_stack_chunk out of TenuredGeneration::promote
8dc5976: 8325809: JFR: Remove unnecessary annotation
84965ea: 8322630: Remove ICStubs and related safepoints
0c2def0: 8325653: Erroneous exhaustivity analysis for primitive patterns
d003996: 8325743: test/jdk/java/nio/channels/unixdomain/SocketOptions.java enhance user name output in error case
ea98de6: 8325449: [BACKOUT] use "dmb.ishst+dmb.ishld" for release barrier
7f6bb71: 8319799: Recursive lightweight locking: x86 implementation
ea41932: 8325395: Missing copyright header in StackFilter.java
8765b17: 8325800: Drop unused cups declaration from Oracle build configuration
... and 10 more: https://git.openjdk.org/jdk/compare/71ff2d717798f1f314b97d97dfbc2b859fb47ae3...master

Your commit was automatically rebased without conflicts.

openjdk · 2024-02-14T14:31:01Z

@JornVernee Pushed as commit 9c852df.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

JornVernee added 5 commits November 15, 2023 19:05

fix BasicByte test

8f66397

regen test files

b8c405c

fix BAVV and BBVV impl and tests

d5ca6fa

restore some tests for direct buffers

963ee63

Use ISE for bbvvh instead of UOE

c22f7a0

openjdk bot added nio nio-dev@openjdk.org core-libs core-libs-dev@openjdk.org labels Nov 15, 2023

JornVernee changed the title ~~8318966: Some methods make promises about Java array element alinment that are too strong~~ 8318966: Some methods make promises about Java array element alignment that are too strong Nov 15, 2023

AlanBateman reviewed Nov 16, 2023

View reviewed changes

src/java.base/share/classes/java/nio/X-Buffer.java.template Show resolved Hide resolved

updated alignedOffset implNote

aba61d6

openjdk bot added the csr Pull request needs approved CSR before integration label Nov 16, 2023

updated alignedSlice implNote as well

e46e4c5

PaulSandoz reviewed Nov 16, 2023

View reviewed changes

JornVernee added 3 commits November 16, 2023 19:13

Merge note about misaligned access in byteBufferViewVarHandle

827907e

simplify spec for alignmentOffset and alignedSlice

5d09668

Add api note pointing to alternatives for client that require non-pla…

d74f47f

…in access

JornVernee commented Nov 16, 2023

View reviewed changes

JornVernee marked this pull request as ready for review November 16, 2023 19:17

openjdk bot added the rfr Pull request is ready for review label Nov 16, 2023

bridgekeeper bot closed this Jan 23, 2024

openjdk bot reopened this Jan 23, 2024

PaulSandoz approved these changes Feb 1, 2024

View reviewed changes

Merge branch 'master' into AlignedOffset

eccf4f4

mcimadamore approved these changes Feb 1, 2024

View reviewed changes

openjdk bot added ready Pull request is ready to be integrated and removed csr Pull request needs approved CSR before integration labels Feb 13, 2024

Merge branch 'master' into AlignedOffset

281deaa

openjdk bot added the integrated Pull request has been integrated label Feb 14, 2024

openjdk bot closed this Feb 14, 2024

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Feb 14, 2024

JornVernee deleted the AlignedOffset branch February 14, 2024 14:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8318966: Some methods make promises about Java array element alignment that are too strong #16681

8318966: Some methods make promises about Java array element alignment that are too strong #16681

JornVernee commented Nov 15, 2023 •

edited by openjdk bot

Loading

bridgekeeper bot commented Nov 15, 2023

openjdk bot commented Nov 15, 2023

PaulSandoz Nov 16, 2023

JornVernee Nov 16, 2023

liach Jan 23, 2024

JornVernee Jan 23, 2024 •

edited

Loading

liach Jan 23, 2024

PaulSandoz Nov 16, 2023

mcimadamore Nov 16, 2023

JornVernee Nov 16, 2023 •

edited

Loading

mcimadamore Nov 16, 2023

PaulSandoz Nov 16, 2023

JornVernee Nov 16, 2023

JornVernee Nov 16, 2023

JornVernee Nov 16, 2023 •

edited

Loading

mlbridge bot commented Nov 16, 2023 •

edited

Loading

JornVernee commented Nov 27, 2023

bridgekeeper bot commented Dec 26, 2023

bridgekeeper bot commented Jan 23, 2024

JornVernee commented Jan 23, 2024

openjdk bot commented Jan 23, 2024

liach commented Jan 23, 2024

JornVernee commented Jan 23, 2024 •

edited

Loading

openjdk bot commented Feb 13, 2024 •

edited

Loading

JornVernee commented Feb 13, 2024

JornVernee commented Feb 14, 2024

openjdk bot commented Feb 14, 2024

openjdk bot commented Feb 14, 2024

	* The supported component types (variables types) are {@code short},
	* {@code char}, {@code int}, {@code long}, {@code float} and
	* {@code double}.

		@@ -527,18 +527,6 @@ private static void checkSlice(CharBuffer b, CharBuffer slice) {

8318966: Some methods make promises about Java array element alignment that are too strong #16681

8318966: Some methods make promises about Java array element alignment that are too strong #16681

Conversation

JornVernee commented Nov 15, 2023 • edited by openjdk bot Loading

Progress

Issues

Reviewers

Reviewing

Webrev

bridgekeeper bot commented Nov 15, 2023

openjdk bot commented Nov 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JornVernee Jan 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JornVernee Nov 16, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JornVernee Nov 16, 2023 • edited Loading

Choose a reason for hiding this comment

mlbridge bot commented Nov 16, 2023 • edited Loading

Webrevs

JornVernee commented Nov 27, 2023

bridgekeeper bot commented Dec 26, 2023

bridgekeeper bot commented Jan 23, 2024

JornVernee commented Jan 23, 2024

openjdk bot commented Jan 23, 2024

liach commented Jan 23, 2024

JornVernee commented Jan 23, 2024 • edited Loading

openjdk bot commented Feb 13, 2024 • edited Loading

JornVernee commented Feb 13, 2024

JornVernee commented Feb 14, 2024

openjdk bot commented Feb 14, 2024

openjdk bot commented Feb 14, 2024

JornVernee commented Nov 15, 2023 •

edited by openjdk bot

Loading

JornVernee Jan 23, 2024 •

edited

Loading

JornVernee Nov 16, 2023 •

edited

Loading

JornVernee Nov 16, 2023 •

edited

Loading

mlbridge bot commented Nov 16, 2023 •

edited

Loading

JornVernee commented Jan 23, 2024 •

edited

Loading

openjdk bot commented Feb 13, 2024 •

edited

Loading