8321786: SegmentAllocator:allocateFrom(ValueLayout, MemorySegment,ValueLayout,long,long) spec mismatch in exception scenario #17079

minborg · 2023-12-12T10:02:00Z

This PR proposes to change the specification for some methods that take long offsets so that they will throw an IllegalArgumentException rather than an IndexOutOfBoundsException for negative values.

The PR also proposes to fix a bug where the allocation size would overflow and the specified exception was not thrown.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Change requires CSR request JDK-8321923 to be approved
Commit message must refer to an issue

Issues

JDK-8321786: SegmentAllocator:allocateFrom(ValueLayout, MemorySegment,ValueLayout,long,long) spec mismatch in exception scenario (Bug - P3)
JDK-8321923: SegmentAllocator:allocateFrom(ValueLayout, MemorySegment,ValueLayout,long,long) spec mismatch in exception scenario (CSR)

Reviewers

Maurizio Cimadamore (@mcimadamore - Reviewer) ⚠️ Review applies to c1407f77

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/17079/head:pull/17079
$ git checkout pull/17079

Update a local copy of the PR:
$ git checkout pull/17079
$ git pull https://git.openjdk.org/jdk.git pull/17079/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 17079

View PR using the GUI difftool:
$ git pr show -t 17079

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/17079.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2023-12-12T10:03:33Z

👋 Welcome back pminborg! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

minborg · 2023-12-12T10:04:35Z

/csr

openjdk · 2023-12-12T10:05:55Z

@minborg has indicated that a compatibility and specification (CSR) request is needed for this pull request.

@minborg please create a CSR request for issue JDK-8321786 with the correct fix version. This pull request cannot be integrated until the CSR request is approved.

openjdk · 2023-12-12T10:06:37Z

@minborg The following label will be automatically applied to this pull request:

core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2023-12-12T10:09:44Z

Webrevs

src/java.base/share/classes/java/lang/foreign/SegmentAllocator.java

src/java.base/share/classes/jdk/internal/foreign/layout/AbstractLayout.java

mcimadamore · 2023-12-12T13:52:10Z

src/java.base/share/classes/java/lang/foreign/MemorySegment.java

     * @throws IllegalArgumentException if {@code layout.byteAlignment() > layout.byteSize()}
     * @throws IndexOutOfBoundsException if {@code index * layout.byteSize()} overflows
     * @throws IndexOutOfBoundsException if {@code index * layout.byteSize() > byteSize() - layout.byteSize()}
+     *         or {@code offset < 0}


All these should say index not offset

mcimadamore · 2023-12-12T13:58:42Z

src/java.base/share/classes/java/lang/foreign/SegmentAllocator.java

-     * @throws IndexOutOfBoundsException if {@code elementCount * sourceElementLayout.byteSize()} overflows
+     * @throws IllegalArgumentException if {@code elementCount * sourceElementLayout.byteSize()} overflows
     * @throws IndexOutOfBoundsException if {@code sourceOffset > source.byteSize() - (elementCount * sourceElementLayout.byteSize())}
     * @throws IndexOutOfBoundsException if either {@code sourceOffset} or {@code elementCount} are {@code < 0}


I think that if elementCount < 0, we should throw IAE, not IOOBE. Note that we explain this method as:

MemorySegment dest = this.allocate(elementLayout, elementCount); MemorySegment.copy(source, sourceElementLayout, sourceOffset, dest, elementLayout, 0, elementCount);

So, the set of exceptions thrown by these two methods should be preserved. This means that allocate(MemoryLayout, long) gets a first pass at checking - this means IAE for overflows and also for negative element count.

After that, the exceptions we can have are those specified in MemorySegment.copy.

(as we did in other patches, I think it's important here to remain true to the semantics of the "desugared" code, because that will guarantee that developers can refactor their code w/o worrying about exceptions changing from under them)

mcimadamore · 2023-12-12T14:00:19Z

src/java.base/share/classes/java/lang/foreign/SegmentAllocator.java

        Objects.requireNonNull(source);
        Objects.requireNonNull(sourceElementLayout);
        Objects.requireNonNull(elementLayout);
+        Utils.checkNonNegativeIndex(elementCount, "elementCount");


I think this check should be omitted (see comment above)

mcimadamore · 2023-12-12T14:01:49Z

src/java.base/share/classes/jdk/internal/foreign/AbstractMemorySegmentImpl.java

                            MemorySegment dstSegment, ValueLayout dstElementLayout, long dstOffset,
                            long elementCount) {

+        Utils.checkNonNegativeIndex(srcOffset, "srcOffset");


These new checks don't change the behavior, right? E.g. they will end up issuing an IIOBE, as before? So, why the changes? (note that these change might lead to double checks)

It seems like this is about generating better exception (while leaving the exception type unchanged). If so, I support the change, as I agree that this patch does improve some of the messages.

mcimadamore · 2023-12-12T14:02:28Z

src/java.base/share/classes/jdk/internal/foreign/Utils.java

        }
    }

+    public static void checkNonNegativeArgument(long value, String name) {


Add @ForceInline to these

mcimadamore · 2023-12-12T14:04:49Z

test/jdk/java/foreign/TestSegmentAllocators.java

+
+            // IllegalStateException if the {@linkplain MemorySegment#scope() scope} associated
+            // with {@code source} is not {@linkplain MemorySegment.Scope#isAlive() alive}
+            Arena closedArena = Arena.ofConfined();


the scope/thread assertions are already checked in TestScopedOperations - I don't think we need this, but you might want to make sure a test case for this new allocator method is added there

mcimadamore · 2023-12-12T14:05:32Z

test/jdk/java/foreign/TestSegmentAllocators.java

+                fail("Unable to create arena", e);
+            }
+
+            // ArithmeticException if {@code elementCount * sourceElementLayout.byteSize()} overflows


comment is wrong

mcimadamore · 2023-12-12T14:07:00Z

test/jdk/java/foreign/TestSegments.java

    }

+    @Test(expectedExceptions = IndexOutOfBoundsException.class)
+    public void testNegativeGetOffset() {


We have a test specifically for instance segment accessors (TestMemoryAccessInstance) which covers all accessors - maybe better to move these new tests there?

mcimadamore · 2023-12-12T15:18:40Z

src/java.base/share/classes/java/lang/foreign/SegmentAllocator.java

     * @throws WrongThreadException if this method is called from a thread {@code T},
     *         such that {@code source.isAccessibleBy(T) == false}
-     * @throws IndexOutOfBoundsException if {@code elementCount * sourceElementLayout.byteSize()} overflows
+     * @throws IllegalArgumentException if {@code elementCount * sourceElementLayout.byteSize()} overflows


if elementCount < 0 -> IAE, as per allocate(layout, size)

mcimadamore · 2023-12-12T15:39:11Z

src/java.base/share/classes/java/lang/foreign/SegmentAllocator.java

     * @throws IndexOutOfBoundsException if {@code sourceOffset > source.byteSize() - (elementCount * sourceElementLayout.byteSize())}
-     * @throws IndexOutOfBoundsException if either {@code sourceOffset} or {@code elementCount} are {@code < 0}
+     * @throws IndexOutOfBoundsException if {@code sourceOffset < 0}
+     * @throws IllegalArgumentException if {@code elementCount < 0}


Please move this closer to the other IAE

turbanoff · 2023-12-20T10:37:34Z

test/jdk/java/foreign/TestSegmentCopy.java

+        MemorySegment src = MemorySegment.ofArray(new byte[] {1, 2, 3, 4});
+        byte[] dst = new byte[] {1, 2, 3, 4};
+        assertThrows(IndexOutOfBoundsException.class, () ->
+                MemorySegment.copy(src, JAVA_BYTE, -1, dst,0, 4)


Suggested change

MemorySegment.copy(src, JAVA_BYTE, -1, dst,0, 4)

MemorySegment.copy(src, JAVA_BYTE, -1, dst, 0, 4)

turbanoff · 2023-12-20T10:37:48Z

test/jdk/java/foreign/TestSegmentCopy.java

+                MemorySegment.copy(src, JAVA_BYTE, -1, dst,0, 4)
+        );
+        assertThrows(IndexOutOfBoundsException.class, () ->
+                MemorySegment.copy(src, JAVA_BYTE, 0, dst,-1, 4)


Suggested change

MemorySegment.copy(src, JAVA_BYTE, 0, dst,-1, 4)

MemorySegment.copy(src, JAVA_BYTE, 0, dst, -1, 4)

turbanoff · 2023-12-20T10:38:00Z

test/jdk/java/foreign/TestSegmentCopy.java

+                MemorySegment.copy(src, JAVA_BYTE, 0, dst,-1, 4)
+        );
+        assertThrows(IndexOutOfBoundsException.class, () ->
+                MemorySegment.copy(src, JAVA_BYTE, 0, dst,0, -1)


Suggested change

MemorySegment.copy(src, JAVA_BYTE, 0, dst,0, -1)

MemorySegment.copy(src, JAVA_BYTE, 0, dst, 0, -1)

openjdk · 2023-12-20T23:49:49Z

@minborg This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8321786: SegmentAllocator:allocateFrom(ValueLayout, MemorySegment,ValueLayout,long,long) spec mismatch in exception scenario

Reviewed-by: mcimadamore

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

minborg · 2024-01-08T08:18:30Z

/integrate

openjdk · 2024-01-08T08:20:09Z

Going to push as commit 7edd10e.
Since your change was applied there have been 18 commits pushed to the master branch:

d75d876: 8322806: Eliminate -Wparentheses warnings in aarch64 code
e442769: 8322754: click JComboBox when dialog about to close causes IllegalComponentStateException
3560e68: 8322815: Eliminate -Wparentheses warnings in shenandoah code
faa9c69: 8322846: Running with -Djdk.tracePinnedThreads set can hang
ace010b: 8319757: java/nio/channels/DatagramChannel/InterruptibleOrNot.java failed: wrong exception thrown
be4614e: 8323016: Improve reporting for bad options
35a1b77: 8322636: [JVMCI] HotSpotSpeculationLog can be inconsistent across a single compile
46965a0: 8322981: Fix 2 locations in JDI that throw IOException without using the "Caused by" exception
700c25f: 8322954: Shenandoah: Convert evac-update closures asserts to rich asserts
631a9f6: 8323073: ProblemList gc/g1/TestSkipRebuildRemsetPhase.java on linux-aarch64
... and 8 more: https://git.openjdk.org/jdk/compare/2a9c3589d941d9a57e536ea0b3d7919c6ddb82dc...master

Your commit was automatically rebased without conflicts.

openjdk · 2024-01-08T08:20:15Z

@minborg Pushed as commit 7edd10e.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

minborg · 2024-01-08T15:31:06Z

/backport jdk22

openjdk · 2024-01-08T15:32:43Z

@minborg the backport was successfully created on the branch backport-minborg-7edd10e5 in my personal fork of openjdk/jdk22. To create a pull request with this backport targeting openjdk/jdk22:master, just click the following link:

➡️ Create pull request

The title of the pull request is automatically filled in correctly and below you find a suggestion for the pull request body:

Hi all,

This pull request contains a backport of commit 7edd10e5 from the openjdk/jdk repository.

The commit being backported was authored by Per Minborg on 8 Jan 2024 and was reviewed by Maurizio Cimadamore.

Thanks!

If you need to update the source branch of the pull then run the following commands in a local clone of your personal fork of openjdk/jdk22:

$ git fetch https://github.com/openjdk-bots/jdk22.git backport-minborg-7edd10e5:backport-minborg-7edd10e5
$ git checkout backport-minborg-7edd10e5
# make changes
$ git add paths/to/changed/files
$ git commit --message 'Describe additional changes made'
$ git push https://github.com/openjdk-bots/jdk22.git backport-minborg-7edd10e5

Change the exception thrown for certain negative long values

27fc39f

openjdk bot added rfr Pull request is ready for review csr Pull request needs approved CSR before integration labels Dec 12, 2023

openjdk bot added the core-libs core-libs-dev@openjdk.org label Dec 12, 2023

mcimadamore reviewed Dec 12, 2023

View reviewed changes

src/java.base/share/classes/java/lang/foreign/SegmentAllocator.java Outdated Show resolved Hide resolved

mcimadamore reviewed Dec 12, 2023

View reviewed changes

src/java.base/share/classes/jdk/internal/foreign/layout/AbstractLayout.java Outdated Show resolved Hide resolved

minborg added 3 commits December 12, 2023 14:07

Update after comments and discussion

28a3393

Remove Math.multiply exact and change exception type

f601b2f

Revert change in allocateNoInit

71b2173

mcimadamore reviewed Dec 12, 2023

View reviewed changes

Update after comments

d67e079

mcimadamore reviewed Dec 12, 2023

View reviewed changes

Correct exception type in JavaDocs for SA:allocateFrom

c1407f7

mcimadamore reviewed Dec 12, 2023

View reviewed changes

mcimadamore approved these changes Dec 12, 2023

View reviewed changes

Move @throws tag

115b6ea

turbanoff reviewed Dec 20, 2023

View reviewed changes

openjdk bot added ready Pull request is ready to be integrated and removed csr Pull request needs approved CSR before integration labels Dec 20, 2023

minborg added 3 commits January 5, 2024 09:16

Fix formatting in test

6709194

Fix docs for read-only exceptions

4d7c58a

Merge branch 'master' into throw-change2

7d5472c

openjdk bot added the integrated Pull request has been integrated label Jan 8, 2024

openjdk bot closed this Jan 8, 2024

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jan 8, 2024

minborg mentioned this pull request Jan 8, 2024

8321387: SegmentAllocator:allocateFrom(AddressLayout, MemorySegment) does not throw stated UnsupportedOperationException openjdk/jdk22#41

Closed

3 tasks

	MemorySegment.copy(src, JAVA_BYTE, -1, dst,0, 4)
	MemorySegment.copy(src, JAVA_BYTE, -1, dst, 0, 4)

	MemorySegment.copy(src, JAVA_BYTE, 0, dst,-1, 4)
	MemorySegment.copy(src, JAVA_BYTE, 0, dst, -1, 4)

8321786: SegmentAllocator:allocateFrom(ValueLayout, MemorySegment,ValueLayout,long,long) spec mismatch in exception scenario #17079

8321786: SegmentAllocator:allocateFrom(ValueLayout, MemorySegment,ValueLayout,long,long) spec mismatch in exception scenario #17079

Uh oh!

Conversation

minborg commented Dec 12, 2023 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issues

Reviewers

Reviewing

Webrev

Uh oh!

bridgekeeper bot commented Dec 12, 2023

Uh oh!

minborg commented Dec 12, 2023

Uh oh!

openjdk bot commented Dec 12, 2023

Uh oh!

openjdk bot commented Dec 12, 2023

Uh oh!

mlbridge bot commented Dec 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcimadamore Dec 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcimadamore Dec 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcimadamore Dec 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openjdk bot commented Dec 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

minborg commented Jan 8, 2024

Uh oh!

openjdk bot commented Jan 8, 2024

Uh oh!

openjdk bot commented Jan 8, 2024

Uh oh!

minborg commented Jan 8, 2024

Uh oh!

openjdk bot commented Jan 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

minborg commented Dec 12, 2023 •

edited by openjdk bot

Loading

mlbridge bot commented Dec 12, 2023 •

edited

Loading

mcimadamore Dec 12, 2023 •

edited

Loading

mcimadamore Dec 12, 2023 •

edited

Loading

mcimadamore Dec 12, 2023 •

edited

Loading

openjdk bot commented Dec 20, 2023 •

edited

Loading