8314236: Overflow in Collections.rotate #15270

nikita-sakharin · 2023-08-14T12:56:46Z

Collections.rotate method contains a bug. This method throws IndexOutOfBoundsException on arrays larger than $2^{30}$ elements. The way to reproduce:

final int size = (1 << 30) + 1;
final List<Byte> list = new ArrayList<>(size);
for (int i = 0; i < size; ++i)
    list.add((byte) 0);
Collections.rotate(list, size - 1);

Output:
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index -2147483648 out of bounds for length 1073741825

In that case private method Collections.rotate1 will be called. And the line:
i += distance;
will cause overflow. I fixed this method and wrote a test for it.

I've signed the Oracle Contributor Agreement, but I don't have permission to raise a bug in the JDK Bug System.

Kindly ask you to raise a bug.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8314236: Overflow in Collections.rotate (Bug - P4)

Reviewers

Aleksey Shipilev (@shipilev - Reviewer)
Stuart Marks (@stuart-marks - Reviewer)

Contributors

Nikita Sakharin <17588081+nikita-sakharin@users.noreply.github.com>

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15270/head:pull/15270
$ git checkout pull/15270

Update a local copy of the PR:
$ git checkout pull/15270
$ git pull https://git.openjdk.org/jdk.git pull/15270/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15270

View PR using the GUI difftool:
$ git pr show -t 15270

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15270.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2023-08-14T12:57:41Z

👋 Welcome back nikita-sakharin! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2023-08-14T12:59:54Z

@nikita-sakharin The following label will be automatically applied to this pull request:

core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

shipilev · 2023-08-14T20:23:13Z

Submitted: JDK-8314236

Please change the PR synopsis to: "8314236: Overflow in Collections.rotate".

Also go to https://github.com/nikita-sakharin/jdk/actions, and enable testing workflows.

mlbridge · 2023-08-14T21:07:42Z

Webrevs

openjdk · 2023-08-14T21:10:09Z

⚠️ @nikita-sakharin a branch with the same name as the source branch for this pull request (pr/15270) is present in the target repository. If you eventually integrate this pull request then the branch pr/15270 in your personal fork will diverge once you sync your personal fork with the upstream repository.

To avoid this situation, create a new branch for your changes and reset the pr/15270 branch. You can do this by running the following commands in a local repository for your personal fork. Note: you do not have to name the new branch NEW-BRANCH-NAME.

$ git checkout -b NEW-BRANCH-NAME
$ git branch -f pr/15270 207bd00c5101fce06b5ac12e76893d989b0093e2
$ git push -f origin pr/15270

Then proceed to create a new pull request with NEW-BRANCH-NAME as the source branch and close this one.

shipilev

Superficially, this looks okay. Some OpenJDK code/test style trivia:

test/jdk/java/util/Collections/RotateHuge.java

src/java.base/share/classes/java/util/Collections.java

stuart-marks · 2023-08-24T02:36:52Z

@nikita-sakharin

Thanks for the updates. With the "Mock List" implementation we can run the test in-JVM and we can avoid allocating several GB of memory. Great!

The implementation logic in the rotate1 method looks correct.

Now that an individual test case is much less expensive, it becomes feasible to add multiple test cases. In particular, for this kind of testing of arithmetic errors, I like to test a variety of edge cases. The one test case you have is for size=(1 << 30) - 1 and distance=(1 << 30). It would be good to have a few other cases where the existing code fails and where the modified code should pass. I was able to come up with a few examples quickly:

size                 distance
Integer.MAX_VALUE    2
Integer.MAX_VALUE    Integer.MIN_VALUE
Integer.MAX_VALUE    Integer.MAX_VALUE - 1

Please add these cases, and any others that you think might be interesting.

openjdk · 2023-08-25T13:15:26Z

@nikita-sakharin Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information.

nikita-sakharin · 2023-08-25T13:44:56Z

@stuart-marks

Thank you for your review!

Tests were failing due to JDK-8313701. So I pulled changes from the master branch and squashed my commits to one.

I increased test coverage with respect to your request. All tests passed successfully.

Awaiting the next step of your review.

stuart-marks · 2023-08-29T03:02:56Z

(The bot has already admonished you about avoiding rebasing. In practice merging works well and is helpful so that reviewers can see incremental changes.)

The additional test cases look good. Sorry I forgot to mention this previously: it would be good to have a comment in the test that says that the test is only testing the index computations and not the actual rotation of elements.

Meanwhile, I'll pull in the changes from your branch and run them through our internal build system, which is more reliable (and comprehensive) than GHA.

nikita-sakharin · 2023-08-29T20:01:21Z

@stuart-marks

Thank you for your review!

I added comment for the test.

stuart-marks

Thanks for the update. Test run looks good! Approved.

openjdk · 2023-08-29T22:15:34Z

@nikita-sakharin This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8314236: Overflow in Collections.rotate

Co-authored-by: Nikita Sakharin <17588081+nikita-sakharin@users.noreply.github.com>
Reviewed-by: shade, smarks

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 250 new commits pushed to the master branch:

eb1f67b: 8315937: Enable parallelism in vmTestbase/nsk/stress/numeric tests
a57b9da: 8316242: Opensource SwingGraphics manual test
3abe798: 8316115: Parallel: Fix -Wconversion warnings around NUMA node ID
83dca62: 8313638: Add test for dump of resolved references
cfa8901: 8315338: RISC-V: Change flags for stable extensions to non-experimental
edd454b: 8315766: Parallelize gc/stress/TestStressIHOPMultiThread.java test
de9b971: 8315933: Serial: Remove empty Generation::ensure_parsability
b48dbf6: 8316181: Move the fast locking implementation out of the .ad files
8f4dfc4: 8316164: Opensource JMenuBar manual test
33c62e4: 8316154: Opensource JTextArea manual tests
... and 240 more: https://git.openjdk.org/jdk/compare/dcd6e756718b656d43f4575558f41ce0c28d0eca...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@shipilev, @stuart-marks) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

nikita-sakharin · 2023-08-30T08:20:58Z

@shipilev

Stuart Marks has approved my changes. I am awaiting your feedback since you are reviewer for that PR too.

Could I kindly ask you to review the PR, please?

nikita-sakharin · 2023-09-02T13:27:34Z

/integrate

openjdk · 2023-09-02T13:27:47Z

@nikita-sakharin @nikita-sakharin was not found in the census.

Syntax: /contributor (add|remove) [@user | openjdk-user | Full Name <email@address>]. For example:

/contributor add @openjdk-bot
/contributor add duke
/contributor add J. Duke <duke@openjdk.org>

User names can only be used for users in the census associated with this repository. For other contributors you need to supply the full name and email address.

openjdk · 2023-09-02T13:28:05Z

@nikita-sakharin
Your change (at version 2bbcffa) is now ready to be sponsored by a Committer.

nikita-sakharin · 2023-09-02T13:41:38Z

/contributor add Nikita Sakharin 17588081+nikita-sakharin@users.noreply.github.com

openjdk · 2023-09-02T13:42:45Z

@nikita-sakharin
Contributor Nikita Sakharin <17588081+nikita-sakharin@users.noreply.github.com> successfully added.

nikita-sakharin · 2023-09-13T17:26:52Z

Aleksey Shipilëv (@shipilev), kindly remind you about the PR.

Could I ask you to review it, please?

shipilev

OK, looks reasonable.

nikita-sakharin · 2023-09-15T12:33:50Z

Aleksey Shipilëv (@shipilev), Stuart Marks (@stuart-marks), thank you for your approvals. Since both reviewers approved the PR, it is ready to be integrated.

I have already executed /integrate command. Now I am awaiting one of you to execute /sponsor command. So I kindly ask you to do that.

shipilev · 2023-09-15T14:37:54Z

I would like to push this early next week.

shipilev · 2023-09-18T09:18:35Z

/sponsor

openjdk · 2023-09-18T09:20:14Z

Going to push as commit 3828dc9.
Since your change was applied there have been 289 commits pushed to the master branch:

1203e11: 8294969: Convert jdk.jdeps javap to use the Classfile API
fbc766e: 8315988: Parallel: Make TestAggressiveHeap use createTestJvm
aa0ebee: 8316341: sun/security/pkcs11/PKCS11Test.java needs adjustment on Linux ppc64le Ubuntu 22
ca3fe7b: 8315932: runtime/InvocationTests spend a lot of time on dependency verification
f440329: 8316391: (zipfs) ZipFileSystem.readFullyAt does not tolerate short reads
4b8f5d0: 8316273: JDK-8315818 broke JVMCIPrintProperties on libgraal
e624198: 8316387: Exclude more failing multicast tests on AIX after JDK-8315651
c92bdb0: 8316187: Modernize examples in StringTokenizer and {Date,Number}Format
8f46abc: 8315889: Open source several Swing HTMLDocument related tests
0050447: 8316148: Remove sun/tools/jhsdb/JStackStressTest.java from problem list
... and 279 more: https://git.openjdk.org/jdk/compare/dcd6e756718b656d43f4575558f41ce0c28d0eca...master

Your commit was automatically rebased without conflicts.

openjdk · 2023-09-18T09:20:26Z

@shipilev @nikita-sakharin Pushed as commit 3828dc9.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

nikita-sakharin changed the title ~~83141345: fix overflow in Collections.rotate~~ 83141345: Fix overflow in Collections.rotate Aug 14, 2023

openjdk bot added the core-libs core-libs-dev@openjdk.org label Aug 14, 2023

nikita-sakharin changed the title ~~83141345: Fix overflow in Collections.rotate~~ 8314236: Overflow in Collections.rotate Aug 14, 2023

openjdk bot added the rfr Pull request is ready for review label Aug 14, 2023

shipilev reviewed Aug 15, 2023

View reviewed changes

nikita-sakharin requested review from shipilev and stuart-marks August 15, 2023 15:06

fix overflow in Collections.rotate and write tests

a7d7722

8314236: add comment for test

2bbcffa

stuart-marks approved these changes Aug 29, 2023

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Aug 29, 2023

openjdk bot added the sponsor Pull request is ready to be sponsored label Sep 2, 2023

shipilev approved these changes Sep 14, 2023

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Sep 18, 2023

openjdk bot closed this Sep 18, 2023

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review sponsor Pull request is ready to be sponsored labels Sep 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8314236: Overflow in Collections.rotate #15270

8314236: Overflow in Collections.rotate #15270

nikita-sakharin commented Aug 14, 2023 •

edited by openjdk bot

Loading

bridgekeeper bot commented Aug 14, 2023

openjdk bot commented Aug 14, 2023

shipilev commented Aug 14, 2023

mlbridge bot commented Aug 14, 2023 •

edited

Loading

openjdk bot commented Aug 14, 2023

shipilev left a comment

stuart-marks commented Aug 24, 2023

openjdk bot commented Aug 25, 2023

nikita-sakharin commented Aug 25, 2023 •

edited

Loading

stuart-marks commented Aug 29, 2023

nikita-sakharin commented Aug 29, 2023

stuart-marks left a comment

openjdk bot commented Aug 29, 2023 •

edited

Loading

nikita-sakharin commented Aug 30, 2023 •

edited

Loading

nikita-sakharin commented Sep 2, 2023

openjdk bot commented Sep 2, 2023

openjdk bot commented Sep 2, 2023

nikita-sakharin commented Sep 2, 2023

openjdk bot commented Sep 2, 2023

nikita-sakharin commented Sep 13, 2023 •

edited

Loading

shipilev left a comment

nikita-sakharin commented Sep 15, 2023 •

edited

Loading

shipilev commented Sep 15, 2023

shipilev commented Sep 18, 2023

openjdk bot commented Sep 18, 2023

openjdk bot commented Sep 18, 2023

8314236: Overflow in Collections.rotate #15270

8314236: Overflow in Collections.rotate #15270

Conversation

nikita-sakharin commented Aug 14, 2023 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Contributors

Reviewing

Webrev

bridgekeeper bot commented Aug 14, 2023

openjdk bot commented Aug 14, 2023

shipilev commented Aug 14, 2023

mlbridge bot commented Aug 14, 2023 • edited Loading

Webrevs

openjdk bot commented Aug 14, 2023

shipilev left a comment

Choose a reason for hiding this comment

stuart-marks commented Aug 24, 2023

openjdk bot commented Aug 25, 2023

nikita-sakharin commented Aug 25, 2023 • edited Loading

stuart-marks commented Aug 29, 2023

nikita-sakharin commented Aug 29, 2023

stuart-marks left a comment

Choose a reason for hiding this comment

openjdk bot commented Aug 29, 2023 • edited Loading

nikita-sakharin commented Aug 30, 2023 • edited Loading

nikita-sakharin commented Sep 2, 2023

openjdk bot commented Sep 2, 2023

openjdk bot commented Sep 2, 2023

nikita-sakharin commented Sep 2, 2023

openjdk bot commented Sep 2, 2023

nikita-sakharin commented Sep 13, 2023 • edited Loading

shipilev left a comment

Choose a reason for hiding this comment

nikita-sakharin commented Sep 15, 2023 • edited Loading

shipilev commented Sep 15, 2023

shipilev commented Sep 18, 2023

openjdk bot commented Sep 18, 2023

openjdk bot commented Sep 18, 2023

nikita-sakharin commented Aug 14, 2023 •

edited by openjdk bot

Loading

mlbridge bot commented Aug 14, 2023 •

edited

Loading

nikita-sakharin commented Aug 25, 2023 •

edited

Loading

openjdk bot commented Aug 29, 2023 •

edited

Loading

nikita-sakharin commented Aug 30, 2023 •

edited

Loading

nikita-sakharin commented Sep 13, 2023 •

edited

Loading

nikita-sakharin commented Sep 15, 2023 •

edited

Loading