6323374: (coll) Optimize Collections.unmodifiable* and synchronized* #2596

igraves · 2021-02-16T21:57:43Z

Modify the unmodifiable* methods in java.util.Collections to be idempotent. That is, when given an immutable collection from java.util.ImmutableCollections or java.util.Collections, these methods will return the reference instead of creating a new immutable collection that wraps the existing one.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-6323374: (coll) Optimize Collections.unmodifiable* and synchronized*

Reviewers

Claes Redestad (@cl4es - Reviewer) ⚠️ Review applies to ebdd0f8
Stuart Marks (@stuart-marks - Reviewer) ⚠️ Review applies to 88127ed
Joe Darcy (@jddarcy - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jdk pull/2596/head:pull/2596
$ git checkout pull/2596

bridgekeeper · 2021-02-16T21:57:53Z

👋 Welcome back igraves! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

igraves · 2021-02-16T21:57:53Z

/csr

openjdk · 2021-02-16T21:58:50Z

@igraves this pull request will not be integrated until the CSR request JDK-8261677 for issue JDK-6323374 has been approved.

openjdk · 2021-02-16T21:59:53Z

@igraves The following label will be automatically applied to this pull request:

core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2021-02-16T22:03:02Z

Webrevs

RogerRiggs

Please add a space after if -> if .

src/java.base/share/classes/java/util/Collections.java

liach

In addition year for license of Collections.java needs an update too

cl4es

This looks good to me.

jddarcy · 2021-02-23T06:12:54Z

Is there any behavior change here that merits a CSR review?

cl4es · 2021-02-23T16:18:29Z

Is there any behavior change here that merits a CSR review?

Maybe. The one observable change is that calling Collections.bar(foo) with a foo that is already a bar will return the instance rather than unnecessarily wrap it. This could change semantics in applications inadvertently or deliberately relying on identity.

igraves · 2021-02-23T16:27:06Z

Is there any behavior change here that merits a CSR review?

Maybe. The one observable change is that calling Collections.bar(foo) with a foo that is already a bar will return the instance rather than unnecessarily wrap it. This could change semantics in applications inadvertently or deliberately relying on identity.

Yes. The CSR was to consider primarily this case. Probably out of an abundance of caution here. @stuart-marks may have another case to consider.

stuart-marks · 2021-02-23T23:32:01Z

Is there any behavior change here that merits a CSR review?

Yes. See my comments in the bug report:

https://bugs.openjdk.java.net/browse/JDK-6323374?focusedCommentId=14296330&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14296330

There is not only the issue of the identity of the object returned, but the change is also observable in the serialized form. Most people would consider the change (less nesting) to be an improvement, but the change is observable, and as we know any observable behavior can become depended upon by applications.

stuart-marks · 2021-02-24T01:58:48Z

Code changes all look good. I'm thinking that we should add @implNote clauses to all the docs of the affected methods, saying something like "This method may return its argument if it is already unmodifiable." Usually it's reasonable to leave these kinds of behaviors unspecified (and we do so elsewhere) but since this is a change in long-standing behavior, it seems reasonable to highlight it explicitly. I don't think we want to specify it though, because of the issue with ImmutableCollections (as discussed previously) and possible future tuning of behavior regarding the various Set and Map subinterfaces. (For example, C.unmodifiableSet(arg) could return arg if it's an UnmodifiableNavigableSet.)

The test seems to have a lot of uncomfortable dependencies, both explicit and implicit, on the various ImmutableCollection and UnmodifiableX implementation classes. Would it be sufficient to test various instances for reference equality and inequality instead? For example, something like

var list0 = List.of();
var list1 = Collections.unmodifiableList(list0);
var list2 = Collections.unmodifiableList(list1);
assertNotSame(list0, list1);
assertSame(list1, list2);

This would avoid having to write test cases that cover various internal classes. The ImmutableCollections classes have been reorganized in the past, and while we don't have any plans to do so again at the moment, there is always the possibility of it happening again.

One could write out all the different cases "by hand" but there are rather a lot of them. It might be fruitful to extract the "wrap once, wrap again, assertNotSame, assertSame" logic into a generic test and drive it somehow with a data provider that provides the base instance and a wrapper function.

igraves · 2021-02-26T20:16:54Z

Per @stuart-marks I rewrote the tests using some of his suggestions, which substantially reduced dependencies and test size.

stuart-marks

The @implNote additions are good, and the test rewrite looks good too.

jddarcy · 2021-03-03T23:29:33Z

src/java.base/share/classes/java/util/Collections.java

     * @param  <T> the class of the objects in the set
     * @param s the sorted set for which an unmodifiable view is to be
     *        returned.
     * @return an unmodifiable view of the specified sorted set.
     */
    public static <T> SortedSet<T> unmodifiableSortedSet(SortedSet<T> s) {
+        if (s.getClass() == UnmodifiableSortedSet.class) {


Should a check like this also included "|| == UnmodifiableNavigableSet.class" or was there an explicit decision that the cost/benefit is not worthwhile, unlike in the case of unmodifiableList below?

This is a good point. The case of unmodifiableList is such because the method can return two different classes depending the nature of the argument. I feel as though if we made this change here, we should consider doing the same check for vanilla unmodifiableSet to ensure it, too, doesn't wrap its subclasses. I'm amenable to this.

To the second part of the question, there was no explicit cost/benefit analysis RE List or this case.

openjdk · 2021-03-03T23:35:11Z

@igraves This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

6323374: (coll) Optimize Collections.unmodifiable* and synchronized*

Reviewed-by: redestad, smarks, darcy

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 331 new commits pushed to the master branch:

ee09bad: 8262300: jpackage app-launcher fails on linux when using JDK11 based runtime
351889f: 8262508: Vector API's ergonomics is incorrect
718d4d4: 8262989: Vectorize VectorShuffle checkIndexes, wrapIndexes and laneIsValid methods
c8b23e2: 8262064: Make compiler/ciReplay tests ignore lambdas in compilation replay
02fbcb5: 8261532: Archived superinterface class cannot be accessed
109af7b: 8261518: jpackage looks for main module in current dir when there is no module-path
e61a3ba: 8239386: handle ContendedPaddingWidth in vm_version_aarch64
f56c918: 8262837: handle split_USE correctly
bd1a806: 8263040: fix for JDK-8262122 fails validate-source
a6427c8: 8259709: Disable SHA-1 XML Signatures
... and 321 more: https://git.openjdk.java.net/jdk/compare/48c932e1f1e5a79a28211f72dc9f10d8fd30b955...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@RogerRiggs, @cl4es, @stuart-marks, @jddarcy) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

stuart-marks · 2021-03-04T03:57:35Z

Hm. I had thought of this previously but I was a bit suspicious, and it didn't seem like it would make much difference, so I didn't say anything. But thinking about this further, the following issues arose:

Suppose you have an UnmodifiableSortedSet and call unmodifiableSet() on it, it returns its argument, and then you hand it out. Its static type is Set but it's really a SortedSet, which means somebody can downcast it and get its comparator. This leaks some information. Offhand this doesn't look dangerous, but it's a bit of a surprise.
Thinking about this further, this allows heap pollution. (Ian, turns out our conversation from the other day wasn't just an idle digression.) If we have class X and class Y extends X, then SortedSet<Y> cannot safely be cast to an unmodifiable SortedSet<X>. That's because comparator() will return Comparator<? super X> which is incorrect, since the actual comparator might be of type Comparator<Y>. Actually the headSet(), subSet(), and tailSet() methods also cause problems, because they both consume and produce the collection element type E.
This can actually happen in practice with code in the JDK! PriorityBlockingQueue's copy constructor does exactly the above. It takes a Collection and does an instanceof check to see if it's a SortedSet; if it is, it does a downcast and uses its comparator. Thus if we do the following:

    SortedSet<Integer> set1 = new TreeSet<>(Integer::compare);
    Set<Number> set2 = Collections.unmodifiableSet(set1); // hypothetical version that returns its argument
    PriorityBlockingQueue<Number> pbq = new PriorityBlockingQueue<>(set2);
    pbq.addAll(Arrays.asList(1.0, 2.0));

this compiles without warnings, but it results in ClassCastException. The culprit is the new upcast that potentially allows SortedSet<? extends T> to be cast to Set<T>, which is slipped in under the already-existing warnings suppression.

In any case, the extra checking in the unmodifiableSortedSet and -Map methods needs to be taken out. Offhand I don't know if there's a similar issue between unmodifiableSortedSet and a NavigableSet (resp., Map), but on general principle I'd say to take it out too. It's likely not buying much anyway.

The UnmodifiableList and UnmodifiableRandomAccessList stuff should stay, since that's how the RandomAccess marker interface is preserved.

igraves · 2021-03-04T04:06:10Z

Good thought on the heap pollution. I had only been considering point 1. I was thinking that perhaps if we could catch some of the wrapping of subclasses we might be able to guard against situations where rewrapping could occur if we were interleaving calls between subclass and superclass wrapper methods. That seems like a bit of a reach and in light of your point about heap pollution I think it makes sense to walk back the changes and stay with the original.

Likewise agree on the point about Lists.

This reverts commit dda174c.

jddarcy · 2021-03-04T21:37:45Z

If the checks for Navigable set and the like are omitted, I'd prefer to a comment in the sources noting this is intention as a Navigable set is-a Sorted set.

igraves · 2021-03-04T22:31:24Z

Added comments to the relevant methods noting our intention not to check for subclasses with a little note why.

igraves · 2021-03-05T02:14:47Z

/integrate

openjdk · 2021-03-05T02:15:18Z

@igraves
Your change (at version 47113ba) is now ready to be sponsored by a Committer.

stuart-marks · 2021-03-05T03:16:52Z

/sponsor

openjdk · 2021-03-05T03:21:03Z

@stuart-marks @igraves Since your change was applied there have been 331 commits pushed to the master branch:

ee09bad: 8262300: jpackage app-launcher fails on linux when using JDK11 based runtime
351889f: 8262508: Vector API's ergonomics is incorrect
718d4d4: 8262989: Vectorize VectorShuffle checkIndexes, wrapIndexes and laneIsValid methods
c8b23e2: 8262064: Make compiler/ciReplay tests ignore lambdas in compilation replay
02fbcb5: 8261532: Archived superinterface class cannot be accessed
109af7b: 8261518: jpackage looks for main module in current dir when there is no module-path
e61a3ba: 8239386: handle ContendedPaddingWidth in vm_version_aarch64
f56c918: 8262837: handle split_USE correctly
bd1a806: 8263040: fix for JDK-8262122 fails validate-source
a6427c8: 8259709: Disable SHA-1 XML Signatures
... and 321 more: https://git.openjdk.java.net/jdk/compare/48c932e1f1e5a79a28211f72dc9f10d8fd30b955...master

Your commit was automatically rebased without conflicts.

Pushed as commit dbef0ec.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Changing Collections.unmodifiable* to idempotent behavior.

e0a06bf

openjdk bot added rfr Pull request is ready for review csr Pull request needs approved CSR before integration labels Feb 16, 2021

openjdk bot added the core-libs core-libs-dev@openjdk.org label Feb 16, 2021

RogerRiggs reviewed Feb 16, 2021

View reviewed changes

cl4es suggested changes Feb 16, 2021

View reviewed changes

src/java.base/share/classes/java/util/Collections.java Outdated Show resolved Hide resolved

src/java.base/share/classes/java/util/Collections.java Outdated Show resolved Hide resolved

src/java.base/share/classes/java/util/Collections.java Outdated Show resolved Hide resolved

michaelhixson reviewed Feb 17, 2021

View reviewed changes

src/java.base/share/classes/java/util/Collections.java Outdated Show resolved Hide resolved

igraves added 2 commits February 17, 2021 11:43

If-block formatting

53e366a

Removing ImmutableCollections.* idempotency

a9588e9

liach reviewed Feb 20, 2021

View reviewed changes

Updating Collections.java copyright

ebdd0f8

cl4es approved these changes Feb 22, 2021

View reviewed changes

Test refactoring. Adding implNote to modified methods

8109e15

stuart-marks approved these changes Feb 26, 2021

View reviewed changes

jddarcy reviewed Mar 3, 2021

View reviewed changes

openjdk bot removed the csr Pull request needs approved CSR before integration label Mar 3, 2021

openjdk bot added the ready Pull request is ready to be integrated label Mar 3, 2021

Avoid wrapping subclasses where relevant. Updated tests.

dda174c

Revert "Avoid wrapping subclasses where relevant. Updated tests."

88127ed

This reverts commit dda174c.

stuart-marks approved these changes Mar 4, 2021

View reviewed changes

Commenting intent to not check for subclasses

47113ba

jddarcy approved these changes Mar 4, 2021

View reviewed changes

openjdk bot added the sponsor Pull request is ready to be sponsored label Mar 5, 2021

openjdk bot closed this Mar 5, 2021

openjdk bot added integrated Pull request has been integrated and removed sponsor Pull request is ready to be sponsored ready Pull request is ready to be integrated rfr Pull request is ready for review labels Mar 5, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

6323374: (coll) Optimize Collections.unmodifiable* and synchronized* #2596

6323374: (coll) Optimize Collections.unmodifiable* and synchronized* #2596

igraves commented Feb 16, 2021 •

edited by openjdk bot

Loading

bridgekeeper bot commented Feb 16, 2021

igraves commented Feb 16, 2021

openjdk bot commented Feb 16, 2021

openjdk bot commented Feb 16, 2021

mlbridge bot commented Feb 16, 2021 •

edited

Loading

RogerRiggs left a comment

liach left a comment

cl4es left a comment

jddarcy commented Feb 23, 2021

cl4es commented Feb 23, 2021 •

edited

Loading

igraves commented Feb 23, 2021

stuart-marks commented Feb 23, 2021

stuart-marks commented Feb 24, 2021

igraves commented Feb 26, 2021

stuart-marks left a comment

jddarcy Mar 3, 2021

igraves Mar 4, 2021

igraves Mar 4, 2021

openjdk bot commented Mar 3, 2021 •

edited

Loading

stuart-marks commented Mar 4, 2021 •

edited

Loading

igraves commented Mar 4, 2021

jddarcy commented Mar 4, 2021

igraves commented Mar 4, 2021

igraves commented Mar 5, 2021

openjdk bot commented Mar 5, 2021

stuart-marks commented Mar 5, 2021

openjdk bot commented Mar 5, 2021

6323374: (coll) Optimize Collections.unmodifiable* and synchronized* #2596

6323374: (coll) Optimize Collections.unmodifiable* and synchronized* #2596

Conversation

igraves commented Feb 16, 2021 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Download

bridgekeeper bot commented Feb 16, 2021

igraves commented Feb 16, 2021

openjdk bot commented Feb 16, 2021

openjdk bot commented Feb 16, 2021

mlbridge bot commented Feb 16, 2021 • edited Loading

Webrevs

RogerRiggs left a comment

Choose a reason for hiding this comment

liach left a comment

Choose a reason for hiding this comment

cl4es left a comment

Choose a reason for hiding this comment

jddarcy commented Feb 23, 2021

cl4es commented Feb 23, 2021 • edited Loading

igraves commented Feb 23, 2021

stuart-marks commented Feb 23, 2021

stuart-marks commented Feb 24, 2021

igraves commented Feb 26, 2021

stuart-marks left a comment

Choose a reason for hiding this comment

jddarcy Mar 3, 2021

Choose a reason for hiding this comment

igraves Mar 4, 2021

Choose a reason for hiding this comment

igraves Mar 4, 2021

Choose a reason for hiding this comment

openjdk bot commented Mar 3, 2021 • edited Loading

stuart-marks commented Mar 4, 2021 • edited Loading

igraves commented Mar 4, 2021

jddarcy commented Mar 4, 2021

igraves commented Mar 4, 2021

igraves commented Mar 5, 2021

openjdk bot commented Mar 5, 2021

stuart-marks commented Mar 5, 2021

openjdk bot commented Mar 5, 2021

igraves commented Feb 16, 2021 •

edited by openjdk bot

Loading

mlbridge bot commented Feb 16, 2021 •

edited

Loading

cl4es commented Feb 23, 2021 •

edited

Loading

openjdk bot commented Mar 3, 2021 •

edited

Loading

stuart-marks commented Mar 4, 2021 •

edited

Loading