8254161: Prevent instantiation of EnumSet subclasses through deserialization

[ChrisHegarty]

TL;DR add EnumSet::readObjectNoData()

EnumSet is an exemplar of the Serialization Proxy Pattern. As such, it
should strictly implement that pattern and demonstrate how best to
defend against inappropriate instantiation through deserialization.

EnumSet is an extensible class. There are two subclasses in the JDK,
RegularEnumSet and JumboEnumSet. Since the serialization of an EnumSet
object writes a replacement object to the serial stream, a serial proxy
object, then stream objects of type RegularEnumSet or JumboEnumSet are
not expected in the serial stream. However, if they are present in the
serial stream, then, during deserialization, the EnumSet::readObject
method will be invoked. EnumSet::readObject unconditionally throws an
exception, thus preventing further deserialization of the stream object.
In this way, stream objects that are subclasses of EnumSet are prevented
from being instantiated through deserialization. But this is not
sufficient to prevent such in all scenarios.

A stream object whose local class equivalent of the specified stream
class descriptor is a subclasses of EnumSet, but whose specified stream
class descriptor does not list EnumSet as a superClass, may be
instantiated through deserialization. Since the stream class descriptor
does not list EnumSet as a superclass, then the defensive
EnumSet::readObject is never invoked. To prevent such objects from
being deserialized, an EnumSet::readObjectNoData() should be added -
whose implementation unconditionally throws an exception, similar to
that of the existing EnumSet::readObject.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Testing

	Linux x64	Windows x64	macOS x64
Build	✔️ (5/5 passed)	✔️ (2/2 passed)	✔️ (2/2 passed)
Test (tier1)	✔️ (9/9 passed)	✔️ (9/9 passed)	✔️ (9/9 passed)

Issue

• JDK-8254161: Prevent instantiation of EnumSet subclasses through deserialization

Reviewers

• Daniel Fuchs (@dfuch - Reviewer)
• Alan Bateman (@AlanBateman - Reviewer)
• Roger Riggs (@RogerRiggs - Reviewer)
• Stuart Marks (@stuart-marks - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jdk pull/611/head:pull/611
$ git checkout pull/611

[bridgekeeper]

👋 Welcome back chegar! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@ChrisHegarty The following label will be automatically applied to this pull request:

• core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[ChrisHegarty]

/csr

[openjdk]

@ChrisHegarty has indicated that a compatibility and specification (CSR) request is needed for this pull request.
@ChrisHegarty please create a CSR request and add link to it in JDK-8254161. This pull request cannot be integrated until the CSR request is approved.

[mlbridge]

Webrevs

• 00: Full (1f5c5f0)

[openjdk]

@ChrisHegarty This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8254161: Prevent instantiation of EnumSet subclasses through deserialization

Reviewed-by: dfuchs, alanb, rriggs, smarks

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 60 new commits pushed to the master branch:

• 96bb6e7: 8251325: Miss 'L' for long value in if statement
• 546620b: 8254192: ExtraSharedClassListFile contains extra white space at end of line
• f3ce45f: 8254799: runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java fails with release VMs
• cda22e3: 8254811: JDK-8254158 broke ppc64, s390 builds
• 8ef88e5: 8249878: jfr_emergency_dump has secondary crashes
• 3c2f5e0: 8254674: G1: Improve root location reference to dead obj verification message
• abe5137: 8254719: ZGC: Clean up includes
• 60159cf: 8253952: Refine ZipOutputStream.putNextEntry() to recalculate ZipEntry's compressed size
• 9359ff0: 8252221: Use multiple workers for Parallel GC pre-touching
• f44fc6d: 8254734: "dead loop detected" assert failure with patch from 8223051
• ... and 50 more: https://git.openjdk.java.net/jdk/compare/c7f00640627eab38b77d23d07876cf0247fa18f3...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[ChrisHegarty]

/integrate

[openjdk]

@ChrisHegarty Since your change was applied there have been 64 commits pushed to the master branch:

• 3d23bd8: 8254814: [Vector API] Fix an AVX512 crash after JDK-8223347
• 7c0d417: 8251535: Partial peeling at unsigned test adds incorrect loop exit check
• 5145bed: 8254125: Assertion in cppVtables.cpp during builds on 32bit Windows
• bdda205: 8254369: Node::disconnect_inputs may skip precedences
• 96bb6e7: 8251325: Miss 'L' for long value in if statement
• 546620b: 8254192: ExtraSharedClassListFile contains extra white space at end of line
• f3ce45f: 8254799: runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java fails with release VMs
• cda22e3: 8254811: JDK-8254158 broke ppc64, s390 builds
• 8ef88e5: 8249878: jfr_emergency_dump has secondary crashes
• 3c2f5e0: 8254674: G1: Improve root location reference to dead obj verification message
• ... and 54 more: https://git.openjdk.java.net/jdk/compare/c7f00640627eab38b77d23d07876cf0247fa18f3...master

Your commit was automatically rebased without conflicts.

Pushed as commit 34583eb.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.