Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8275731: CDS archived enums objects are recreated at runtime #6653

Closed
wants to merge 10 commits into from

Conversation

iklam
Copy link
Member

@iklam iklam commented Dec 1, 2021

Background:

In the Java Language, Enums can be tested for equality, so the constants in an Enum type must be unique. Javac compiles an enum declaration like this:

public enum Day {  SUNDAY, MONDAY ... } 

to

public class Day extends java.lang.Enum {
    public static final SUNDAY = new Day("SUNDAY");
    public static final MONDAY = new Day("MONDAY"); ...
}

With CDS archived heap objects, Day::<clinit> is executed twice: once during java -Xshare:dump, and once during normal JVM execution. If the archived heap objects references one of the Enum constants created at dump time, we will violate the uniqueness requirements of the Enum constants at runtime. See the test case in the description of JDK-8275731

Fix:

During -Xshare:dump, if we discovered that an Enum constant of type X is archived, we archive all constants of type X. At Runtime, type X will skip the normal execution of X::<clinit>. Instead, we run HeapShared::initialize_enum_klass() to retrieve all the constants of X that were saved at dump time.

This is safe as we know that X::<clinit> has no observable side effect -- it only creates the constants of type X, as well as the synthetic value X::$VALUES, which cannot be observed until X is fully initialized.

Verification:

To avoid future problems, I added a new tool, CDSHeapVerifier, to look for similar problems where the archived heap objects reference a static field that may be recreated at runtime. There are some manual steps involved, but I analyzed the potential problems found by the tool are they are all safe (after the current bug is fixed). See cdsHeapVerifier.cpp for gory details. An example trace of this tool can be found at https://bugs.openjdk.java.net/secure/attachment/97242/enum_warning.txt

Testing:

Passed Oracle CI tiers 1-4. WIll run tier 5 as well.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8275731: CDS archived enums objects are recreated at runtime

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/6653/head:pull/6653
$ git checkout pull/6653

Update a local copy of the PR:
$ git checkout pull/6653
$ git pull https://git.openjdk.java.net/jdk pull/6653/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 6653

View PR using the GUI difftool:
$ git pr show -t 6653

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/6653.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Dec 1, 2021

👋 Welcome back iklam! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Dec 1, 2021

@iklam The following label will be automatically applied to this pull request:

  • hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot label Dec 1, 2021
@iklam
Copy link
Member Author

@iklam iklam commented Dec 1, 2021

/label add core-libs

@openjdk openjdk bot added the core-libs label Dec 1, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Dec 1, 2021

@iklam
The core-libs label was successfully added.

@iklam iklam marked this pull request as ready for review Dec 1, 2021
@openjdk openjdk bot added the rfr label Dec 1, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Dec 1, 2021

@openjdk openjdk bot removed the rfr label Dec 8, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Jan 5, 2022

Mailing list message from Ioi Lam on hotspot-dev:

Still looking for reviewers ....

Thanks
- Ioi

On 12/1/21 1:02 PM, Ioi Lam wrote:

Copy link
Contributor

@coleenp coleenp left a comment

I don't really know this code well enough to do a good code review. I had some comments though.

@@ -377,6 +479,7 @@ void HeapShared::archive_objects(GrowableArray<MemRegion>* closed_regions,
log_info(cds)("Dumping objects to open archive heap region ...");
copy_open_objects(open_regions);

CDSHeapVerifier::verify();
Copy link
Contributor

@coleenp coleenp Jan 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should all this be DEBUG_ONLY ?

Copy link
Member Author

@iklam iklam Jan 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed CDSHeapVerifier::verify() to a NOT_DEBUG_RETURN function.

KlassSubGraphInfo* _subgraph_info;
oop _referrer;
oop _obj;
CachedOopInfo() :_subgraph_info(), _referrer(), _obj() {}
Copy link
Contributor

@coleenp coleenp Jan 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these be initialized to nullptr? does this do this?

Copy link
Member Author

@iklam iklam Jan 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These three fields are initialized with the default initializer (empty parenthesis) so they will be initialized to the null pointer.

}

ResourceMark rm;
for (JavaFieldStream fs(ik); !fs.done(); fs.next()) {
Copy link
Contributor

@coleenp coleenp Jan 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this call instead
void InstanceKlass::do_local_static_fields(void f(fieldDescriptor*, Handle, TRAPS), Handle mirror, TRAPS) {
and have this next few lines in the function?

Copy link
Member Author

@iklam iklam Jan 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the code inside a new class CDSHeapVerifier::CheckStaticFields so I can call InstanceKlass::do_local_static_fields

if (k->is_instance_klass()) {
InstanceKlass* ik = InstanceKlass::cast(k);
for (JavaFieldStream fs(ik); !fs.done(); fs.next()) {
if (!fs.access_flags().is_static()) {
Copy link
Contributor

@coleenp coleenp Jan 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here. It only saves a couple of lines but then you can have the function outside this large function.

Copy link
Member Author

@iklam iklam Jan 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You actually found a bug here. I am iterating over non-static fields and should walk the inherited fields as well. I changed the code to call InstanceKlass::do_nonstatic_fields()

ResourceObj::C_HEAP,
mtClassShared,
HeapShared::oop_hash> _table;

Copy link
Contributor

@coleenp coleenp Jan 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this only used inside cdsHeapVerifier? if so it should be in the .cpp file. There's also an ArchiveableStaticFieldInfo. Not sure how they are related.

Copy link
Member Author

@iklam iklam Jan 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This _table is part of the CDSHeapVerifier instance, which is stack allocated. So I need to declare it as part of the CDSHeapVerifier class declaration in the hpp file.


oop mirror = k->java_mirror();
int i = 0;
for (JavaFieldStream fs(k); !fs.done(); fs.next()) {
Copy link
Contributor

@coleenp coleenp Jan 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like it should also use InstanceKlass::do_local_static_fields.

Copy link
Member Author

@iklam iklam Jan 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converting this to InstanceKlass::do_nonstatic_fields() is difficult because the loop body references 7 different variables declared outside of the loop.

One thing I tried is to add a new version of do_nonstatic_fields2() that supports C++ lambdas. You can see my experiment from here:

https://github.com/openjdk/jdk/compare/master...iklam:lambda-for-instanceklass-do_local_static_fields2?expand=1

I changed all my new code to use the do_nonstatic_fields2() function with lambda.

Copy link
Contributor

@coleenp coleenp Feb 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, if it requires lambdas and additional change, never mind then.

@openjdk
Copy link

@openjdk openjdk bot commented Jan 19, 2022

@iklam this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout 8275731-heapshared-enum
git fetch https://git.openjdk.java.net/jdk master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

@openjdk openjdk bot added the merge-conflict label Jan 19, 2022
@iklam
Copy link
Member Author

@iklam iklam commented Jan 19, 2022

I don't really know this code well enough to do a good code review. I had some comments though.

Hi Coleen, thanks for taking a look.

This PR has two major parts:

  1. Check for inappropriate reference to static fields. This is mainly done in cdsHeapVerifier.cpp. These checks don't affect the contents of the CDS archive. They just print out warnings if problems are found.
  2. Special initialization of enum classes. Essentially if any instance of an enum class X is archived, then X::<clinit> will not be executed, and we'll take this path instead (in instanceKlass.cpp):
  // This is needed to ensure the consistency of the archived heap objects.
  if (has_archived_enum_objs()) {
    assert(is_shared(), "must be");
    bool initialized = HeapShared::initialize_enum_klass(this, CHECK);
    if (initialized) {
      return;
    }
  }

Could you check if (2) is correct?

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Feb 16, 2022

@iklam This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@iklam
Copy link
Member Author

@iklam iklam commented Feb 16, 2022

@iklam This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

keepalive

//
// class Bar {
// // this field is initialized in both CDS dump time and runtime.
// static final Bar bar = new Bar;
Copy link
Member

@calvinccheung calvinccheung Feb 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new Bar should be new Bar()?

Copy link
Member

@calvinccheung calvinccheung left a comment

Looks good. Minor comment below.
Also, several files with copyright year 2021 need updating.

@openjdk openjdk bot removed the merge-conflict label Feb 23, 2022
@openjdk
Copy link

@openjdk openjdk bot commented Feb 23, 2022

@iklam This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8275731: CDS archived enums objects are recreated at runtime

Reviewed-by: coleenp, ccheung

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Feb 23, 2022
@openjdk openjdk bot added the rfr label Feb 23, 2022
Copy link
Contributor

@coleenp coleenp left a comment

Sorry for the long delay. It's a big change, but a lot in debug so that's ok. Looks good.


oop mirror = k->java_mirror();
int i = 0;
for (JavaFieldStream fs(k); !fs.done(); fs.next()) {
Copy link
Contributor

@coleenp coleenp Feb 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, if it requires lambdas and additional change, never mind then.

@iklam
Copy link
Member Author

@iklam iklam commented Feb 28, 2022

Thanks @calvinccheung and @coleenp for the review. Passed tiers 1-5.
/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Feb 28, 2022

Going to push as commit d983d10.
Since your change was applied there have been 8 commits pushed to the master branch:

  • c7cd148: 8282240: Add _name field to Method for NOT_PRODUCT only
  • 59b3ecc: 8277976: Break up SEQUENCE in X509Certificate::getSubjectAlternativeNames and X509Certificate::getIssuerAlternativeNames in otherName
  • 4e7fb41: 8282172: CompileBroker::log_metaspace_failure is called from non-Java/compiler threads
  • 0ae3d1d: 8282131: java.time.ZoneId should be a sealed abstract class
  • c58f5c6: 8282360: Merge POSIX implementations of ThreadCritical
  • 06cadb3: 8230382: Clean up ConvI2L, CastII and CastLL::Ideal methods
  • efd3967: 8267265: Use new IR Test Framework to create tests for C2 Ideal transformations
  • 86723d4: 8281507: Two javac tests have bad jtreg @clean tags

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated label Feb 28, 2022
@openjdk openjdk bot closed this Feb 28, 2022
@openjdk openjdk bot removed ready rfr labels Feb 28, 2022
@openjdk
Copy link

@openjdk openjdk bot commented Feb 28, 2022

@iklam Pushed as commit d983d10.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-libs hotspot integrated
4 participants