Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8253079: DeterministicDump.java fails due to garbage in structure padding #267

Conversation

iklam
Copy link
Member

@iklam iklam commented Sep 20, 2020

(EDITED) In product builds, when PackageEntry and ModuleEntry objects are allocated, the memory is not zeroed. As a result, the structure padding slots (such as the 32-bits after BasicHashtableEntry::_hash) may contain garbage values that are different on every run of java -Xshare:dump. As a result, java -Xshare:dump cannot reproduce deterministic result.

The fix is to clear the memory for the newly allocated HashtableEntry objects when DumpSharedSpaces == true.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8253079: DeterministicDump.java fails due to garbage in structure padding

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/267/head:pull/267
$ git checkout pull/267

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Sep 20, 2020

👋 Welcome back iklam! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Sep 20, 2020

@iklam The following label will be automatically applied to this pull request: hotspot-runtime.

When this pull request is ready to be reviewed, an RFR email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label (add|remove) "label" command.

@openjdk openjdk bot added the hotspot-runtime label Sep 20, 2020
@iklam iklam changed the title 8253079: runtime/cds/DeterministicDump.java fails due to garbage in structure padding 8253079: DeterministicDump.java fails due to garbage in structure padding Sep 20, 2020
@iklam iklam marked this pull request as ready for review Sep 20, 2020
@openjdk openjdk bot added the rfr label Sep 20, 2020
@mlbridge
Copy link

@mlbridge mlbridge bot commented Sep 20, 2020

Webrevs

@yminqi
Copy link
Contributor

@yminqi yminqi commented Sep 20, 2020

The implementation is quite complex, every derivatives from BasicHashTableEntry need to have a copy_from function to avoid such problem. Maybe a brutal one to avoid such padding issue for all cases is fill the object allocated in AllocateHeap with \0?

@iklam
Copy link
Member Author

@iklam iklam commented Sep 21, 2020

The implementation is quite complex, every derivatives from BasicHashTableEntry need to have a copy_from function to avoid such problem. Maybe a brutal one to avoid such padding issue for all cases is fill the object allocated in AllocateHeap with \0?

You're right. My original analysis was wrong: set_hash() didn't write garbage into the padding. Instead, the garbage was there because AllocaHeap didn't initialize the new buffer in product builds.

I reverted the original fix. Instead, I added code to call memset() when allocating a new hashtable entry (but only when DumpSharedSpaces is true).

@@ -454,6 +454,7 @@ void ModuleEntry::init_as_archived_entry() {
if (_location != NULL) {
_location = ArchiveBuilder::get_relocated_symbol(_location);
}
JFR_ONLY(memset(trace_id_addr(), 0, sizeof(traceid)));
Copy link
Contributor

@shipilev shipilev Sep 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

memset looks dodgy here. Maybe JFR_ONLY(set_trace_id(0))?

Copy link
Member Author

@iklam iklam Sep 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed these from my previous commit, because the trace_id seemed to be predictable and wouldn't cause the archive content to change. Anyway, I've added the code you suggested in commit 5706a82. That way, if trace_ids become unpredictable due to future JFR implementation changes, the CDS code won't be affected.

@@ -238,6 +238,7 @@ void PackageEntry::init_as_archived_entry() {
_module = ModuleEntry::get_archived_entry(_module);
_qualified_exports = (GrowableArray<ModuleEntry*>*)archived_qualified_exports;
_defined_by_cds_in_class_path = 0;
JFR_ONLY(memset(trace_id_addr(), 0, sizeof(traceid)));
Copy link
Contributor

@shipilev shipilev Sep 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, JFR_ONLY(set_trace_id(0))?

@DamonFool
Copy link
Member

@DamonFool DamonFool commented Sep 21, 2020

It passed with this fix.
Looks good to me.
Thanks.

@DamonFool
Copy link
Member

@DamonFool DamonFool commented Sep 21, 2020

/reviewer jiefu

@openjdk
Copy link

@openjdk openjdk bot commented Sep 21, 2020

@DamonFool Only the author (@iklam) is allowed to issue the reviewer command.

yminqi
yminqi approved these changes Sep 21, 2020
Copy link
Contributor

@yminqi yminqi left a comment

looks good!

@openjdk
Copy link

@openjdk openjdk bot commented Sep 21, 2020

@iklam This change now passes all automated pre-integration checks. In addition to the automated checks, the change must also fulfill all project specific requirements

After integration, the commit message will be:

8253079: DeterministicDump.java fails due to garbage in structure padding

Reviewed-by: minqi, jiefu, shade
  • If you would like to add a summary, use the /summary command.
  • To credit additional contributors, use the /contributor command.
  • To add additional solved issues, use the /issue command.

Since the source branch of this PR was last updated there have been 19 commits pushed to the master branch:

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid automatic rebasing, please merge master into your branch, and then specify the current head hash when integrating, like this: /integrate a4affd5b04ab9ab409871561bdfa7536f2c32e79.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Sep 21, 2020
@iklam
Copy link
Member Author

@iklam iklam commented Sep 21, 2020

/reviewer jiefu

@openjdk
Copy link

@openjdk openjdk bot commented Sep 21, 2020

@iklam Syntax: /reviewer (credit|remove) [@user | openjdk-user]+. For example:

  • /reviewer credit @openjdk-bot
  • /reviewer credit duke
  • /reviewer credit @user1 @user2

@iklam
Copy link
Member Author

@iklam iklam commented Sep 21, 2020

/reviewer credit jiefu

@openjdk
Copy link

@openjdk openjdk bot commented Sep 21, 2020

@iklam
Reviewer jiefu successfully credited.

@iklam
Copy link
Member Author

@iklam iklam commented Sep 22, 2020

/integrate

@openjdk openjdk bot closed this Sep 22, 2020
@openjdk openjdk bot added the integrated label Sep 22, 2020
@openjdk openjdk bot removed ready rfr labels Sep 22, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Sep 22, 2020

@iklam Since your change was applied there have been 19 commits pushed to the master branch:

Your commit was automatically rebased without conflicts.

Pushed as commit 284bbf0.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-runtime integrated
4 participants