Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8273152: Refactor CDS FileMapHeader loading code #5768

Closed
wants to merge 9 commits into from

Conversation

yminqi
Copy link
Contributor

@yminqi yminqi commented Sep 30, 2021

Please review,
Refactor fundamental CDS FileMapHeader code for reliable reading of basic info from shared archive.
With the change, it makes it possible to read an archive generated by different version of hotspot. Also it is possible to automatically generate a CDS archive If the archive supplied is not readable or fails to pass the check.

Tests: tier1-4
jtreg on sa.

Thanks
Yumin


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/5768/head:pull/5768
$ git checkout pull/5768

Update a local copy of the PR:
$ git checkout pull/5768
$ git pull https://git.openjdk.java.net/jdk pull/5768/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 5768

View PR using the GUI difftool:
$ git pr show -t 5768

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/5768.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Sep 30, 2021

👋 Welcome back minqi! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Sep 30, 2021

@yminqi The following labels will be automatically applied to this pull request:

  • hotspot-runtime
  • serviceability

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added serviceability hotspot-runtime labels Sep 30, 2021
@yminqi yminqi marked this pull request as ready for review Sep 30, 2021
@openjdk openjdk bot added the rfr label Sep 30, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Sep 30, 2021

Copy link
Member

@calvinccheung calvinccheung left a comment

Looks good. Just a few minor comments.

src/hotspot/share/cds/filemap.cpp Outdated Show resolved Hide resolved
src/hotspot/share/cds/filemap.cpp Outdated Show resolved Hide resolved
src/hotspot/share/cds/filemap.cpp Outdated Show resolved Hide resolved
src/hotspot/share/cds/filemap.cpp Outdated Show resolved Hide resolved
Copy link
Member

@iklam iklam left a comment

Looks good overall. Just a few nits.

src/hotspot/share/cds/filemap.cpp Outdated Show resolved Hide resolved
*target = NEW_C_HEAP_ARRAY(char, name_size, mtInternal);
lseek(_fd, _header->_base_archive_path_offset, SEEK_SET); // position to correct offset.
size_t n = os::read(_fd, *target, (unsigned int)name_size);
if (n != name_size) {
Copy link
Member

@iklam iklam Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's no need to do another read. The base name string is already inside the buffer that was read on line 1079. You can just do a strncpy into *target

Copy link
Contributor Author

@yminqi yminqi Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FileHeaderHelper only read sizeof(GenericCDSFileMapHeader) which is the stable information for reading other information and checking the archive. It does not include name in the buffer.

FileMapInfo::fail_continue("The shared archive file has a bad magic number.");
return false;
}

_header = (FileMapHeader*)os::malloc(gen_header->_header_size, mtInternal);
Copy link
Member

@iklam iklam Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need to allocate and read the header again. It's already in gen_header. This should be enough:

_header = (FileMapHeader*)gen_header;

Copy link
Contributor Author

@yminqi yminqi Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see above reply. We need read the full _header_size for _header. Also note that helper class will delete gen_header when out of scope.

Copy link
Member

@iklam iklam Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I think your current code is fine.

Note that the current code writes the header as a FileMapHeader, but it reads it as both a FileMapHeader and a CDSFileMapHeaderBase. So it has the basic assumption FileMapHeader::_generic_header is at offset 0 of FileMapHeader. Therefore, in FileHeaderHelper::initialize, we should add an assert:

static_assert(offsetof(FileMapHeader, _generic_header) == 0, "must be");

Copy link
Contributor Author

@yminqi yminqi Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This caused compiling error: since FileHeaderHelper is not a friend of FileMapHeader so no access to its fields:
error: 'GenericCDSFileMapHeader CDSFileMapHeaderBase::_generic_header' is inaccessible within this context

print_debug("can't read shared archive file map header from %s\n", classes_jsa);
close(fd);
return false;
}

// check file magic
if (header._magic != CDS_ARCHIVE_MAGIC) {
if (header._generic_header._magic != CDS_ARCHIVE_MAGIC) {
Copy link
Member

@iklam iklam Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use header.magic() instead?

Copy link
Contributor Author

@yminqi yminqi Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

header is a structure of CDSFileMapHeaderBase, unless we cast it to FileMapHeader.

src/hotspot/share/cds/filemap.cpp Outdated Show resolved Hide resolved
iklam
iklam approved these changes Oct 7, 2021
Copy link
Member

@iklam iklam left a comment

LGTM

@openjdk
Copy link

@openjdk openjdk bot commented Oct 7, 2021

@yminqi This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8273152: Refactor CDS FileMapHeader loading code

Reviewed-by: ccheung, iklam

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 112 new commits pushed to the master branch:

  • 920e707: 8274920: ProblemList 2 VectorAPI tests failing due to "assert(!vbox->is_Phi()) failed"
  • 7de2cf8: 8273910: Redundant condition and assignment in java.net.URI
  • 8ca0846: 8274407: (tz) Update Timezone Data to 2021c
  • 4ab274a: 8274858: Remove unused dictionary_classes_do functions
  • 03a8d34: 8273826: Correct Manifest file name and NPE checks
  • 722d639: 8274838: runtime/cds/appcds/TestSerialGCWithCDS.java fails on Windows
  • 340c715: 8273929: Remove GzipRandomAccess in heap dump test
  • 8319836: 8274546: Shenandoah: Remove unused ShenandoahUpdateRootsTask copy
  • d5ccfa2: 8274755: Replace 'while' cycles with iterator with enhanced-for in jdk.jdi
  • 29dcbb7: 8274716: JDWP Spec: the description for the Dispose command confuses suspend with resume.
  • ... and 102 more: https://git.openjdk.java.net/jdk/compare/d8a278f3948e9a5c881e64cce0a588e31246a4b5...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Oct 7, 2021
static void runTwo(String base, String top,
String jarName, String mainClassName, int exitValue,
String ... checkMessages) throws Exception {
Copy link
Member

@calvinccheung calvinccheung Oct 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this method is necessary since the existing run2() method seems to do the same thing.
The only advantage I see is the call site is less verbose than before.

Copy link
Member

@calvinccheung calvinccheung left a comment

I just have one minor comment. It's up to you if you want to change it.

@yminqi
Copy link
Contributor Author

@yminqi yminqi commented Oct 7, 2021

I just have one minor comment. It's up to you if you want to change it.

Thanks for review --- I will leave as it is now.

@yminqi
Copy link
Contributor Author

@yminqi yminqi commented Oct 7, 2021

@iklam @calvinccheung Thanks for review!
/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Oct 7, 2021

Going to push as commit 8de7763.
Since your change was applied there have been 112 commits pushed to the master branch:

  • 920e707: 8274920: ProblemList 2 VectorAPI tests failing due to "assert(!vbox->is_Phi()) failed"
  • 7de2cf8: 8273910: Redundant condition and assignment in java.net.URI
  • 8ca0846: 8274407: (tz) Update Timezone Data to 2021c
  • 4ab274a: 8274858: Remove unused dictionary_classes_do functions
  • 03a8d34: 8273826: Correct Manifest file name and NPE checks
  • 722d639: 8274838: runtime/cds/appcds/TestSerialGCWithCDS.java fails on Windows
  • 340c715: 8273929: Remove GzipRandomAccess in heap dump test
  • 8319836: 8274546: Shenandoah: Remove unused ShenandoahUpdateRootsTask copy
  • d5ccfa2: 8274755: Replace 'while' cycles with iterator with enhanced-for in jdk.jdi
  • 29dcbb7: 8274716: JDWP Spec: the description for the Dispose command confuses suspend with resume.
  • ... and 102 more: https://git.openjdk.java.net/jdk/compare/d8a278f3948e9a5c881e64cce0a588e31246a4b5...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Oct 7, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Oct 7, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Oct 7, 2021

@yminqi Pushed as commit 8de7763.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@yminqi yminqi deleted the jdk-8273152 branch Oct 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-runtime integrated serviceability
3 participants