Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8286978: SIGBUS in libz during CDS initialization #8799

Closed

Conversation

calvinccheung
Copy link
Member

@calvinccheung calvinccheung commented May 20, 2022

Please review this fix to ensure all requested bytes are transferred by calling FileChannel.transferFrom in a loop and checking its return value.

Tested by running the SharedArchiveConsistency.java test on linux-aarch64 30 times on each of the following tier3 options:

  • -XX:+CreateCoredumpOnCrash -XX:+UseSerialGC
  • -XX:+CreateCoredumpOnCrash -XX:+UseParallelGC -XX:+UseNUMA

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed (1 review required, with at least 1 reviewer)

Issue

  • JDK-8286978: SIGBUS in libz during CDS initialization

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/8799/head:pull/8799
$ git checkout pull/8799

Update a local copy of the PR:
$ git checkout pull/8799
$ git pull https://git.openjdk.java.net/jdk pull/8799/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 8799

View PR using the GUI difftool:
$ git pr show -t 8799

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/8799.diff

@bridgekeeper
Copy link

bridgekeeper bot commented May 20, 2022

👋 Welcome back ccheung! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@calvinccheung
Copy link
Member Author

calvinccheung commented May 20, 2022

/label add hotspot-runtime

@openjdk openjdk bot added the hotspot-runtime label May 20, 2022
@openjdk
Copy link

openjdk bot commented May 20, 2022

@calvinccheung
The hotspot-runtime label was successfully added.

@calvinccheung calvinccheung marked this pull request as ready for review May 20, 2022
@openjdk openjdk bot added the rfr label May 20, 2022
@mlbridge
Copy link

mlbridge bot commented May 20, 2022

Webrevs

inputChannel.position(offset + nBytes);
outputChannel.transferFrom(inputChannel, offset, orgSize - nBytes);
long length = orgSize - nBytes;
Copy link
Member

@iklam iklam May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 458 doesn't seem to be needed.

Copy link
Member Author

@calvinccheung calvinccheung May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// dstFile will keep original size so will remove corresponding bytes.length bytes at end of file
public static File insertBytesRandomlyAfterHeader(File orgFile, String newFileName, byte[] bytes) throws Exception {
long offset = fileHeaderSize(orgFile) + getRandomBetween(0L, 4096L);
File dstFile = new File(newFileName);
try (FileChannel inputChannel = new FileInputStream(orgFile).getChannel();
FileChannel outputChannel = new FileOutputStream(dstFile).getChannel()) {
long orgSize = inputChannel.size();
outputChannel.transferFrom(inputChannel, 0, offset);
transferFrom(inputChannel, outputChannel, 0, offset);
outputChannel.position(offset);
outputChannel.write(ByteBuffer.wrap(bytes));
Copy link
Member

@iklam iklam May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes for transferFrom looks good, but we still have a problem with outputChannel.write(), which can also return fewer bytes than requested (or even zero).

For simplicity, I think it's best to ditch FileChannel and use FileOutputStream.write() instead.

Copy link
Member

@iklam iklam May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option is to avoid using outputChannel.write(). Instead, duplicate some bytes from the end of the header.

    public static File insertBytesRandomlyAfterHeader(File orgFile, String newFileName) throws Exception {
        long headerSize = fileHeaderSize(orgFile);
        long dupSize = getRandomBetween(0L, headerSize);
        File dstFile = new File(newFileName);
        try (FileChannel inputChannel = new FileInputStream(orgFile).getChannel();
             FileChannel outputChannel = new FileOutputStream(dstFile).getChannel()) {
            long orgSize = inputChannel.size();
            // Copy the header
            transferFrom(inputChannel, outputChannel, 0, headerSize);
            // Copy dupSize bytes from the end of the header. Then, copy the rest
            // of the input such that the new file will have the same size as
            // the old file.
            inputChannel.position(headerSize - dupSize);
            transferFrom(inputChannel, outputChannel, headerSize, orgSize - headerSize);
        }
        return dstFile;
    }

Copy link
Member Author

@calvinccheung calvinccheung May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestions. I've pushed another commit with the above changes. I also changed the callsite in SharedArchiveConsistency.java since there's no need to pass in a byte array.

iklam
iklam approved these changes May 20, 2022
Copy link
Member

@iklam iklam left a comment

LGTM

@openjdk
Copy link

openjdk bot commented May 20, 2022

@calvinccheung This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8286978: SIGBUS in libz during CDS initialization

Reviewed-by: iklam

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 16 new commits pushed to the master branch:

  • 4587337: 8286858: Remove dead code in sun.reflect.misc.MethodUtil
  • 6d56caf: 8285962: NimbusDefaults has a typo in a L&F property
  • d5d19f5: 8287013: StringConcatFactory: remove short and byte mixers/prependers
  • 828dc89: 8286893: G1: Recent card set coarsening statistics wrong
  • 6569666: 8286943: G1: With virtualized remembered sets, maximum number of cards configured is wrong
  • 40e99a1: 8285308: Win: Japanese logical fonts are drawn with wrong size
  • 890771e: 8285517: System.getenv() returns unexpected value if environment variable has non ASCII character
  • de74e0e: 8280035: Use Class.isInstance instead of Class.isAssignableFrom where applicable
  • 9f562ef: 8286872: Refactor add/modify notification icon (TrayIcon)
  • b089229: 8271078: jdk/incubator/vector/Float128VectorTests.java failed a subtest
  • ... and 6 more: https://git.openjdk.java.net/jdk/compare/83cec4b6cc25b9b0db751bbe873444259cdcb21c...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label May 20, 2022
@calvinccheung
Copy link
Member Author

calvinccheung commented May 20, 2022

Thanks @iklam for the review.
/integrate

@openjdk
Copy link

openjdk bot commented May 20, 2022

Going to push as commit 087bccf.
Since your change was applied there have been 16 commits pushed to the master branch:

  • 4587337: 8286858: Remove dead code in sun.reflect.misc.MethodUtil
  • 6d56caf: 8285962: NimbusDefaults has a typo in a L&F property
  • d5d19f5: 8287013: StringConcatFactory: remove short and byte mixers/prependers
  • 828dc89: 8286893: G1: Recent card set coarsening statistics wrong
  • 6569666: 8286943: G1: With virtualized remembered sets, maximum number of cards configured is wrong
  • 40e99a1: 8285308: Win: Japanese logical fonts are drawn with wrong size
  • 890771e: 8285517: System.getenv() returns unexpected value if environment variable has non ASCII character
  • de74e0e: 8280035: Use Class.isInstance instead of Class.isAssignableFrom where applicable
  • 9f562ef: 8286872: Refactor add/modify notification icon (TrayIcon)
  • b089229: 8271078: jdk/incubator/vector/Float128VectorTests.java failed a subtest
  • ... and 6 more: https://git.openjdk.java.net/jdk/compare/83cec4b6cc25b9b0db751bbe873444259cdcb21c...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated label May 20, 2022
@openjdk openjdk bot closed this May 20, 2022
@openjdk openjdk bot removed ready rfr labels May 20, 2022
@openjdk
Copy link

openjdk bot commented May 20, 2022

@calvinccheung Pushed as commit 087bccf.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@calvinccheung calvinccheung deleted the CDSArchiveUtils-transferFrom branch May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-runtime integrated
2 participants