Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8272297: FileInputStream should override transferTo() for better performance #5097

Closed
wants to merge 4 commits into from

Conversation

bplb
Copy link
Member

@bplb bplb commented Aug 12, 2021

Please consider this request to add an override java.io.FileInputStream.transferTo(OutputStream) with improved performance if the parameter is a FileOutputStream.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8272297: FileInputStream should override transferTo() for better performance

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/5097/head:pull/5097
$ git checkout pull/5097

Update a local copy of the PR:
$ git checkout pull/5097
$ git pull https://git.openjdk.java.net/jdk pull/5097/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 5097

View PR using the GUI difftool:
$ git pr show -t 5097

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/5097.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Aug 12, 2021

👋 Welcome back bpb! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@bplb
Copy link
Member Author

@bplb bplb commented Aug 12, 2021

Invoking transferTo() on a FileInputStream will use the superclass method which copies using looping over buffers of a fixed size. If the parameter is a FileOutputStream, the copy is greatly accelerated by using the java.nio.channels.FileChannel.transferTo() method with the FileChannels of the source and destination streams. Performance measurements were made for streams of length 100000, 1000000, 100000000, and 1000000000 bytes. Throughput improvement was observed for all cases on the three usual platforms, with the largest of nearly 5X being seen on Linux because in this case the sendfile() system call is used underneath. In no case did throughput decrease.

@openjdk openjdk bot added the rfr label Aug 12, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Aug 12, 2021

@bplb The following label will be automatically applied to this pull request:

  • core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the core-libs label Aug 12, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Aug 12, 2021

Webrevs

@bplb
Copy link
Member Author

@bplb bplb commented Aug 12, 2021

Without the source change the test fails, but passes with the change in place.

@rkennke
Copy link
Contributor

@rkennke rkennke commented Aug 13, 2021

Is this what you have been asking @mkarg in #4263 to do? Optimize transferTo() only for FileInputStream? Would it interfere with #4263?

@bplb
Copy link
Member Author

@bplb bplb commented Aug 13, 2021

There is no overlap with #4263.

@AlanBateman
Copy link
Contributor

@AlanBateman AlanBateman commented Aug 13, 2021

Is this what you have been asking @mkarg in #4263 to do? Optimize transferTo() only for FileInputStream? Would it interfere with #4263?

#4263 is the input stream returned by Channels.newInputStream where the source may be a FileChannel and the output stream specified to InputStream::transferTo may be connected to a FileChannel.

}
}
return transferred + super.transferTo(out);
}
Copy link
Contributor

@AlanBateman AlanBateman Aug 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good, I think this version is right.

@openjdk
Copy link

@openjdk openjdk bot commented Aug 15, 2021

@bplb This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8272297: FileInputStream should override transferTo() for better performance

Reviewed-by: alanb

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 84 new commits pushed to the master branch:

  • 17b9350: 8266079: Lanai: AlphaComposite shows differences on Metal compared to OpenGL
  • 87d2761: 8271883: Math CopySign optimization for x86
  • 6b8b160: 8272396: mismatching debug output streams
  • 0af645a: 8205138: Remove Applet references from Font2DTest
  • bd7f9b4: 8272459: ProblemList compiler/codecache/TestStressCodeBuffers.java on aarch64
  • 717792c: 8263940: NPE when creating default file system when default file system provider is packaged as JAR file on class path
  • d06d0b9: 8272112: Arena code simplifications
  • 0c4be76: 8058704: Nimbus does not honor JTextPane background color
  • 020aec5: 8271366: [REDO] JDK-8266054 VectorAPI rotate operation optimization
  • 4d4ba5c: 8272116: Update PerfDisableSharedMem with FLAG_SET_ERGO in PerfMemory::create_memory_region
  • ... and 74 more: https://git.openjdk.java.net/jdk/compare/3ab95d19174100f440e2274d5b2c3119fb55e9e1...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Aug 15, 2021
@rkennke
Copy link
Contributor

@rkennke rkennke commented Aug 15, 2021

Is this what you have been asking @mkarg in #4263 to do? Optimize transferTo() only for FileInputStream? Would it interfere with #4263?

#4263 is the input stream returned by Channels.newInputStream where the source may be a FileChannel and the output stream specified to InputStream::transferTo may be connected to a FileChannel.

Having followed both #4263 and this PR from the sidelines, may I ask why for #4263 much more rigorous testing is asked but not here? E.g. test for NPE, random-sized files/buffers, random position, return value, comprehensive JMH tests to show performance, etc? I'm all for high quality standards and good test coverage, but why are we seemingly measuring with double standards?

@AlanBateman
Copy link
Contributor

@AlanBateman AlanBateman commented Aug 16, 2021

Having followed both #4263 and this PR from the sidelines, may I ask why for #4263 much more rigorous testing is asked but not here? E.g. test for NPE, random-sized files/buffers, random position, return value, comprehensive JMH tests to show performance, etc? I'm all for high quality standards and good test coverage, but why are we seemingly measuring with double standards?

I don't think your comment is fair. The FIS PR is modest, covering one scenario, it's easy to review. The serious bugs that we pointed out have been addressed and test coverage has been added. The Channels PR is much more ambitious and is trying to cover a matrix of scenarios involving file channel and selectable channels, lots of corner cases and potential issues, and will require a lot of review cycles. None of the scenarios had any test coverage initially. This has been partially addressed but the proposal is still unwieldy, hence the discussion on reducing the scope to something manageable.

@bplb
Copy link
Member Author

@bplb bplb commented Aug 16, 2021

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Aug 16, 2021

Going to push as commit 8268825.
Since your change was applied there have been 94 commits pushed to the master branch:

  • 3677734: 8271471: [IR Framework] Rare occurrence of "" in PrintIdeal/PrintOptoAssembly can let tests fail
  • 0a03481: 8272231: G1: Refactor G1CardSet::get_card_set to return G1CardSetHashTableValue*
  • 83d0e12: 8267833: Improve G1CardSetInlinePtr::add()
  • 69cc588: 8272235: G1: update outdated code root fixup
  • 5db36ce: 8272158: SoftReference related bugs under memory pressure
  • 7a5b37b: 8272439: G1: add documentation to G1CardSetInlinePtr
  • 0209d9f: 8272461: G1: remove empty declaration of cleanup_after_scan_heap_roots
  • 36e2dda: 8272348: Update CDS tests in anticipation of JDK-8270489
  • 3f38a50: 8271203: C2: assert(iff->Opcode() == Op_If || iff->Opcode() == Op_CountedLoopEnd || iff->Opcode() == Op_RangeCheck) failed: Check this code when new subtype is added
  • 6a5241c: 8272491: Problem list javax/swing/JFrame/NSTexturedJFrame/NSTexturedJFrame.java on macos
  • ... and 84 more: https://git.openjdk.java.net/jdk/compare/3ab95d19174100f440e2274d5b2c3119fb55e9e1...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Aug 16, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Aug 16, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Aug 16, 2021

@bplb Pushed as commit 8268825.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-libs integrated
3 participants