Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8268435: (ch) ChannelInputStream could override readAllBytes #5645

Closed

Conversation

bplb
Copy link
Member

@bplb bplb commented Sep 23, 2021

This change would override readAllBytes() and readNBytes(int) in ChannelInputStream thereby improving performance for all but smaller streams.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8268435: (ch) ChannelInputStream could override readAllBytes

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/5645/head:pull/5645
$ git checkout pull/5645

Update a local copy of the PR:
$ git checkout pull/5645
$ git pull https://git.openjdk.java.net/jdk pull/5645/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 5645

View PR using the GUI difftool:
$ git pr show -t 5645

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/5645.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Sep 23, 2021

👋 Welcome back bpb! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Sep 23, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Sep 23, 2021

@bplb The following label will be automatically applied to this pull request:

  • nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the nio label Sep 23, 2021
@bplb
Copy link
Member Author

@bplb bplb commented Sep 23, 2021

Throughput measured without the change for various stream lengths was

Benchmark                          (length)   Mode  Cnt       Score      Error  Units
ChannelInputStream.readAllBytes        1000  thrpt   10  418393.824 ± 2969.676  ops/s
ChannelInputStream.readAllBytes       10000  thrpt   10  196087.387 ± 1947.272  ops/s
ChannelInputStream.readAllBytes      100000  thrpt   10   29089.090 ±  430.841  ops/s
ChannelInputStream.readAllBytes     1000000  thrpt   10    2534.655 ±    5.373  ops/s
ChannelInputStream.readAllBytes    10000000  thrpt   10     209.193 ±    1.106  ops/s
ChannelInputStream.readAllBytes   100000000  thrpt   10      20.010 ±    0.047  ops/s
ChannelInputStream.readAllBytes  1000000000  thrpt   10       1.455 ±    0.204  ops/s
ChannelInputStream.readNBytes          1000  thrpt   10  544002.863 ± 1900.158  ops/s
ChannelInputStream.readNBytes         10000  thrpt   10  421766.791 ± 2928.367  ops/s
ChannelInputStream.readNBytes        100000  thrpt   10   57035.078 ±  214.834  ops/s
ChannelInputStream.readNBytes       1000000  thrpt   10    5855.087 ±   52.283  ops/s
ChannelInputStream.readNBytes      10000000  thrpt   10     427.560 ±    2.925  ops/s
ChannelInputStream.readNBytes     100000000  thrpt   10      40.118 ±    0.091  ops/s
ChannelInputStream.readNBytes    1000000000  thrpt   10       3.315 ±    0.434  ops/s

and with the change in place was

Benchmark                          (length)   Mode  Cnt       Score      Error  Units
ChannelInputStream.readAllBytes        1000  thrpt   10  337785.548 ± 2702.517  ops/s
ChannelInputStream.readAllBytes       10000  thrpt   10  245969.721 ± 1362.465  ops/s
ChannelInputStream.readAllBytes      100000  thrpt   10   75736.303 ±  321.443  ops/s
ChannelInputStream.readAllBytes     1000000  thrpt   10    8835.464 ±   73.547  ops/s
ChannelInputStream.readAllBytes    10000000  thrpt   10     370.141 ±    0.675  ops/s
ChannelInputStream.readAllBytes   100000000  thrpt   10      34.029 ±    0.152  ops/s
ChannelInputStream.readAllBytes  1000000000  thrpt   10       3.414 ±    0.032  ops/s
ChannelInputStream.readNBytes          1000  thrpt   10  354884.272 ± 2264.808  ops/s
ChannelInputStream.readNBytes         10000  thrpt   10  296775.457 ± 4296.873  ops/s
ChannelInputStream.readNBytes        100000  thrpt   10  120579.221 ±  374.626  ops/s
ChannelInputStream.readNBytes       1000000  thrpt   10   18079.568 ±  109.945  ops/s
ChannelInputStream.readNBytes      10000000  thrpt   10     860.772 ±    2.333  ops/s
ChannelInputStream.readNBytes     100000000  thrpt   10      69.266 ±    0.293  ops/s
ChannelInputStream.readNBytes    1000000000  thrpt   10       6.840 ±    0.016  ops/s

In effect, degraded performance for very small lengths is traded off against better performance for larger lengths. Note however that the readNBytes() measurements were made for len = length/2 and so are less telling. Otherwise said, the readAllBytes() results are more straightforwardly interpreted. Adding a threshold was not investigated.

Unfortunately this is mostly code duplication from FileInputStream. It would be better if there were a common location where the implementation could be placed and used by both cases.

@mlbridge
Copy link

@mlbridge mlbridge bot commented Sep 23, 2021

Webrevs

@AlanBateman
Copy link
Contributor

@AlanBateman AlanBateman commented Sep 24, 2021

The overrides of readAllBytes and readNBytes in ChannelInputStream look okay.
I don't have time to spend on the test right now but I wonder if we should create a new TestNG with simple tests rather than a expanding Basic.testNewInputStream with lots of cases. I can't immediately see if we have tests for an empty file, position at EOF, files > 2GB, and other cases that we might want to add.

@bplb
Copy link
Member Author

@bplb bplb commented Sep 24, 2021

The overrides of readAllBytes and readNBytes in ChannelInputStream look okay.
I don't have time to spend on the test right now but I wonder if we should create a new TestNG with simple tests rather than a expanding Basic.testNewInputStream with lots of cases. I can't immediately see if we have tests for an empty file, position at EOF, files > 2GB, and other cases that we might want to add.

The third commit, version 02, adds a TestNG test which replaces the previously expanded Basic.testNewInputStream(). The new test checks the expected behavior in normal operation, for a file > 2 GB, for an empty file, and for a file at EOF.

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class ChannelInputStream {
Copy link
Contributor

@AlanBateman AlanBateman Sep 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests for Channels API are in java/nio/channels/Channels and we should try keep them in the same tree if possible. If the test is focused on readXXXBytes then we should come up with a better name for the test, maybe "ReadXBytesTest" rather than "ChannelInputStream".

}
} finally {
file.delete();
}
Copy link
Contributor

@AlanBateman AlanBateman Sep 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be easier to maintain, and allow us to add more cases, if we split this up so there are separate tests for empty files, OOME, etc.
Are you planning to update the test cover the case that the initial position is > 0?
Has the setup been copied from an older test, I'm wondering why it needs to use File and FileChannel::getChannel as you should be able to use FileChannel.open.

}
} finally {
file.delete();
}
Copy link
Contributor

@AlanBateman AlanBateman Sep 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be easier to maintain and add new cases if this one were split up too.

Copy link
Contributor

@AlanBateman AlanBateman left a comment

Thanks for the update, the test is much better now.
@LanceAndersen Do you have any comments on this one?

@openjdk
Copy link

@openjdk openjdk bot commented Oct 1, 2021

@bplb This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8268435: (ch) ChannelInputStream could override readAllBytes

Reviewed-by: alanb, lancea

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 209 new commits pushed to the master branch:

  • 3d7671b: 8274562: (fs) UserDefinedFileAttributeView doesn't correctly determine if supported when using OverlayFS
  • c05dc26: 8274435: EXCEPTION_ACCESS_VIOLATION in BFSClosure::closure_impl
  • cc14c6f: 8274227: Remove "impl.prefix" jdk system property usage from InetAddress
  • 292d7bb: 8274363: Transitively sealed classes not considered exhaustive in switches
  • 1887028: 8269113: Javac throws when compiling switch (null)
  • b7b78ff: 8274561: sun/net/ftp/TestFtpTimeValue.java timed out on slow machines
  • 5e4b514: 8274550: c2i entry barriers read int as long on PPC
  • 2e690ba: 8274322: Problems with oopDesc construction
  • a8edd1b: 8274527: Minimal VM build fails after JDK-8273459
  • 7326481: 8274393: Suppress more warnings on non-serializable non-transient instance fields in security libs
  • ... and 199 more: https://git.openjdk.java.net/jdk/compare/cbffecc61e4a9ac1172926ef4f20d918d73adde9...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Oct 1, 2021
Copy link
Contributor

@LanceAndersen LanceAndersen left a comment

Hi Brian,

Looks good overall. A couple minor suggestions for you to consider but I am good to go either way.


private static final int BIG_LENGTH = ArraysSupport.SOFT_MAX_ARRAY_LENGTH;
private static final long HUGE_LENGTH = Integer.MAX_VALUE + 27L;

Copy link
Contributor

@LanceAndersen LanceAndersen Oct 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps consider adding a comment for the above length constants

assertNotNull(bytes);
assertEquals(bytes.length, (long)length);
}
);
Copy link
Contributor

@LanceAndersen LanceAndersen Oct 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been suggested in past PR reviews (including a few of mine) that we use assertThrows(). vs the expectedException annotation element. Something you might want to consider using here

Copy link
Contributor

@LanceAndersen LanceAndersen left a comment

Thank you Brian for the last updates. Looks good. :-)

@bplb
Copy link
Member Author

@bplb bplb commented Oct 1, 2021

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Oct 1, 2021

Going to push as commit 0786d8b.
Since your change was applied there have been 211 commits pushed to the master branch:

  • bb4500d: 8274465: Fix javax/swing/text/ParagraphView/6364882/bug6364882.java failures
  • 05d3860: 8274605: Fix predicate guarantees on returned values in (Doc)SourcePositions
  • 3d7671b: 8274562: (fs) UserDefinedFileAttributeView doesn't correctly determine if supported when using OverlayFS
  • c05dc26: 8274435: EXCEPTION_ACCESS_VIOLATION in BFSClosure::closure_impl
  • cc14c6f: 8274227: Remove "impl.prefix" jdk system property usage from InetAddress
  • 292d7bb: 8274363: Transitively sealed classes not considered exhaustive in switches
  • 1887028: 8269113: Javac throws when compiling switch (null)
  • b7b78ff: 8274561: sun/net/ftp/TestFtpTimeValue.java timed out on slow machines
  • 5e4b514: 8274550: c2i entry barriers read int as long on PPC
  • 2e690ba: 8274322: Problems with oopDesc construction
  • ... and 201 more: https://git.openjdk.java.net/jdk/compare/cbffecc61e4a9ac1172926ef4f20d918d73adde9...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Oct 1, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Oct 1, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Oct 1, 2021

@bplb Pushed as commit 0786d8b.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@bplb bplb deleted the ChannelInputStream-readAllBytes-8268435 branch Oct 1, 2021
@fweimer-rh
Copy link

@fweimer-rh fweimer-rh commented Oct 5, 2021

This introduced JDK-8274780. Fixed proposed via #5824.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated nio
5 participants