8268435: (ch) ChannelInputStream could override readAllBytes #5645

bplb · 2021-09-23T01:52:31Z

This change would override readAllBytes() and readNBytes(int) in ChannelInputStream thereby improving performance for all but smaller streams.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8268435: (ch) ChannelInputStream could override readAllBytes

Reviewers

Alan Bateman (@AlanBateman - Reviewer) ⚠️ Review applies to 2b131c0
Lance Andersen (@LanceAndersen - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/5645/head:pull/5645
$ git checkout pull/5645

Update a local copy of the PR:
$ git checkout pull/5645
$ git pull https://git.openjdk.java.net/jdk pull/5645/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 5645

View PR using the GUI difftool:
$ git pr show -t 5645

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/5645.diff

bridgekeeper · 2021-09-23T01:53:34Z

👋 Welcome back bpb! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2021-09-23T01:56:15Z

@bplb The following label will be automatically applied to this pull request:

nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

bplb · 2021-09-23T02:00:55Z

Throughput measured without the change for various stream lengths was

Benchmark                          (length)   Mode  Cnt       Score      Error  Units
ChannelInputStream.readAllBytes        1000  thrpt   10  418393.824 ± 2969.676  ops/s
ChannelInputStream.readAllBytes       10000  thrpt   10  196087.387 ± 1947.272  ops/s
ChannelInputStream.readAllBytes      100000  thrpt   10   29089.090 ±  430.841  ops/s
ChannelInputStream.readAllBytes     1000000  thrpt   10    2534.655 ±    5.373  ops/s
ChannelInputStream.readAllBytes    10000000  thrpt   10     209.193 ±    1.106  ops/s
ChannelInputStream.readAllBytes   100000000  thrpt   10      20.010 ±    0.047  ops/s
ChannelInputStream.readAllBytes  1000000000  thrpt   10       1.455 ±    0.204  ops/s
ChannelInputStream.readNBytes          1000  thrpt   10  544002.863 ± 1900.158  ops/s
ChannelInputStream.readNBytes         10000  thrpt   10  421766.791 ± 2928.367  ops/s
ChannelInputStream.readNBytes        100000  thrpt   10   57035.078 ±  214.834  ops/s
ChannelInputStream.readNBytes       1000000  thrpt   10    5855.087 ±   52.283  ops/s
ChannelInputStream.readNBytes      10000000  thrpt   10     427.560 ±    2.925  ops/s
ChannelInputStream.readNBytes     100000000  thrpt   10      40.118 ±    0.091  ops/s
ChannelInputStream.readNBytes    1000000000  thrpt   10       3.315 ±    0.434  ops/s

and with the change in place was

Benchmark                          (length)   Mode  Cnt       Score      Error  Units
ChannelInputStream.readAllBytes        1000  thrpt   10  337785.548 ± 2702.517  ops/s
ChannelInputStream.readAllBytes       10000  thrpt   10  245969.721 ± 1362.465  ops/s
ChannelInputStream.readAllBytes      100000  thrpt   10   75736.303 ±  321.443  ops/s
ChannelInputStream.readAllBytes     1000000  thrpt   10    8835.464 ±   73.547  ops/s
ChannelInputStream.readAllBytes    10000000  thrpt   10     370.141 ±    0.675  ops/s
ChannelInputStream.readAllBytes   100000000  thrpt   10      34.029 ±    0.152  ops/s
ChannelInputStream.readAllBytes  1000000000  thrpt   10       3.414 ±    0.032  ops/s
ChannelInputStream.readNBytes          1000  thrpt   10  354884.272 ± 2264.808  ops/s
ChannelInputStream.readNBytes         10000  thrpt   10  296775.457 ± 4296.873  ops/s
ChannelInputStream.readNBytes        100000  thrpt   10  120579.221 ±  374.626  ops/s
ChannelInputStream.readNBytes       1000000  thrpt   10   18079.568 ±  109.945  ops/s
ChannelInputStream.readNBytes      10000000  thrpt   10     860.772 ±    2.333  ops/s
ChannelInputStream.readNBytes     100000000  thrpt   10      69.266 ±    0.293  ops/s
ChannelInputStream.readNBytes    1000000000  thrpt   10       6.840 ±    0.016  ops/s

In effect, degraded performance for very small lengths is traded off against better performance for larger lengths. Note however that the readNBytes() measurements were made for len = length/2 and so are less telling. Otherwise said, the readAllBytes() results are more straightforwardly interpreted. Adding a threshold was not investigated.

Unfortunately this is mostly code duplication from FileInputStream. It would be better if there were a common location where the implementation could be placed and used by both cases.

mlbridge · 2021-09-23T02:01:07Z

Webrevs

src/java.base/share/classes/sun/nio/ch/ChannelInputStream.java

AlanBateman · 2021-09-24T12:41:56Z

The overrides of readAllBytes and readNBytes in ChannelInputStream look okay.
I don't have time to spend on the test right now but I wonder if we should create a new TestNG with simple tests rather than a expanding Basic.testNewInputStream with lots of cases. I can't immediately see if we have tests for an empty file, position at EOF, files > 2GB, and other cases that we might want to add.

…java

bplb · 2021-09-24T22:33:15Z

The overrides of readAllBytes and readNBytes in ChannelInputStream look okay.
I don't have time to spend on the test right now but I wonder if we should create a new TestNG with simple tests rather than a expanding Basic.testNewInputStream with lots of cases. I can't immediately see if we have tests for an empty file, position at EOF, files > 2GB, and other cases that we might want to add.

The third commit, version 02, adds a TestNG test which replaces the previously expanded Basic.testNewInputStream(). The new test checks the expected behavior in normal operation, for a file > 2 GB, for an empty file, and for a file at EOF.

AlanBateman · 2021-09-25T11:41:44Z

test/jdk/sun/nio/ch/ChannelInputStream.java

+import org.testng.annotations.DataProvider;
+import org.testng.annotations.Test;
+
+public class ChannelInputStream {


The tests for Channels API are in java/nio/channels/Channels and we should try keep them in the same tree if possible. If the test is focused on readXXXBytes then we should come up with a better name for the test, maybe "ReadXBytesTest" rather than "ChannelInputStream".

AlanBateman · 2021-09-25T11:47:06Z

test/jdk/sun/nio/ch/ChannelInputStream.java

+            }
+        } finally {
+            file.delete();
+        }


It might be easier to maintain, and allow us to add more cases, if we split this up so there are separate tests for empty files, OOME, etc.
Are you planning to update the test cover the case that the initial position is > 0?
Has the setup been copied from an older test, I'm wondering why it needs to use File and FileChannel::getChannel as you should be able to use FileChannel.open.

AlanBateman · 2021-09-25T11:48:01Z

test/jdk/sun/nio/ch/ChannelInputStream.java

+            }
+        } finally {
+            file.delete();
+        }


It might be easier to maintain and add new cases if this one were split up too.

AlanBateman

Thanks for the update, the test is much better now.
@LanceAndersen Do you have any comments on this one?

openjdk · 2021-10-01T07:42:17Z

@bplb This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8268435: (ch) ChannelInputStream could override readAllBytes

Reviewed-by: alanb, lancea

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 209 new commits pushed to the master branch:

3d7671b: 8274562: (fs) UserDefinedFileAttributeView doesn't correctly determine if supported when using OverlayFS
c05dc26: 8274435: EXCEPTION_ACCESS_VIOLATION in BFSClosure::closure_impl
cc14c6f: 8274227: Remove "impl.prefix" jdk system property usage from InetAddress
292d7bb: 8274363: Transitively sealed classes not considered exhaustive in switches
1887028: 8269113: Javac throws when compiling switch (null)
b7b78ff: 8274561: sun/net/ftp/TestFtpTimeValue.java timed out on slow machines
5e4b514: 8274550: c2i entry barriers read int as long on PPC
2e690ba: 8274322: Problems with oopDesc construction
a8edd1b: 8274527: Minimal VM build fails after JDK-8273459
7326481: 8274393: Suppress more warnings on non-serializable non-transient instance fields in security libs
... and 199 more: https://git.openjdk.java.net/jdk/compare/cbffecc61e4a9ac1172926ef4f20d918d73adde9...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

LanceAndersen

Hi Brian,

Looks good overall. A couple minor suggestions for you to consider but I am good to go either way.

LanceAndersen · 2021-10-01T11:02:07Z

test/jdk/java/nio/channels/Channels/ReadXBytes.java

+
+    private static final int  BIG_LENGTH = ArraysSupport.SOFT_MAX_ARRAY_LENGTH;
+    private static final long HUGE_LENGTH = Integer.MAX_VALUE + 27L;
+


Perhaps consider adding a comment for the above length constants

LanceAndersen · 2021-10-01T11:05:28Z

test/jdk/java/nio/channels/Channels/ReadXBytes.java

+                assertNotNull(bytes);
+                assertEquals(bytes.length, (long)length);
+            }
+        );


It has been suggested in past PR reviews (including a few of mine) that we use assertThrows(). vs the expectedException annotation element. Something you might want to consider using here

LanceAndersen

Thank you Brian for the last updates. Looks good. :-)

bplb · 2021-10-01T20:14:50Z

/integrate

openjdk · 2021-10-01T20:15:57Z

Going to push as commit 0786d8b.
Since your change was applied there have been 211 commits pushed to the master branch:

bb4500d: 8274465: Fix javax/swing/text/ParagraphView/6364882/bug6364882.java failures
05d3860: 8274605: Fix predicate guarantees on returned values in (Doc)SourcePositions
3d7671b: 8274562: (fs) UserDefinedFileAttributeView doesn't correctly determine if supported when using OverlayFS
c05dc26: 8274435: EXCEPTION_ACCESS_VIOLATION in BFSClosure::closure_impl
cc14c6f: 8274227: Remove "impl.prefix" jdk system property usage from InetAddress
292d7bb: 8274363: Transitively sealed classes not considered exhaustive in switches
1887028: 8269113: Javac throws when compiling switch (null)
b7b78ff: 8274561: sun/net/ftp/TestFtpTimeValue.java timed out on slow machines
5e4b514: 8274550: c2i entry barriers read int as long on PPC
2e690ba: 8274322: Problems with oopDesc construction
... and 201 more: https://git.openjdk.java.net/jdk/compare/cbffecc61e4a9ac1172926ef4f20d918d73adde9...master

Your commit was automatically rebased without conflicts.

openjdk · 2021-10-01T20:16:09Z

@bplb Pushed as commit 0786d8b.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

fweimer-rh · 2021-10-05T14:53:27Z

This introduced JDK-8274780. Fixed proposed via #5824.

8268435: (ch) ChannelInputStream could override readAllBytes

11baa37

openjdk bot added the rfr Pull request is ready for review label Sep 23, 2021

openjdk bot added the nio nio-dev@openjdk.org label Sep 23, 2021

delvh reviewed Sep 23, 2021

View reviewed changes

8268435: Use instanceof pattern matching

ca188f0

8268435: Create a TestNG test based on the previous changes to Basic.…

4ad8e2a

…java

AlanBateman suggested changes Sep 25, 2021

View reviewed changes

8268435: Split up @tests into multiple, smaller @tests

2b131c0

AlanBateman approved these changes Oct 1, 2021

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Oct 1, 2021

LanceAndersen approved these changes Oct 1, 2021

View reviewed changes

8268435: More comments; use assertThrows() instead of expectedExceptions

5d1eec3

LanceAndersen approved these changes Oct 1, 2021

View reviewed changes

openjdk bot closed this Oct 1, 2021

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 1, 2021

bplb deleted the ChannelInputStream-readAllBytes-8268435 branch October 1, 2021 21:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8268435: (ch) ChannelInputStream could override readAllBytes #5645

8268435: (ch) ChannelInputStream could override readAllBytes #5645

bplb commented Sep 23, 2021 •

edited by openjdk bot

Loading

bridgekeeper bot commented Sep 23, 2021

openjdk bot commented Sep 23, 2021

bplb commented Sep 23, 2021

mlbridge bot commented Sep 23, 2021 •

edited

Loading

AlanBateman commented Sep 24, 2021

bplb commented Sep 24, 2021

AlanBateman Sep 25, 2021 •

edited

Loading

AlanBateman Sep 25, 2021 •

edited

Loading

AlanBateman Sep 25, 2021 •

edited

Loading

AlanBateman left a comment

openjdk bot commented Oct 1, 2021 •

edited

Loading

LanceAndersen left a comment

LanceAndersen Oct 1, 2021

LanceAndersen Oct 1, 2021

LanceAndersen left a comment

bplb commented Oct 1, 2021

openjdk bot commented Oct 1, 2021

openjdk bot commented Oct 1, 2021

fweimer-rh commented Oct 5, 2021


		private static final int BIG_LENGTH = ArraysSupport.SOFT_MAX_ARRAY_LENGTH;
		private static final long HUGE_LENGTH = Integer.MAX_VALUE + 27L;

8268435: (ch) ChannelInputStream could override readAllBytes #5645

8268435: (ch) ChannelInputStream could override readAllBytes #5645

Conversation

bplb commented Sep 23, 2021 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented Sep 23, 2021

openjdk bot commented Sep 23, 2021

bplb commented Sep 23, 2021

mlbridge bot commented Sep 23, 2021 • edited Loading

Webrevs

AlanBateman commented Sep 24, 2021

bplb commented Sep 24, 2021

AlanBateman Sep 25, 2021 • edited Loading

Choose a reason for hiding this comment

AlanBateman Sep 25, 2021 • edited Loading

Choose a reason for hiding this comment

AlanBateman Sep 25, 2021 • edited Loading

Choose a reason for hiding this comment

AlanBateman left a comment

Choose a reason for hiding this comment

openjdk bot commented Oct 1, 2021 • edited Loading

LanceAndersen left a comment

Choose a reason for hiding this comment

LanceAndersen Oct 1, 2021

Choose a reason for hiding this comment

LanceAndersen Oct 1, 2021

Choose a reason for hiding this comment

LanceAndersen left a comment

Choose a reason for hiding this comment

bplb commented Oct 1, 2021

openjdk bot commented Oct 1, 2021

openjdk bot commented Oct 1, 2021

fweimer-rh commented Oct 5, 2021

bplb commented Sep 23, 2021 •

edited by openjdk bot

Loading

mlbridge bot commented Sep 23, 2021 •

edited

Loading

AlanBateman Sep 25, 2021 •

edited

Loading

AlanBateman Sep 25, 2021 •

edited

Loading

AlanBateman Sep 25, 2021 •

edited

Loading

openjdk bot commented Oct 1, 2021 •

edited

Loading