8297632: InputStream.transferTo() method should specify what the return value should be when the number of bytes transfered is larger than Long.MAX_VALUE #11403

bplb · 2022-11-29T00:56:58Z

java.io.InputStream::transferTo could conceivably return a negative value if the count of bytes transferred overflows a long. Modify the method to limit the number of bytes transferred to Long.MAX_VALUE per invocation.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue
Change requires CSR request JDK-8297785 to be approved

Issues

JDK-8297632: InputStream.transferTo() method should specify what the return value should be when the number of bytes transfered is larger than Long.MAX_VALUE
JDK-8297785: InputStream.transferTo() method should specify what the return value should be when the number of bytes transfered is larger than Long.MAX_VALUE (CSR)

Reviewers

Alan Bateman (@AlanBateman - Reviewer)
Lance Andersen (@LanceAndersen - Reviewer) ⚠️ Review applies to 558cbc8a

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/11403/head:pull/11403
$ git checkout pull/11403

Update a local copy of the PR:
$ git checkout pull/11403
$ git pull https://git.openjdk.org/jdk pull/11403/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 11403

View PR using the GUI difftool:
$ git pr show -t 11403

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/11403.diff

…rn value should be when the number of bytes transfered is larger than Long.MAX_VALUE

bridgekeeper · 2022-11-29T00:57:52Z

👋 Welcome back bpb! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2022-11-29T01:01:05Z

@bplb The following label will be automatically applied to this pull request:

core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2022-11-29T01:05:43Z

Webrevs

AlanBateman

InputStream.transferTo is specified to transfer "all bytes from this input stream" so changing it to do a "short write" is major change and also creates a big inconsistency with java.io methods such as OutputStream.write that write all bytes.

I think other options will need to be explored. We have this same issue in other APIs where they specify that they return Long.MAX_VALUE when the size is larger.

It is likely that some of the overrides will need attention too.

bplb · 2022-11-29T18:24:25Z

Modified in 254d699 to transfer all bytes but clamp the return value to Long.MAX_VALUE. Overrides in java.io were already examined and do not appear problematic but a second check would not hurt.

AlanBateman · 2022-11-29T18:32:07Z

/csr

openjdk · 2022-11-29T18:34:12Z

@AlanBateman has indicated that a compatibility and specification (CSR) request is needed for this pull request.

@bplb please create a CSR request for issue JDK-8297632 with the correct fix version. This pull request cannot be integrated until the CSR request is approved.

AlanBateman · 2022-11-29T18:35:40Z

I assume Reader.transferTo will need similar treatment.

bplb · 2022-11-29T18:37:24Z

I assume Reader.transferTo will need similar treatment.

Looks like it. I will re-scan for other places as well.

AlanBateman · 2022-11-29T18:47:35Z

I assume this will require adjustments to SequenceInputStream.transferTo proposed in pull/11248.

bplb · 2022-11-29T18:52:23Z

I assume this will require adjustments to SequenceInputStream.transferTo proposed in pull/11248.

I will scan for all overrides.

bplb · 2022-11-29T18:56:23Z

I assume this will require adjustments to SequenceInputStream.transferTo proposed in pull/11248.

I will scan for all overrides.

Correction: yes, that one has the same problem.

…errides

AlanBateman · 2022-11-30T16:26:59Z

Can you check ZipInputStream.transferTo? It has a partial copy of a lot of the InputStream javadoc because it has to insert "current ZIP entry" into the text. It's not possible to have a ZIP file larger than Long.MAX_VALUE so there may be nothing to do there but we should check the javadoc.

bplb · 2022-11-30T17:33:17Z

Can you check ZipInputStream.transferTo?

That implementation just calls the overridden method, but the single sentence added to the InputStream::transferTo javadoc should probably be inserted here as well. Do you concur?

LanceAndersen · 2022-11-30T18:05:33Z

Can you check ZipInputStream.transferTo?

That implementation just calls the overridden method, but the single sentence added to the InputStream::transferTo javadoc should probably be inserted here as well. Do you concur?

I believe that makes sense

AlanBateman · 2022-12-01T07:54:18Z

Latest version (558cbc8) looks okay but this change will likely have a tail so I think we should wait until after the fork for JDK 20 to integrate this. To that end, I've changed the fixVersion to 21 and the same on the CSR, I hope that is okay.

bplb · 2022-12-01T16:27:02Z

Latest version (558cbc8) looks okay but this change will likely have a tail so I think we should wait until after the fork for JDK 20 to integrate this.

It can wait a week.

To that end, I've changed the fixVersion to 21 and the same on the CSR, I hope that is okay.

No problem.

bridgekeeper · 2023-01-05T22:43:47Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bridgekeeper · 2023-02-03T02:18:42Z

@bplb This pull request has been inactive for more than 8 weeks and will now be automatically closed. If you would like to continue working on this pull request in the future, feel free to reopen it! This can be done using the /open pull request command.

bplb · 2023-02-03T16:35:47Z

/open

openjdk · 2023-02-03T16:37:02Z

@bplb This pull request is now open

AlanBateman

This version looks okay.

src/java.base/share/classes/java/io/SequenceInputStream.java

openjdk · 2023-02-13T20:53:42Z

@bplb This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8297632: InputStream.transferTo() method should specify what the return value should be when the number of bytes transfered is larger than Long.MAX_VALUE

Reviewed-by: alanb, lancea

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 894 new commits pushed to the master branch:

d503c66: 8302155: [AIX] NUM_LCPU is not required variable
c37e9d1: 8298293: NMT: os::realloc() should verify that flags do not change between reallocations
101db26: 8301697: [s390] Optimized-build is broken
f4d4fa5: 8300255: Introduce interface for GC oop verification in the assembler
99b6c0e: 8302289: RISC-V: Use bgez instruction in arraycopy_simple_check when possible
57aef85: 8301838: PPC: continuation yield intrinsic: exception check not needed if yield succeeded
df93880: 8301942: java/net/httpclient/DigestEchoClientSSL.java fail with -Xcomp
0025764: 8040793: vmTestbase/nsk/monitoring/stress/lowmem fails on calling isCollectionUsageThresholdExceeded()
1f9c110: 8301958: Reduce Arrays.copyOf/-Range overheads
cb81073: 8300139: [AIX] Use pthreads to avoid JNI_createVM call from primordial thread
... and 884 more: https://git.openjdk.org/jdk/compare/d35e840024b80f9f686fb5522dc03b2c9233a6d3...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

bplb · 2023-02-14T16:28:45Z

/integrate

openjdk · 2023-02-14T16:31:03Z

Going to push as commit 5b2d430.
Since your change was applied there have been 911 commits pushed to the master branch:

f7dee77: 8301274: update for deprecated sprintf for security components
ec901f2: 8301279: update for deprecated sprintf for management components
8933c2d: 8298278: JFR: Turn MEMFLAGS into a type for use with the NativeMemoryUsage events
77519e5: 8302354: InstanceKlass init state/thread should be atomic
2ef001e: 8207017: Type annotations on anonymous classes in initializer blocks not written to class file
8c2c8b3: 8295344: Harden runtime/StackGuardPages/TestStackGuardPages.java
6d4b02b: 8302324: Inheritance tree does not show correct type parameters/arguments
7dfe75c: 8301842: JFR: increase checkpoint event size for stacktrace and string pool
66742c8: 8302368: [ZGC] Client build fails after JDK-8300255
7c50ab1: 8225409: G1: Remove the Hot Card Cache
... and 901 more: https://git.openjdk.org/jdk/compare/d35e840024b80f9f686fb5522dc03b2c9233a6d3...master

Your commit was automatically rebased without conflicts.

openjdk · 2023-02-14T16:31:19Z

@bplb Pushed as commit 5b2d430.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

vlsi · 2023-12-14T06:12:40Z

src/java.base/share/classes/java/io/SequenceInputStream.java

+            long transferred = 0;
            while (in != null) {
-                c += in.transferTo(out);
+                if (transferred < Long.MAX_VALUE) {
+                    try {
+                        transferred = Math.addExact(transferred, in.transferTo(out));
+                    } catch (ArithmeticException ignore) {
+                        return Long.MAX_VALUE;


@bplb , this looks like a bug to me: once transferred reaches Long.MAX_VALUE the transfer loop will terminate and the transfer will stop even in the case there are more streams available in the sequence.

I think the proper code should be transferred = Long.MAX_VALUE instead of return Long.MAX_VALUE (and there should be no if (transferred < Long.MAX_VALUE) check)

What do you think?

What do you think?

I think that you are looking at an obsolete repository:

jdk/src/java.base/share/classes/java/io/InputStream.java

Line 802 in c328f95

transferred = Long.MAX_VALUE;

I mean SequenceInputStream, not the base class:

jdk/src/java.base/share/classes/java/io/SequenceInputStream.java

Line 249 in c328f95

return Long.MAX_VALUE;

Could you please double-check?

IMHO @vlsi is right. It is incorrect that we stop the transfer in the overflow case. We should fix it as he suggested. I can do that if you like.

Right, the base class. The suggested change was made for InputStream in 254d699. It looks like it should be in SequenceInputStream as well. I created JDK-8322141 to track this.

We don't have any tests to check the behavior of the transferTo overrides when they transfer more than nine quintillion bytes so dependent on catching issues like this during code review.

The solution proposed by @vlsi and outlined by @bplb in JDK-8322141 now can be reviewed in #17119. I suggest to continue the discussion there.

8297632: InputStream.transferTo() method should specify what the retu…

11d73c6

…rn value should be when the number of bytes transfered is larger than Long.MAX_VALUE

openjdk bot added the rfr Pull request is ready for review label Nov 29, 2022

openjdk bot added the core-libs core-libs-dev@openjdk.org label Nov 29, 2022

AlanBateman suggested changes Nov 29, 2022

View reviewed changes

8297632: Transfer all bytes but clamp returned value to Long.MAX_VALUE

254d699

openjdk bot added the csr Pull request needs approved CSR before integration label Nov 29, 2022

8297632: Add Reader::transferTo and select InputStream::transferTo ov…

6751627

…errides

bplb mentioned this pull request Nov 29, 2022

JDK-8297298 - SequenceInputStream should override transferTo #11248

Closed

3 tasks

8297632: Add verbiage to ZipInputStream::transferTo

558cbc8

LanceAndersen approved these changes Nov 30, 2022

View reviewed changes

bplb added 2 commits December 8, 2022 11:22

Merge

c7c4266

8297632: Propagate change to SequenceInputStream

446771d

bridgekeeper bot closed this Feb 3, 2023

openjdk bot reopened this Feb 3, 2023

AlanBateman approved these changes Feb 10, 2023

View reviewed changes

src/java.base/share/classes/java/io/SequenceInputStream.java Show resolved Hide resolved

8297632: Align SequenceInputStream style with other changes in patch

6bdf322

AlanBateman approved these changes Feb 10, 2023

View reviewed changes

openjdk bot removed the csr Pull request needs approved CSR before integration label Feb 13, 2023

openjdk bot added the ready Pull request is ready to be integrated label Feb 13, 2023

openjdk bot added the integrated Pull request has been integrated label Feb 14, 2023

openjdk bot closed this Feb 14, 2023

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Feb 14, 2023

bplb deleted the InputStream-transferTo-8297632 branch February 14, 2023 20:41

vlsi reviewed Dec 14, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8297632: InputStream.transferTo() method should specify what the return value should be when the number of bytes transfered is larger than Long.MAX_VALUE #11403

8297632: InputStream.transferTo() method should specify what the return value should be when the number of bytes transfered is larger than Long.MAX_VALUE #11403

bplb commented Nov 29, 2022 •

edited by openjdk bot

Loading

bridgekeeper bot commented Nov 29, 2022

openjdk bot commented Nov 29, 2022

mlbridge bot commented Nov 29, 2022 •

edited

Loading

AlanBateman left a comment

bplb commented Nov 29, 2022

AlanBateman commented Nov 29, 2022

openjdk bot commented Nov 29, 2022

AlanBateman commented Nov 29, 2022

bplb commented Nov 29, 2022

AlanBateman commented Nov 29, 2022

bplb commented Nov 29, 2022

bplb commented Nov 29, 2022

AlanBateman commented Nov 30, 2022

bplb commented Nov 30, 2022

LanceAndersen commented Nov 30, 2022

AlanBateman commented Dec 1, 2022

bplb commented Dec 1, 2022

bridgekeeper bot commented Jan 5, 2023

bridgekeeper bot commented Feb 3, 2023

bplb commented Feb 3, 2023

openjdk bot commented Feb 3, 2023

AlanBateman left a comment

openjdk bot commented Feb 13, 2023

bplb commented Feb 14, 2023

openjdk bot commented Feb 14, 2023

openjdk bot commented Feb 14, 2023

vlsi Dec 14, 2023 •

edited

Loading

bplb Dec 14, 2023

vlsi Dec 14, 2023

mkarg Dec 14, 2023

bplb Dec 14, 2023

AlanBateman Dec 15, 2023

mkarg Dec 15, 2023

8297632: InputStream.transferTo() method should specify what the return value should be when the number of bytes transfered is larger than Long.MAX_VALUE #11403

8297632: InputStream.transferTo() method should specify what the return value should be when the number of bytes transfered is larger than Long.MAX_VALUE #11403

Conversation

bplb commented Nov 29, 2022 • edited by openjdk bot Loading

Progress

Issues

Reviewers

Reviewing

bridgekeeper bot commented Nov 29, 2022

openjdk bot commented Nov 29, 2022

mlbridge bot commented Nov 29, 2022 • edited Loading

Webrevs

AlanBateman left a comment

Choose a reason for hiding this comment

bplb commented Nov 29, 2022

AlanBateman commented Nov 29, 2022

openjdk bot commented Nov 29, 2022

AlanBateman commented Nov 29, 2022

bplb commented Nov 29, 2022

AlanBateman commented Nov 29, 2022

bplb commented Nov 29, 2022

bplb commented Nov 29, 2022

AlanBateman commented Nov 30, 2022

bplb commented Nov 30, 2022

LanceAndersen commented Nov 30, 2022

AlanBateman commented Dec 1, 2022

bplb commented Dec 1, 2022

bridgekeeper bot commented Jan 5, 2023

bridgekeeper bot commented Feb 3, 2023

bplb commented Feb 3, 2023

openjdk bot commented Feb 3, 2023

AlanBateman left a comment

Choose a reason for hiding this comment

openjdk bot commented Feb 13, 2023

bplb commented Feb 14, 2023

openjdk bot commented Feb 14, 2023

openjdk bot commented Feb 14, 2023

vlsi Dec 14, 2023 • edited Loading

Choose a reason for hiding this comment

bplb Dec 14, 2023

Choose a reason for hiding this comment

vlsi Dec 14, 2023

Choose a reason for hiding this comment

mkarg Dec 14, 2023

Choose a reason for hiding this comment

bplb Dec 14, 2023

Choose a reason for hiding this comment

AlanBateman Dec 15, 2023

Choose a reason for hiding this comment

mkarg Dec 15, 2023

Choose a reason for hiding this comment

bplb commented Nov 29, 2022 •

edited by openjdk bot

Loading

mlbridge bot commented Nov 29, 2022 •

edited

Loading

vlsi Dec 14, 2023 •

edited

Loading