8080272 Refactor I/O stream copying to use InputStream.transferTo/readAllBytes and Files.copy

[turbanoff]

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8080272: Refactor I/O stream copying to use InputStream.transferTo/readAllBytes and Files.copy

Reviewers

• Maurizio Cimadamore (@mcimadamore - Reviewer) ⚠️ Review applies to 6614a10
• Alan Bateman (@AlanBateman - Reviewer) ⚠️ Review applies to 6e71e96

Download

$ git fetch https://git.openjdk.java.net/jdk pull/1853/head:pull/1853
$ git checkout pull/1853

[bridgekeeper]

👋 Welcome back turbanoff! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@turbanoff The following labels will be automatically applied to this pull request:

• 2d
• awt
• compiler
• core-libs
• hotspot-jfr
• net
• nio
• security
• serviceability
• sound
• swing

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 12: Full - Incremental (96920ee)
• 11: Full - Incremental (1b30471)
• 10: Full - Incremental (6e71e96)
• 09: Full - Incremental (6614a10)
• 08: Full - Incremental (6a8a3ae)
• 07: Full - Incremental (94e9981)
• 06: Full - Incremental (fceb20e)
• 05: Full - Incremental (6f8ec71136e45b4f4fa154d133fefc464e4ad7eb)
• 04: Full - Incremental (ec602c1a6d5c7e7bcf4fd6f5ea1e0d223fb4eb5f)
• 03: Full - Incremental (c4133d32683287903c5e3098d12e1e30fd841546)
• 02: Full - Incremental (2ae884716f29643e429d66f7d1f1b14a9f2b4ab1)
• 01: Full - Incremental (90b1a4559df56cb4c1c1685d7aed1433ae242883)
• 00: Full (fa3ae2018b294e945121ac8a1a78025a48ead1de)

[AlanBateman]

jrtfs is compiled twice, the second is to --release 8 so it can be packaged into jrt-fs.jar for use by IDEs/tools running on older JDK releases. So need to be careful with the changes here as it will likely causing build breakage.

We try to keep the changes to ASM to a minimum, might be better to leave this change out of the patch.

One or two of the sources changes should probably uses Files.copy, e.g. ZipPath, sjavac/CopyFile.java.

[turbanoff]

One or two of the sources changes should probably uses Files.copy, e.g. ZipPath, sjavac/CopyFile.java.

Good idea! Replaced in few places. But not in ZipPath: it's actually implementation of underlying call from Files.copy: jdk.nio.zipfs.ZipFileSystemProvider#copy. So, Files.copy call will be recursive.

[prrace]

So these changes are all over the place which means no one person knows how to test all of it.
Have you run the sound, swing tests, and the printing tests on unix and windows ?
For printing for sure you'll need to do some manual testing.

[mrserb]

I already suggested this, but anyway please always extract the changes to the java.desktop module to the separate PR.

[turbanoff]

I've extracted changes in java.desktop into separate PR #1856
Reverted this changes from current PR.

[AlanBateman]

Probably best to drop the changes to src/java.xml.crypto/share/classes/com/sun/org/apache/xml/internal/security/** too as it gets refreshed periodically from the upstream Apache Santuario project.

[stsypanov]

I'm not a reviewer, but still think we could simplify sun.security.tools.keytool.Main

[mcimadamore]

The changes to sjavac changes look good

[openjdk]

@turbanoff This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8080272: Refactor I/O stream copying to use InputStream.transferTo/readAllBytes and Files.copy

Reviewed-by: mcimadamore, alanb

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 1076 new commits pushed to the master branch:

• 0c718ab: 8262277: URLClassLoader.getResource throws undocumented IllegalArgumentException
• 4f1cda4: 8263387: G1GarbageCollection JFR event gets gc phase, not gc type
• 5ab5244: 8263514: Minor issue in JavacFileManager.SortFiles.REVERSE
• 771b146: 8245025: MoveAndUpdateClosure::do_addr calls function with side-effects in an assert
• 46d78f0: 6539707: (fc) MappedByteBuffer.force() method throws an IOException in a very simple test
• 189289d: 8262326: MaxMetaspaceSize does not have to be aligned to metaspace commit alignment
• d825198: 8263556: remove @modules java.base from tests
• d6b5e18: 8263191: Consolidate ThreadInVMfromJavaNoAsyncException and ThreadBlockInVMWithDeadlockCheck with existing wrappers
• 80cdf78: 8263544: Unused argument in ConstantPoolCacheEntry::set_field()
• c0176c4: 8263552: Use String.valueOf() for char-to-String conversions
• ... and 1066 more: https://git.openjdk.java.net/jdk/compare/fa50877c2e86d1a4e00724dd29d934f52d51f42c...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@wangweij, @mcimadamore, @AlanBateman) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

[AlanBateman]

Thanks for perceiving with this one. I think you've addressed all issues in the latest revision.

[turbanoff]

/integrate

[openjdk]

@turbanoff
Your change (at version 6e71e96) is now ready to be sponsored by a Committer.

[FrauBoes]

Hi @turbanoff, I'm happy to sponsor but I see two comments by @marschall - have they been addressed?
#1853 (comment)
#1853 (comment)

[turbanoff]

@FrauBoes fixed in the last commit. Is there any way to de-integrate PR (to include last commit)?

[FrauBoes]

To re-integrate, just run the /integrate command again.

Before doing that, could someone from security libs acknowledge the changes in their area?

[seanjmullan]

I don't think you should remove lines 228-232. These methods are called by methods of CertificateFactory that take InputStream (which may contain a stream of security data) and they are designed such that they try to read one Certificate, CRL, or CertPath from the InputStream and leave the InputStream ready to parse the next structure instead of consuming all of the bytes. Thus they check if the InputStream supports mark in order to try to preserve that behavior. If mark is not supported, then it's ok to use InputStream.readAllBytes, otherwise, leave the stream as-is.

[turbanoff]

As I can see only ByteArrayInputStream is actually passed in InputStream in current JDK code:

PKCS7 pkcs7 = new PKCS7(is.readAllBytes());
    private static List<X509Certificate> parsePKCS7(InputStream is)
        certs = parsePKCS7(is);
            public X509CertPath(InputStream is, String encoding)
                return new X509CertPath(new ByteArrayInputStream(data), encoding);

PKCS7 pkcs7 = new PKCS7(is.readAllBytes());
    private static List<X509Certificate> parsePKCS7(InputStream is)
        certs = parsePKCS7(is);
            public X509CertPath(InputStream is, String encoding)
                this(is, PKIPATH_ENCODING);
                    public X509CertPath(InputStream is) throws CertificateException {
                        return new X509CertPath(new ByteArrayInputStream(encoding));

Perhaps original marking approach was lost during refactoring?

[seanjmullan]

You are right, the code was refactored (way back in 2010) [1] to read one block at a time, so this check on mark can be removed. So, in this case, I think it is probably safe to just pass the InputStream as-is to PKCS7(InputStream), but maybe you can add a comment that says this should always be a ByteArrayInputStream. We can look at refactoring this code and clean it up a bit more later.

[1] https://hg.openjdk.java.net/jdk/jdk/rev/337ae296b6d6

[turbanoff]

I find implementation of sun.security.pkcs.PKCS7#PKCS7(java.io.InputStream) a bit confusing (or even buggy). It uses only InputStream.available() to parse block.
So I would prefer to not use it.

[FrauBoes]

@turbanoff Given that this PR has been lingering for a while, you could drop the change in X509CertPath.java for now and integrate the remaining changes. I'm happy to sponsor in that case.

[FrauBoes]

@turbanoff Tier 1-3 still all clear. If you /integrate, I will sponsor this tomorrow.

[turbanoff]

/integrate

[openjdk]

@turbanoff
Your change (at version 96920ee) is now ready to be sponsored by a Committer.

[FrauBoes]

/sponsor

[openjdk]

@FrauBoes @turbanoff Since your change was applied there have been 1086 commits pushed to the master branch:

• a31a23d: 8263595: Remove oop type punning in JavaCallArguments
• a1f6591: 8263589: Introduce JavaValue::get_oop/set_oop
• 20297a1: 8263577: C2: reachable nodes shouldn't have dead uses at the end of optimizations
• c484d89: 8263557: Possible NULL dereference in Arena::destruct_contents()
• ba35193: 8263559: Add missing initializers to VM_PopulateDumpSharedSpace
• e03a594: 8262504: Some CLHSDB command cannot know they run on remote debugger
• d896246: 8263420: Incorrect function name in NSAccessibilityStaticText native peer implementation
• 8c1112a: 8261916: gtest/GTestWrapper.java vmErrorTest.unimplemented1_vm_assert failed
• 1e57087: 8263392: Allow current thread to be specified in ExceptionMark
• 4d1c08c: 8263616: 'Deprecatd' typo in src/hotspot/share/classfile/classFileParser.cpp
• ... and 1076 more: https://git.openjdk.java.net/jdk/compare/fa50877c2e86d1a4e00724dd29d934f52d51f42c...master

Your commit was automatically rebased without conflicts.

Pushed as commit 68deb24.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.