Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8298865: Excessive memory allocation in CipherOutputStream AEAD decryption #11693

Closed

Conversation

djelinski
Copy link
Member

@djelinski djelinski commented Dec 15, 2022

This patch modifies CipherOutputStream to avoid pointless memory allocations when decrypting data using AEAD ciphers. This is related to #11597, which fixed a similar issue in CipherInputStream.

Cipher.update does not output any data when doing AEAD decryption; all data is buffered, and is later returned in one shot from doFinal. Cipher.getOutputSize returns the buffer size required by doFinal, which increases after every update, triggering new allocation in ensureCapacity.

This patch addresses the issue by calling the update overload that returns the output buffer until one of the update calls returns some data. When that happens, we know that the cipher does not buffer everything until doFinal, and revert to original behavior.

This PR adds a new benchmark for AES/GCM encryption and decryption using CipherOutputStream.

Benchmark results before:

Benchmark                         (dataSize)  (keyLength)  (provider)   Mode  Cnt       Score      Error  Units
AESGCMCipherOutputStream.decrypt       16384          128              thrpt   40   27949,624 ±  301,408  ops/s
AESGCMCipherOutputStream.decrypt     1048576          128              thrpt   40      20,730 ±    0,875  ops/s
AESGCMCipherOutputStream.encrypt       16384          128              thrpt   40  175358,641 ± 4235,808  ops/s
AESGCMCipherOutputStream.encrypt     1048576          128              thrpt   40    2588,111 ±   35,469  ops/s

after:

Benchmark                         (dataSize)  (keyLength)  (provider)   Mode  Cnt       Score      Error  Units
AESGCMCipherOutputStream.decrypt       16384          128              thrpt   40   69644,217 ± 1081,032  ops/s
AESGCMCipherOutputStream.decrypt     1048576          128              thrpt   40     949,667 ±    9,431  ops/s
AESGCMCipherOutputStream.encrypt       16384          128              thrpt   40  173144,038 ± 3279,149  ops/s
AESGCMCipherOutputStream.encrypt     1048576          128              thrpt   40    2514,840 ±   87,935  ops/s

Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue

Issue

  • JDK-8298865: Excessive memory allocation in CipherOutputStream AEAD decryption

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/11693/head:pull/11693
$ git checkout pull/11693

Update a local copy of the PR:
$ git checkout pull/11693
$ git pull https://git.openjdk.org/jdk pull/11693/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 11693

View PR using the GUI difftool:
$ git pr show -t 11693

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/11693.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Dec 15, 2022

👋 Welcome back djelinski! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot changed the title 8298865 8298865: Excessive memory allocation in CipherOutputStream AEAD decryption Dec 15, 2022
@openjdk
Copy link

openjdk bot commented Dec 15, 2022

@djelinski The following label will be automatically applied to this pull request:

  • security

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the security security-dev@openjdk.org label Dec 15, 2022
@djelinski djelinski marked this pull request as ready for review December 15, 2022 14:18
@openjdk openjdk bot added the rfr Pull request is ready for review label Dec 15, 2022
@mlbridge
Copy link

mlbridge bot commented Dec 15, 2022

Webrevs

@mcpowers
Copy link
Contributor

LGTM

@@ -91,11 +91,17 @@ public class CipherOutputStream extends FilterOutputStream {
* Ensure obuffer is big enough for the next update or doFinal
* operation, given the input length {@code inLen} (in bytes)
*
* If obuffer is null/zero-sized, do not allocate a new buffer.
* This reduces allocation for AEAD ciphers that never return data from update
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: AEAD ciphers do return data for update() calls for encryption. Perhaps we should add "when used for decryption" or some other similar wordings to the above sentence? Same goes for the comment in CipherInputStream class.
Rest looks fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it's the way you read the sentence, when I read it bit before your comment, I interpreted the change as an open-ended comment where AEAD may or may not return without being specific. I'm neutral to changing it to specify encryption vs decryption.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good suggestion; I changed the wording now. What do you think?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good, thanks!

@openjdk
Copy link

openjdk bot commented Dec 15, 2022

@djelinski This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8298865: Excessive memory allocation in CipherOutputStream AEAD decryption

Reviewed-by: valeriep, ascarpino

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 14 new commits pushed to the master branch:

  • fa322e4: 8298709: Fix typos in src/java.desktop/ and various test classes of client component
  • e41686b: 8298710: Fix typos in test/jdk/sun/security/tools/jarsigner/
  • a336461: 8298081: DiagnoseSyncOnValueBasedClasses doesn't report useful information for virtual threads
  • 2bb727c: 8290899: java/lang/String/StringRepeat.java test requests too much heap on windows x86
  • 5412439: 8298187: (fs) BsdFileAttributeViews::setTimes does not support lastAccessTime on HFS+
  • 3cdbd87: 8298241: Replace C-style casts with JavaThread::cast
  • 10737e1: 8298468: Clean up class_loader parameters
  • 4b313b5: 8297798: Timeout with DTLSOverDatagram test template
  • ae8988e: 8297912: HotSpot Style Guide should permit alignas (Second Proposal Attempt)
  • 0ef3539: 8298416: Console should be declared sealed
  • ... and 4 more: https://git.openjdk.org/jdk/compare/5f63f7a742a1071a87ca69463bae6e04a44fe462...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Dec 15, 2022
@djelinski
Copy link
Member Author

/integrate

@openjdk
Copy link

openjdk bot commented Dec 20, 2022

Going to push as commit 36de61c.
Since your change was applied there have been 51 commits pushed to the master branch:

  • dd15d30: 8299043: test/jdk/javax/swing/AbstractButton/5049549/bug5049549.java fails with java.lang.NullPointerException
  • 05f9e76: 8298974: Add ftcolor.c to imported freetype sources
  • abc1297: 8299044: test/jdk/javax/swing/JComboBox/JComboBoxBorderTest.java fails on non mac
  • 5d330f5: 8299045: tools/doclint/BadPackageCommentTest.java fails after JDK-8298943
  • 40cb431: 8298943: Missing escapes for single quote marks in compiler.properties
  • 9194e91: 8298701: Cleanup SA entries in ProblemList-zgc.txt.
  • 4c927df: 8298470: Short cut java.lang.Object super class loading
  • 756a06d: 8299022: Linux ppc64le and s390x build issues after JDK-8160404
  • de0ce79: 8297801: printnm crashes with invalid address due to null pointer dereference
  • da38d43: 8296412: Special case infinite loops with unmerged backedges in IdealLoopTree::check_safepts
  • ... and 41 more: https://git.openjdk.org/jdk/compare/5f63f7a742a1071a87ca69463bae6e04a44fe462...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Dec 20, 2022
@openjdk openjdk bot closed this Dec 20, 2022
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Dec 20, 2022
@openjdk
Copy link

openjdk bot commented Dec 20, 2022

@djelinski Pushed as commit 36de61c.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@djelinski djelinski deleted the cipheroutputstream-allocation branch December 20, 2022 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated Pull request has been integrated security security-dev@openjdk.org
4 participants