8315135: Memory leak in the native implementation of Pack200.Unpacker.unpack() #2100

simonis · 2023-08-28T19:17:36Z

This issue was found by Yakov Shafranovich (yakovsh@amazon.com) who also provided the reproducer and proposed a fix.

The native implementation of the Pack200.Unpacker class included in OpenJDK 8 and 11 has a native and heap memory leak that gets triggered when corrupted files are processed. If the native NativeUnpack::start() method throws an exception (because of a corrupted input file) its caller NativeUnpack::run() fails to call the native NativeUnpack::finish() method which is responsible for freeing the allocated native memory and releasing the created global JNI handles. A Java application processing large number of malformed Pack200 files will eventually run either out of native memory or out of heap space and exit with an OutOfMemoryError.

The problem can be demonstrated with the following short test program which will exit with an OutOfMemoryError quite quickly if run with java -Xmx32m NativePack200POC:

import java.io.*;
import java.util.jar.*;

@SuppressWarnings("removal")
public class NativePack200POC {
  public static void main(String[] args) {
    try {
      ByteArrayInputStream in = new ByteArrayInputStream("foobar".getBytes());
      for(int i=0; i < 1_000_000; i++) {
        try {
          JarOutputStream out = new JarOutputStream(new ByteArrayOutputStream());
          Pack200.Unpacker unpacker = Pack200.newUnpacker();
          unpacker.unpack(in, out);
        } catch (IOException e) {
        }
      }
    } catch (OutOfMemoryError e) {
      System.out.println(e);
      throw e;
    }
  }
}

The problem can be worked around by disabling the native Pack200 implementation with -Dcom.sun.java.util.jar.pack.disable.native=true but the default setting is -Dcom.sun.java.util.jar.pack.disable.native=false.

Notice that this bug can not be fixed in HEAD because the Pack200 functionality has been removed in JDK 14 (https://openjdk.org/jeps/367). I therefore propose to fix this in jdk11u-dev first and then downport the fix to jdk8u-dev as well.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8315135: Memory leak in the native implementation of Pack200.Unpacker.unpack() (Bug - P3)

Reviewers

Christoph Langer (@RealCLanger - Reviewer)
Thomas Stuefe (@tstuefe - Reviewer)

Contributors

Yakov Shafranovich <yakovsh@amazon.com>

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk11u-dev.git pull/2100/head:pull/2100
$ git checkout pull/2100

Update a local copy of the PR:
$ git checkout pull/2100
$ git pull https://git.openjdk.org/jdk11u-dev.git pull/2100/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 2100

View PR using the GUI difftool:
$ git pr show -t 2100

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk11u-dev/pull/2100.diff

Webrev

Link to Webrev Comment

…r.unpack()

bridgekeeper · 2023-08-28T19:17:56Z

👋 Welcome back simonis! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

simonis · 2023-08-28T19:20:29Z

/contributor add Yakov Shafranovich yakovsh@amazon.com

openjdk · 2023-08-28T19:21:19Z

@simonis
Contributor Yakov Shafranovich <yakovsh@amazon.com> successfully added.

mlbridge · 2023-08-28T19:25:57Z

Webrevs

RealCLanger

Looks good overall.

The test doesn't use the formatting conventions of OpenJDK Java code, though. Indentation should be 4 spaces and also a space between if and the following ( is missing. Please clean this up before integrating.

simonis · 2023-08-29T14:04:47Z

Looks good overall.

The test doesn't use the formatting conventions of OpenJDK Java code, though. Indentation should be 4 spaces and also a space between if and the following ( is missing. Please clean this up before integrating.

Thanks for looking into this @RealCLanger. I've updated the indentation as requested.

src/java.base/share/classes/com/sun/java/util/jar/pack/UnpackerImpl.java

…if NativeUnpack::finish() is called more than once

tstuefe

Looks good, Volker. I agree, passing false as value to noCreate seems odd. But a default parameter as negation in itself is odd.

openjdk · 2023-08-29T19:44:58Z

@simonis This change now passes all automated pre-integration checks.

After integration, the commit message for the final commit will be:

8315135: Memory leak in the native implementation of Pack200.Unpacker.unpack()

Co-authored-by: Yakov Shafranovich <yakovsh@amazon.com>
Reviewed-by: clanger, stuefe

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 7 new commits pushed to the master branch:

b984b0a: 8293107: GHA: Bump to Ubuntu 22.04
c86ba43: 8312972: Bump update version for OpenJDK: jdk-11.0.22
39986e0: 8314086: [11u] A typo in the fix for JDK-8312462 is causing test failure in ChildAlwaysOnTopTest.java
749ecd5: Merge
fffaff3: 8313765: Invalid CEN header (invalid zip64 extra data field size)
1b3c516: 8314678: Bump update version for OpenJDK: jdk-11.0.20.1
89a88f0: 8276651: java/lang/ProcessHandle tests fail with "RuntimeException: Input/output error" in java.lang.ProcessHandleImpl$Info.info0

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

simonis · 2023-08-30T11:22:04Z

/backport jdk8u-dev

openjdk · 2023-08-30T11:22:57Z

@simonis Backport for repo openjdk/jdk8u-dev on branch master was successfully enabled and will be performed once this pull request has been integrated. Further instructions will be provided at that time.

simonis · 2023-08-30T15:16:17Z

/integrate

openjdk · 2023-08-30T15:16:54Z

Going to push as commit b77c161.
Since your change was applied there have been 7 commits pushed to the master branch:

b984b0a: 8293107: GHA: Bump to Ubuntu 22.04
c86ba43: 8312972: Bump update version for OpenJDK: jdk-11.0.22
39986e0: 8314086: [11u] A typo in the fix for JDK-8312462 is causing test failure in ChildAlwaysOnTopTest.java
749ecd5: Merge
fffaff3: 8313765: Invalid CEN header (invalid zip64 extra data field size)
1b3c516: 8314678: Bump update version for OpenJDK: jdk-11.0.20.1
89a88f0: 8276651: java/lang/ProcessHandle tests fail with "RuntimeException: Input/output error" in java.lang.ProcessHandleImpl$Info.info0

Your commit was automatically rebased without conflicts.

openjdk · 2023-08-30T15:16:57Z

Creating backport for repo openjdk/jdk8u-dev on branch master

/backport openjdk/jdk8u-dev master

openjdk · 2023-08-30T15:17:07Z

@simonis Pushed as commit b77c161.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

openjdk · 2023-08-30T15:18:19Z

@openjdk[bot] @simonis Could not automatically backport b77c161e to openjdk/jdk8u-dev due to conflicts in the following files:

jdk/test/tools/pack200/UnpackMalformed.java
src/jdk.pack/share/native/libunpack/jni.cpp

Please fetch the appropriate branch/commit and manually resolve these conflicts by using the following commands in your personal fork of openjdk/jdk8u-dev. Note: these commands are just some suggestions and you can use other equivalent commands you know.

# Fetch the up-to-date version of the target branch
$ git fetch --no-tags https://git.openjdk.org/jdk8u-dev.git master:master

# Check out the target branch and create your own branch to backport
$ git checkout master
$ git checkout -b simonis-backport-b77c161e

# Fetch the commit you want to backport
$ git fetch --no-tags https://git.openjdk.org/jdk11u-dev.git b77c161e7509aa3b09ebf3e6b2b1490c0667bbdc

# Backport the commit
$ git cherry-pick --no-commit b77c161e7509aa3b09ebf3e6b2b1490c0667bbdc
# Resolve conflicts now

# Commit the files you have modified
$ git add files/with/resolved/conflicts
$ git commit -m 'Backport b77c161e7509aa3b09ebf3e6b2b1490c0667bbdc'

Once you have resolved the conflicts as explained above continue with creating a pull request towards the openjdk/jdk8u-dev with the title Backport b77c161e7509aa3b09ebf3e6b2b1490c0667bbdc.

simonis · 2023-08-30T15:54:05Z

/backport openjdk/jdk11u

openjdk · 2023-08-30T15:55:45Z

@simonis the backport was successfully created on the branch simonis-backport-b77c161e in my personal fork of openjdk/jdk11u. To create a pull request with this backport targeting openjdk/jdk11u:master, just click the following link:

➡️ Create pull request

The title of the pull request is automatically filled in correctly and below you find a suggestion for the pull request body:

Hi all,

This pull request contains a backport of commit b77c161e from the openjdk/jdk11u-dev repository.

The commit being backported was authored by Volker Simonis on 30 Aug 2023 and was reviewed by Christoph Langer and Thomas Stuefe.

Thanks!

If you need to update the source branch of the pull then run the following commands in a local clone of your personal fork of openjdk/jdk11u:

$ git fetch https://github.com/openjdk-bots/jdk11u.git simonis-backport-b77c161e:simonis-backport-b77c161e
$ git checkout simonis-backport-b77c161e
# make changes
$ git add paths/to/changed/files
$ git commit --message 'Describe additional changes made'
$ git push https://github.com/openjdk-bots/jdk11u.git simonis-backport-b77c161e

8315135: Memory leak in the native implementation of Pack200O.Unpacke…

5213884

…r.unpack()

openjdk bot added the rfr Pull request is ready for review label Aug 28, 2023

RealCLanger reviewed Aug 29, 2023

View reviewed changes

Fixed indentation in test

d27b82a

tstuefe reviewed Aug 29, 2023

View reviewed changes

src/java.base/share/classes/com/sun/java/util/jar/pack/UnpackerImpl.java Show resolved Hide resolved

Make sure we don't unnecessarily create and free the native unpacker …

1cf8f35

…if NativeUnpack::finish() is called more than once

tstuefe approved these changes Aug 29, 2023

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Aug 29, 2023

RealCLanger approved these changes Aug 30, 2023

View reviewed changes

openjdk bot added the backport=openjdk/jdk8u-dev:master label Aug 30, 2023

openjdk bot added integrated Pull request has been integrated and removed backport=openjdk/jdk8u-dev:master labels Aug 30, 2023

openjdk bot closed this Aug 30, 2023

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Aug 30, 2023

8315135: Memory leak in the native implementation of Pack200.Unpacker.unpack() #2100

8315135: Memory leak in the native implementation of Pack200.Unpacker.unpack() #2100

Uh oh!

Conversation

simonis commented Aug 28, 2023 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Contributors

Reviewing

Webrev

Uh oh!

bridgekeeper bot commented Aug 28, 2023

Uh oh!

simonis commented Aug 28, 2023

Uh oh!

openjdk bot commented Aug 28, 2023

Uh oh!

mlbridge bot commented Aug 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

RealCLanger left a comment

Choose a reason for hiding this comment

Uh oh!

simonis commented Aug 29, 2023

Uh oh!

Uh oh!

tstuefe left a comment

Choose a reason for hiding this comment

Uh oh!

openjdk bot commented Aug 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

simonis commented Aug 30, 2023

Uh oh!

openjdk bot commented Aug 30, 2023

Uh oh!

simonis commented Aug 30, 2023

Uh oh!

openjdk bot commented Aug 30, 2023

Uh oh!

openjdk bot commented Aug 30, 2023

Uh oh!

openjdk bot commented Aug 30, 2023

Uh oh!

openjdk bot commented Aug 30, 2023

Uh oh!

simonis commented Aug 30, 2023

Uh oh!

openjdk bot commented Aug 30, 2023

Uh oh!

Uh oh!

simonis commented Aug 28, 2023 •

edited by openjdk bot

Loading

mlbridge bot commented Aug 28, 2023 •

edited

Loading

openjdk bot commented Aug 29, 2023 •

edited

Loading