Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZOOKEEPER-4246: Resource leaks in org.apache.zookeeper.server.persistence.SnapStream#getInputStream and #getOutputStream #1638

Closed
wants to merge 4 commits into from

Conversation

kelloggm
Copy link
Contributor

Bug report is here: https://issues.apache.org/jira/browse/ZOOKEEPER-4246

This fix is simple: it just closes the possibly-leaked streams and re-throws the exception. We can't use a try-with-resources or a finally block here because in the happy case the resulting streams need to be returned open. I checked each of the other constructor calls in these methods, and none of the others can throw an exception as far as I can tell.

@maoling
Copy link
Member

maoling commented Mar 28, 2021

@kelloggm Looks reasonable.

I noticed the first because of the use of the error-prone GZIPOutputStream, and the other two after looking at the surrounding code.

Could you please show us how the error-prone complain about this?

@kelloggm
Copy link
Contributor Author

@maoling Sorry for the confusion. You asked:

Could you please show us how the error-prone complain about this?

in reference to my comment:

I noticed the first because of the use of the error-prone GZIPOutputStream, and the other two after looking at the surrounding code.

In this comment, I'm using "error-prone" as an adjective - i.e. I'm saying that I think GZIPOutputStream is easy to use incorrectly. I did not intend to refer to error-prone, Google's code analysis tool. From your comment, it sounds like you understood me to mean that tool - sorry for the confusion! As far as I'm aware, that tool cannot find this bug, but I haven't run it on this code.

@maoling
Copy link
Member

maoling commented Apr 4, 2021

@kelloggm Sorry for my misunderstanding for code analysis tool: error-prone. Do you observe this resource leaks in the production or just your code analysis?

Copy link
Contributor

@anmolnar anmolnar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch.

@@ -104,10 +104,20 @@ public static CheckedInputStream getInputStream(File file) throws IOException {
InputStream is;
switch (getStreamMode(file.getName())) {
case GZIP:
is = new GZIPInputStream(fis);
try {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put the entire block in a try-catch:

try {
        switch (getStreamMode(file.getName())) {
        case GZIP:
            is = new GZIPInputStream(fis);
            break;
        case SNAPPY:
            is = new SnappyInputStream(fis);
            break;
        case CHECKED:
        default:
            is = new BufferedInputStream(fis);
        }
        return new CheckedInputStream(is, new Adler32());
} catch (IOException e) {
    fis.close();
    throw e;
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made this change. Would you all also prefer one try around the entire switch in the other method, where only one of the cases can throw? It would make the code look more symmetrical, but it's otherwise unnecessary.

@kelloggm
Copy link
Contributor Author

kelloggm commented Apr 6, 2021

@maoling no worries, sorry again for the confusion

@kelloggm Sorry for my misunderstanding for code analysis tool: error-prone. Do you observe this resource leaks in the production or just your code analysis?

I didn't encounter these in production, to be clear. I was already familiar with the JDK bug I linked in my bug report, and got suspicious when I saw GZIP was supported here so I looked at the code and noticed these.

@kelloggm
Copy link
Contributor Author

@anmolnar is there anything else you need from me on this PR? It's been a few weeks and I don't want it to be forgotten.

Copy link
Contributor

@ztzg ztzg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I've made this change. Would you all also prefer one try around the entire switch in the other method, where only one of the cases can throw? It would make the code look more symmetrical, but it's otherwise unnecessary.

I, for one, have a slight preference for the "symmetrical" version, but let's not bikeshed.

@anmolnar, @maoling: do you approve?

@maoling
Copy link
Member

maoling commented May 9, 2021

LGTM.
If no other concerns, I'll merge it at this weekend(05-15). Cc @ztzg @anmolnar @kelloggm

@maoling maoling closed this May 15, 2021
@maoling maoling reopened this May 15, 2021
Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@asfgit asfgit closed this in 766e173 May 15, 2021
@maoling
Copy link
Member

maoling commented May 15, 2021

@kelloggm
Thanks for your contribution.

anuragmadnawat1 pushed a commit to anuragmadnawat1/zookeeper that referenced this pull request Nov 2, 2022
…ence.SnapStream#getInputStream and #getOutputStream

Bug report is here: https://issues.apache.org/jira/browse/ZOOKEEPER-4246

This fix is simple: it just closes the possibly-leaked streams and re-throws the exception. We can't use a try-with-resources or a `finally` block here because in the happy case the resulting streams need to be returned open. I checked each of the other constructor calls in these methods, and none of the others can throw an exception as far as I can tell.

Author: Martin Kellogg <kelloggm@cs.washington.edu>

Reviewers: Andor Molnar <anmolnar@apache.org>, Enrico Olivelli <eolivelli@apache.org>, maoling <maoling@apache.org>

Closes apache#1638 from kelloggm/ZOOKEEPER-4246 and squashes the following commits:

fa1c0f0 [Martin Kellogg] surround whole switch with one try block instead of two inside the switch
f023851 [Martin Kellogg] remove all the tabs for real
04c4b2f [Martin Kellogg] fix accidental tab
e896efa [Martin Kellogg] ZOOKEEPER-4246: Resource leaks in org.apache.zookeeper.server.persistence.SnapStream#getInputStream and #getOutputStream
anuragmadnawat1 added a commit to anuragmadnawat1/zookeeper that referenced this pull request Nov 2, 2022
…ence.SnapStream#getInputStream and #getOutputStream (#144)

Bug report is here: https://issues.apache.org/jira/browse/ZOOKEEPER-4246

This fix is simple: it just closes the possibly-leaked streams and re-throws the exception. We can't use a try-with-resources or a `finally` block here because in the happy case the resulting streams need to be returned open. I checked each of the other constructor calls in these methods, and none of the others can throw an exception as far as I can tell.

Author: Martin Kellogg <kelloggm@cs.washington.edu>

Reviewers: Andor Molnar <anmolnar@apache.org>, Enrico Olivelli <eolivelli@apache.org>, maoling <maoling@apache.org>

Closes apache#1638 from kelloggm/ZOOKEEPER-4246 and squashes the following commits:

fa1c0f0 [Martin Kellogg] surround whole switch with one try block instead of two inside the switch
f023851 [Martin Kellogg] remove all the tabs for real
04c4b2f [Martin Kellogg] fix accidental tab
e896efa [Martin Kellogg] ZOOKEEPER-4246: Resource leaks in org.apache.zookeeper.server.persistence.SnapStream#getInputStream and #getOutputStream

Co-authored-by: Martin Kellogg <kelloggm@cs.washington.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants