Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make SnappyOutputStream.close() idempotent; guard against writing to closed stream #108

Merged
merged 4 commits into from
May 14, 2015
Merged

Make SnappyOutputStream.close() idempotent; guard against writing to closed stream #108

merged 4 commits into from
May 14, 2015

Conversation

JoshRosen
Copy link
Contributor

This patch will fix #107, an issue where SnappyOutputStream.close() is not idempotent.

The first commit is a failing regression test which shows how a non-idempotent close() can lead to stream corruption issues.

@JoshRosen JoshRosen force-pushed the close-should-be-idempotent branch 2 times, most recently from 1285253 to 11f08de Compare May 14, 2015 07:28
@JoshRosen
Copy link
Contributor Author

Here's a failing run of the regression test: https://travis-ci.org/xerial/snappy-java/jobs/62514893

@JoshRosen
Copy link
Contributor Author

Even though this patch makes close() idempotent, I suppose there's still the potential for stream corruption if a user continues to call write() methods on a SnappyOutputStream after calling close() on it. Do you think that we should also add a !closed check in rawWrite() to guard against this case? Alternatively, I guess that we could null out the references to the buffers to be absolutely sure that we can't access them after close().

@xerial
Copy link
Owner

xerial commented May 14, 2015

Thanks for the fix.

I think checking !close has negligible overhead.
And also nulling out the buffers is also good for GC.

Please add both.

@JoshRosen JoshRosen changed the title Make SnappyOutputStream.close() idempotent Make SnappyOutputStream.close() idempotent; guard against writing to closed stream May 14, 2015
@JoshRosen
Copy link
Contributor Author

I've updated this patch to include nulling of output buffers and if(closed) checks.

xerial
xerial reviewed May 14, 2015
View reviewed changes
src/test/java/org/xerial/snappy/SnappyOutputStreamTest.java
WeakReference<byte[]> inputBuffer = new WeakReference<byte[]>(os.inputBuffer);
WeakReference<byte[]> outputBuffer = new WeakReference<byte[]>(os.inputBuffer);
os.close();
System.gc();
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal here was to test that the now weakly-referencable buffers get reclaimed by GC.

@xerial
Copy link
Owner

xerial commented May 14, 2015

Looks good! Thank you for the improvements.

xerial added a commit that referenced this pull request May 14, 2015
@xerial
Merge pull request #108 from JoshRosen/close-should-be-idempotent
5bd06f4 
Make SnappyOutputStream.close() idempotent; guard against writing to closed stream
@xerial xerial merged commit 5bd06f4 into xerial:develop May 14, 2015
@JoshRosen JoshRosen deleted the close-should-be-idempotent branch May 14, 2015 21:47
@JoshRosen JoshRosen mentioned this pull request May 15, 2015
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@JoshRosen @xerial