Safely close compressor stream after failure

[fredrikekre]

This is perhaps not a bug report, but looking for input on how to improve.

HTTP.jl uses the following pattern for decompressing request bodies (see https://github.com/JuliaWeb/HTTP.jl/blob/master/src/clientlayers/StreamRequest.jl#L122-L140):

using SimpleBufferStream, CodecZlib

# Create a dummy file to read from, this is the compressed HTTP body
file = "no-gzip"
write(file, "this is not gzipped")
io = open(file)

# Intermediate bufferstream so that we can read/write in chunks
buf = SimpleBufferStream.BufferStream()
decompressor = GzipDecompressorStream(buf)

# Write body into the decompressor, which decompresses into buf
tsk = @async begin
    try
        write(decompressor, io) # Doesn't throw
    finally
        close(decompressor) # This throws *before* closing buf
    end
end

write(stderr, buf) # Deadlocks because buf is never closed
wait(tsk)

The problem here is that there is a deadlock if the content can't be decompressed, e.g. if the content isn't gzipped to begin with. What happens is that write(decompressor, io) works, but then changemode! inside of close(decompressor) errors. Specifically,

TranscodingStreams.jl/src/stream.jl

Line 182 in 35edb68

stopped = stream.state.mode == :stop

is false, meaning that https://github.com/JuliaIO/TranscodingStreams.jl/blob/master/src/stream.jl#L186-L188 is never executed.

Ultimately this means that tsk fails, but since buf is still open, write(stderr, buf) deadlocks.

I found that using something like

tsk = @async begin
    try
        write(decompressor, io)
    finally
        try
            close(decompressor)
        catch
            close(buf) # explicity close buf in case of errors in close(decompressor)
            rethrow()
        end
    end
end

mitigates the issue but doesn't really seem like a satisfactory solution.