Cannot decompress an encoded gzip stream #52

Open
crockpotveggies opened this Issue Apr 4, 2013 · 3 comments

3 participants

@crockpotveggies

Getting an encoded stream is rather common from big data APIs. Having a default handler similar to the basic as.stream.Lines would be really handy for this kind of use case.

Here's an example implementation to connect to a gzip encoded stream:

Http(
      host("stream.data.com")
      .secure
      .addHeader("Accept-Encoding", "gzip")
       /  "streams" / "track" / "tweets.json"
      > as.stream.EncodedLines(println) )()```
@SethTisue

I ended up resorting to bypassing dispatch-json4s-native and Future entirely and doing:

  val stream = Http(req OK as.Response(_.getResponseBodyAsStream)).apply
  val result =
    JsonParser.parse(
      new java.io.InputStreamReader(
        new java.util.zip.GZIPInputStream(stream)))
@n8han

It looks like the right place to implement this is as a ResponseFilter, if I'm reading it correctly:
https://github.com/AsyncHttpClient/async-http-client/blob/f2be57b3c2a85f021ca0b5a1b1bee69ac94d89aa/src/main/java/com/ning/http/client/filter/ResponseFilter.java

Otherwise it needs to be implemented for as.String and stream.Strings.

But it should be a pretty straightforward enhancement if anybody wants to jump on it. I would just use a GZIPInputStream with a ByteArrayInputStream for each byte array.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment