Skip to content
This repository

Cannot decompress an encoded gzip stream #52

Open
crockpotveggies opened this Issue · 3 comments

3 participants

crockpotveggies Seth Tisue Nathan Hamblen
crockpotveggies

Getting an encoded stream is rather common from big data APIs. Having a default handler similar to the basic as.stream.Lines would be really handy for this kind of use case.

Here's an example implementation to connect to a gzip encoded stream:

Http(
      host("stream.data.com")
      .secure
      .addHeader("Accept-Encoding", "gzip")
       /  "streams" / "track" / "tweets.json"
      > as.stream.EncodedLines(println) )()```
Seth Tisue

I ended up resorting to bypassing dispatch-json4s-native and Future entirely and doing:

  val stream = Http(req OK as.Response(_.getResponseBodyAsStream)).apply
  val result =
    JsonParser.parse(
      new java.io.InputStreamReader(
        new java.util.zip.GZIPInputStream(stream)))
Nathan Hamblen
Owner
n8han commented

It looks like the right place to implement this is as a ResponseFilter, if I'm reading it correctly:
https://github.com/AsyncHttpClient/async-http-client/blob/f2be57b3c2a85f021ca0b5a1b1bee69ac94d89aa/src/main/java/com/ning/http/client/filter/ResponseFilter.java

Otherwise it needs to be implemented for as.String and stream.Strings.

But it should be a pretty straightforward enhancement if anybody wants to jump on it. I would just use a GZIPInputStream with a ByteArrayInputStream for each byte array.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.