Create default entity decoder for InputStream

[balatbn]

If we want to retrieved Inputstream directly from HTTP client (we want to decompress stream with BZIP), currently we have to write our own EntityDecoder.

It would be nice to have this support by default.

[balatbn]

We have this implementation for input stream. Is this the correct way to get Inputstream from response?

  implicit val inputStreamDecoder: EntityDecoder[InputStream] = EntityDecoder.decodeBy(MediaRange.`*/*`) { msg =>
    val inputStream = new PipedInputStream()
    DecodeResult.success(msg.body.to(io.chunkW(new PipedOutputStream(inputStream))).run).map(_ => inputStream)
  }

If this is correct and can be added to main branch, please let me know. I'll create a Pull request.

[aryairani]

How would you feel about decoding the Process[Task, ByteVector] body directly instead of going through InputStream? Or about writing a middleware for bzipped content, similar to org.http4s.server.middleware.GZip?

[aryairani]

Oops, that GZip middleware is for output, not input; sorry.

[balatbn]

Bzip2 is an external dependency in Java. So, we didn't want to implement it as a middleware in the client.

[rossabaker]

scalaz.stream.io.toInputStream(response.body) gets you an InputStream without the deadlock issues that tend to come with piped streams.

Note that client methods like fetch dispose of the HTTP connection as soon as the Task returns, such that the body can't be read. You would want to use something like client.toHttpService(req).map(resp => scalaz.stream.io.toInputStream(resp.body)). Be sure to close the InputStream, or you will leak a connection.

[balatbn]

Our code didn't work(probably because of some deadlock issue). We solved it with the same technique given in the above response.

[rossabaker]

The PipedOutputStream and PipedInputStream need to operate on different threads, or else one can starve the other. The joys of blocking I/O. :)

[mkeshav]

Is there any easy way to get the body of the request as String. Due to reasons not in my control, I would like to parse the json body into Map[String, Any] which circe does not allow (I understand why). If I can get the body as a string then I can use something like Jackson which uses reflection to parse.

Really appreciate some guidance

[rossabaker]

req.as[String] gets you an F[String] (or Task[String] before 0.18). You can then (flat)Map that with your Jackson parser to get JSON.

The toInputStream advice above is still valid, if Jackson prefers that.

[mkeshav]

Thank you so much