add ZipToStreamFlow #170
add ZipToStreamFlow #170
Conversation
Adds a facility for predictable-memory streaming zipfile creation, as in the instance of sending multiple files to a browser over an HTTP connection (even when they originate from a remote service such as S3). Incoming streams must be able to be created synchronously, but are delayed such that only a single InputStream is open at any given time. Original gist at https://gist.github.com/kirked/03c7f111de0e9a1f74377bf95d3f0f60 (original license MIT). The author is happy to contribute this and derivative works for inclusion in the akka-stream-contrib library, and hereby assigns full control over this code, including copyright protection and licensing worldwide, in perpetuity, to Lightbend, Inc.
| } | ||
|
|
||
| object ZipToStreamFlow { | ||
| final case class ZipSource(filePath: String, streamGenerator: () => InputStream) |
There was a problem hiding this comment.
It might be nicer to use Akka streams in this API (even if they'd eventually need to be converted to InputStream - WDYT?
There was a problem hiding this comment.
Yes. When I wrote this, our need was for zipping multiple S3-contained assets to the browser, and the AWS API (at least that we were using) provided synchronous access to the InputStreams. There was no Alpakka, and Akka was just joining our tech stack (I still have the original Play Iteratee version as a gist).
I see this as a good start for an often-needed utility that can be greatly improved.
There was a problem hiding this comment.
I also prefer the implementation to use Akka Streams API. Convertion from Java stream to Akka Stream is always possible with StreamConverters: https://github.com/akka/akka/blob/master/akka-stream/src/main/scala/akka/stream/scaladsl/StreamConverters.scala#L25
|
This is useful, but as discussed, it would be better to implement this functionality using Akka Stream APIs. I am going to close this PR here, but the way forward for Zip streams would be to move the implementation of #171 to Alpakka File connector and then enrich that with some examples of Java Streams and using @kirked feel free to move the #171 to Alpakka and work together with @michalbogacz on this. |
Adds a facility for predictable-memory streaming zipfile creation, as in the instance of sending multiple files to a browser over an HTTP connection (even when they originate from a remote service such as S3).
Incoming InputStreams must be able to be created synchronously, but are delayed such that only a single InputStream is open at any given time.
Original gist at https://gist.github.com/kirked/03c7f111de0e9a1f74377bf95d3f0f60 (original license MIT).
The author is happy to contribute this and derivative works for inclusion in the akka-stream-contrib library, and hereby assigns full control over this code, including copyright protection and licensing worldwide, in perpetuity, to Lightbend, Inc.
Purpose
References
References #xxxx
Changes
Background Context