Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Compressed streams #76

Closed
Mortal opened this Issue · 4 comments

2 participants

@Mortal
Collaborator

Some day we should do a prototype of stream compression. Stream blocks to be written can be sent to a compression thread, which compresses the data and sends it to a writer thread.
Hopefully, the compressor can vary its effort according to available I/O bandwidth and CPU cycles.

@Mortal Mortal was assigned
@thomasmoelhave

The Google Snappy library might be interesting here: https://code.google.com/p/snappy/

@Mortal
Collaborator

This would be cool to test out. I added support in our CMakeLists in the snappy branch.

What should we try it out on?

I think it would be cool to adapt it to the serialization streams, since they already do not support seeking.

@Mortal
Collaborator

In the snappy branch I have now enabled stream compression at the stream_accessor level, meaning all streams are compressed with no change in API. However, seeking is not supported except to the beginning of the stream, so I will have to change this solution to something more permanent. In particular, stream reversing and TPIE stacks do not work, so I cannot really test this on a real world application yet.

@thomasmoelhave

@Mortal I guess we can close this issue now?

@Mortal Mortal closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.