Suggestion: serialize option for compression

[iwelch]

Please add a compression option to serialize, so that deserialize seamlessly decompresses (presumably identified by the 8-byte header). I would suggest something simple, like lz4. In some experiments with mixed data, it reduces serialized data storage requirements by a factor of 3, and speeds up deserialization.

[ikirill]

I had the same question, in case somebody else finds this old issue, but serialize is not the right place to implement that. At the moment, there is support for drop-in compression of streams in TranscodingStreams.jl, you even get better compression than simple:

    io = open(tmp, "w")
    io = TranscodingStream(ZstdCompressor(), io)
    serialize(io, value)
    close(io)

    io = open(path)
    io = TranscodingStream(ZstdDecompressor(), io)
    value = deserialize(io)
    close(io)

It works with both files and other streams like pipes, so this is not an issue with (de)serialize any more.

[vtjnash]

Agreed, layering this with composable stream API is a better way to go