Metadata Compression

[apavlenko]

Requirements:

1. ability to compresses the entire metadata schema on save into media file and de-compress back on load
2. ability to compress and de-compress exported XML/JSON metadata representation (including incremental mode)
3. out-of-box compression algo + API (and a sample) of user-provided algo using open-source compression lib(s) (bz2, gz, lzma)
4. test(s) and sample(s)

[savuor]

A prototype implemented: transparent encoding/decoding from/to base64 inside XMPDataSource.
See rv/compression_prototype branch for details.

[savuor]

Design proposal

1. Each compression method should be implemented as a class implementing ICompressor interface:

   • void compress(const string& in, string& out);
   • void decompress(const string& in, string& out);
   • string getId();

   If user wants to use his own compression method, the function vmf::registerCompressor(ICompressor*) should be called before that.

2. Exporting metadata to separate file is performed now using IReader and IWriter interfaces.
   I propose to create CompressionReader class derived from IReader interface.

   It should use the following user-provided instances of interfaces:

   • ICompressor to decompress data
   • IReader to parse data

   The same for CompressionWriter class.

3. To write metadata to video file a user now calls the method vmf::MetadataStream::save() or saveTo(string).
   I propose to add the optional argument ICompressor* to the methods.

   Implementation of the in-video metadata compression should be the following:

   • create methods XMPDataSource::loadXMPStructs() and XMPDataSource::saveXMPStructs() as an intermediate layer
   • reimplement them in derived class CompressedXMPDataSource taking ICompressor* as the constructor argument
   • this class should perform encoding to and decoding from base64

[apavlenko]

I suggest

class ICompressor
{
public:
    void compress(const std::string& in, vmf_rawbuffer& out);
    void decompress(const vmf_rawbuffer& in, std::string& out);
    const std::string& getID();
};

since VMF uses vmf_rawbuffer type for arbitrary binary data (BTW, vmf_rawbuffer implementation can be improved).

Also vmf::registerCompressor(std::shared_ptr<ICompressor> compressor) is much safer than use of naked pointer.

I'm not sure we really need CompressedXMPDataSource as an inheritor of XMPDataSource for compression support - please consider possibility of adding compression-related code directly into XMPDataSource. (Same for IReader and IWriter.)

[apavlenko]

#10 is merged