Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Partial/incremental decompression of clusters #411

Closed
wants to merge 13 commits into from
Closed

Partial/incremental decompression of clusters #411

wants to merge 13 commits into from

Commits on Sep 3, 2020

  1. Introduced zim::IDataStream

    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    3dc7700 View commit details
    Browse the repository at this point in the history

  2. Enter BufDataStream

    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    eed7ade View commit details
    Browse the repository at this point in the history

  3. zim::ReaderDataStreamWrapper

    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    ff249c7 View commit details
    Browse the repository at this point in the history

  4. Enter embryonic CompressedCluster

    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    d344009 View commit details
    Browse the repository at this point in the history

  5. Got rid of read_size() in cluster.cpp

    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    9c46d28 View commit details
    Browse the repository at this point in the history

  6. CompressedCluster::blobs_ 

    CompressedCluster got its own collection of blobs. The latter is filled
by simply breaking down the fully uncompressed data with the help of
ReaderDataStreamWrapper.
    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    56505a5 View commit details
    Browse the repository at this point in the history

  7. CompressedCluster is fully read via IDataStream 

    However the IDataStream object used at this point is still a
ReaderDataStreamWrapper reading from the fully uncompressed data.
    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    eaf0aed View commit details
    Browse the repository at this point in the history

  8. Enter NonCompressedCluster 

    Cluster is now an abstract base class for NonCompressedCluster and
CompressedCluster.
    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    a9f2fde View commit details
    Browse the repository at this point in the history

  9. Enter DecodedDataStream

    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    34c1326 View commit details
    Browse the repository at this point in the history

  10. Streaming decompression of clusters 

    From now on, compressed clusters are decompressed using the streaming
decompression (DecodedDataStream) API. However, a cluster is fully
decompressed (i.e. all blobs are extracted and stored inside the
cluster).
    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    757cd27 View commit details
    Browse the repository at this point in the history

  11. Lazy/on-demand decompression of clusters

    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    ac857be View commit details
    Browse the repository at this point in the history

  12. Removed unnecessary indentation

    @veloman-yunkan
    veloman-yunkan committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    1c2a7b9 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2020

  1. DecodedDataStreamTest.largeCompressedData 

    Added a unit-test covering the scenario where the loop body in
`DecodedDataStream::readImpl()` has to be executed more than once.
    @veloman-yunkan
    veloman-yunkan committed Sep 5, 2020
    Configuration menu
    Copy the full SHA
    1ac51bd View commit details
    Browse the repository at this point in the history