Skip to content

Commit

Permalink
[feature] Include indexed_bzip2 classes and CLI method in rapidgzip m…
Browse files Browse the repository at this point in the history
…odule

rapidgzip was split off from indexed_bzip2 because it adds many
dependencies and makes the build process more error-prone.
The other way around, indexed_bzip2 is fully self-contained, relatively
small because of fewer precomputed lookup tables.

Ergo, there is basically no downside to including it with rapidgzip
and has the upside that, in the future, ratarmount might only have to
depend on rapidgzip when/if it gets zstd or even lz4 support.

This adds ~15% overhead to the compressed and uncompressed precompiled
wheels and binaries:

zipinfo on rapidgzip 0.10.3 wheel:

    35230512   rapidgzip.cpython-310-x86_64-linux-gnu.so
    8 files, 35266211 bytes uncompressed, 7523567 bytes compressed:  78.7%

zipinfo on rapidgzip wheel with indexed_bzip2:

    39876504   rapidgzip.cpython-310-x86_64-linux-gnu.so
    8 files, 39915164 bytes uncompressed, 8936562 bytes compressed:  77.6%

zipinfo on indexed_bzip2 1.5.0:

    12061016   indexed_bzip2.cpython-310-x86_64-linux-gnu.so
    8 files, 12088983 bytes uncompressed, 3795195 bytes compressed:  68.6%

Notably, this merge saves ~15% when comparing it to having both
installed, which is helpful for the AppImage.

The versioning of indexed_bzip2 is not easily accessible via the
rapidgzip module, it can be thought of as assuming the one by rapidgzip.

The rapidgzip.open method does not (yet) recognize bzip2 files.

This is one step towards the unification mentioned in:
Rogdham/python-xz#3
  • Loading branch information
mxmlnkn committed Nov 25, 2023
1 parent 1e73692 commit bbf94dd
Show file tree
Hide file tree
Showing 9 changed files with 393 additions and 57 deletions.
3 changes: 2 additions & 1 deletion python/indexed_bzip2/MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ include CHANGELOG.md
include indexed_bzip2.pyx
recursive-include indexed_bzip2 *.hpp
recursive-include core *.hpp
include tools/CLIHelper.hpp
include tools/ibzip2.cpp
include tools/licenses.cpp
include tools/licenses.hpp
include external/cxxopts/LICENSE
include external/cxxopts/include/cxxopts.hpp
5 changes: 4 additions & 1 deletion python/rapidgzip/MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,11 @@ include rapidgzip.svg
include CHANGELOG.md
include rapidgzip.pyx
recursive-include core *.hpp
recursive-include indexed_bzip2 *.hpp
recursive-include rapidgzip *.hpp
include tools/licenses.cpp
include tools/CLIHelper.hpp
include tools/ibzip2.cpp
include tools/licenses.hpp
include tools/rapidgzip.cpp
include external/cxxopts/LICENSE
include external/cxxopts/include/cxxopts.hpp
Expand Down
1 change: 1 addition & 0 deletions python/rapidgzip/indexed_bzip2
Loading

0 comments on commit bbf94dd

Please sign in to comment.