Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flatpak-build-bundle: Add --oci-layer-compress=zlib #5540

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

owtaylor
Copy link
Contributor

This PR adds an option to build OCI bundles with zstd compressed layers, speeding up compression by several times, and resulting in about a 20% smaller result (timings are in a comment in the patch).

gzip is kept as the default for maximum compatibility:

Notes

  • This generally will work pretty much everywhere now that OCI support is quite widespread. I made it an option rather than just switching it over because I'd like to see it tested in the Fedora and (in particular) RHEL Flatpak pipelines before making the zstd the default. But I could see making zstd the default in the future.
  • Flatpak itself supports zstd layers somewhat accidentally - because it is autodetecting tar compression via libarchive.
  • I didn't add an option for compression level, because zstd with level=9 is better for both speed and size than what we have already with zlib. Using a lower level would speed up things for devel workflows, but that would matter only for the largest images.
  • For simplicity, I didn't try to enable multithreaded compression with zstd, but that could potentially make things even faster.
  • The ZSTD_CStream data type I'm using is somewhat obsolescent, though not deprecated - according to the docs, ZSTD_CStream was unified with ZSTD_CCtx in 1.3.0, released in 2017. Might make sense to upgrade the Flatpak min zstd version from the current 0.8.1 - hard to say whether this would actually work with 0.8.1.

Ecosystem support

distribution/distribution: no explicit support, but works
quay.io: sinc 2021
Amazon ECR: supported
pulp_container: since 2022
flatpak: since first-OCI supporting version
tardiff: since first version

Add an option to build OCI bundles with zstd compressed layers.

gzip is kept as the default for maximum compatibility:

Ecosystem support:

 distribution/distribution: no explicit support, but works
 quay.io: sinc 2021
 Amazon ECR: supported
 pulp_container: since 2022
 flatpak: since first-OCI supporting version
 tardiff: since first version
tests/test-oci-registry.sh Fixed Show fixed Hide fixed
Copy link
Collaborator

@smcv smcv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add --oci-layer-compress=zlib

I think you mean zstd.

Copy link
Collaborator

@smcv smcv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a full review, I'd prefer someone who is more involved in the package-manager/OCI side of Flatpak to review this properly.


<listitem><para>
Choose how to compress the layers in OCI images. gzip (the default)
has is universally supported, but zstd compresses faster and results in
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
has is universally supported, but zstd compresses faster and results in
is universally supported, but zstd compresses faster and results in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants