Skip to content

Conversation

@rr-it
Copy link
Contributor

@rr-it rr-it commented Mar 29, 2025

Description:

This uses ZLIB compression-level 9 instead of default 6 for archive-blob-tables. Thereby the needed storage size is reduced by 5%.

This implements #23186

Review

This uses ZLIB compression-level 9 instead of default 6 for archive-blob-tables. Thereby the needed storage size is reduced by 5%.
@rr-it rr-it marked this pull request as draft March 29, 2025 17:52
@rr-it rr-it marked this pull request as ready for review March 30, 2025 18:12
@sgiehl sgiehl added the Needs Review PRs that need a code review label Apr 7, 2025
@sgiehl sgiehl requested a review from a team April 7, 2025 12:09
@github-actions
Copy link
Contributor

This issue is in "needs review" but there has been no activity for 7 days. ping @matomo-org/core-reviewers

@github-actions github-actions bot added the Stale The label used by the Close Stale Issues action label Apr 15, 2025
@mneudert mneudert added Do not close PRs with this label won't be marked as stale by the Close Stale Issues action and removed Stale The label used by the Close Stale Issues action labels Apr 24, 2025
@mneudert
Copy link
Member

Hi @rr-it,

thank you for the patch, and sorry for the delay on this one.

Do you have, by chance, any in-depth resources on the CPU time it takes for the compression? Any good benchmark of different data sizes that also shows CPU times?

I only did a really quick check on a local dataset to see how that changes, and it was somewhere in the range of a threefold to fourfold increase in the compression time. For CLI archiving, it looked like an increase of 0.5% in total runtime. But when requesting a custom date range in the browser (that often has to compress a new range archive), the times were varying on a level of 2% to 3% in increased request time.

Those checks were far from scientific, or even being a well-taken average, just one-off requests to get some first numbers.

If there is a way to get a good estimate on the expected increase in compression time, it would help tremendously to decide if the compression level should change to the value you suggest, or maybe get a configuration if someone wants to do the CPU/storage tradeoff in the other direction.

@michalkleiner
Copy link
Contributor

Replaced by #23257, thanks for your contribution regardless!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Do not close PRs with this label won't be marked as stale by the Close Stale Issues action Needs Review PRs that need a code review

Development

Successfully merging this pull request may close these issues.

Reduce matomo_archive_blob DB table size by 5% using ZLIB compression-level 9 instead of default 6 for BLOB

4 participants