Skip to content

uorc v0.5.0

Choose a tag to compare

@github-actions github-actions released this 13 Nov 15:48
· 19 commits to main since this release
5aeeb04

uorc v0.5.0

Client: https://uor-foundation.github.io/compression/

Closed claims

Codec point 3cd4e997c1460e85 (export 6b5653bf17c4f34e, plan portfolio, 1 lane). Sizes and decode steps are exact: each was decided by running this codec over the κ-named corpus and is recorded with its decision in the release's frontier.toml. A step is one model input evaluated for one coded bit.

corpus corpus κ input bytes archive bytes decode steps steps per byte status decided by
uorc-v0.3.0 9b781bfc9840 1442315 575439 198707536 137.77 proved factory
canterbury c97b3c539721 2810784 351887 474920872 168.96 proved factory
silesia 45094a500116 211938580 40987562 40067382768 189.05 proved factory

UC-PERF-01 target

On every public corpus, the archive is no larger than each pinned comparator's. A refuted row is a comparator this codec does not yet reach.

corpus comparator comparator bytes uorc bytes status
uorc-v0.3.0 gzip -9 650271 575439 proved
uorc-v0.3.0 bzip2 -9 606478 575439 proved
uorc-v0.3.0 xz -9e 595432 575439 proved
canterbury gzip -9 730625 351887 proved
canterbury bzip2 -9 542710 351887 proved
canterbury xz -9e 493080 351887 proved
canterbury zstd -19 --long 516216 351887 proved
canterbury brotli -q 11 490662 351887 proved
canterbury zpaq -m5 361718 351887 proved
canterbury 7z PPMd -mx9 575170 351887 proved
silesia gzip -9 67631918 40987562 proved
silesia bzip2 -9 54506769 40987562 proved
silesia xz -9e 48456004 40987562 proved
silesia zstd -19 --long 52646619 40987562 proved
silesia brotli -q 11 49564563 40987562 proved
silesia zpaq -m5 39112562 40987562 refuted
silesia 7z PPMd -mx9 47684624 40987562 proved
Per family (first claim's corpus)
family members archive bytes decode steps
tiny 2 1475 631296
small 2 2487 6441216
text 3 259617 162957848
other 1 311860 28677176

The register holds 24 claims over 3 corpora and 2 codec points; 4 concern other points (earlier releases, engine candidates) and are kept by κ, never rerun.

The model on the page

Performance rows: UC-PERF-01 (benchmark_target_unmet), UC-PERF-02 (kernel_theorem), UC-PERF-03 (kernel_theorem), UC-PERF-04 (kernel_theorem), UC-PERF-05 (kernel_theorem), UC-PERF-06 (kernel_theorem), UC-PERF-07 (kernel_theorem), UC-PERF-08 (kernel_theorem), UC-PERF-09 (kernel_theorem), UC-PERF-10 (kernel_theorem), UC-PERF-11 (build_evidence), UC-PERF-16 (build_evidence), UC-PERF-17 (build_evidence).

Claims: 24 recorded, 23 proved, 1 refuted (listed above).

Engine: tick 0, rule30 register 0x9e3779b97f4a7c15, 10 strategies in the portfolio, 70 weights.

The full register is model.json, on the Pages site and attached to this release.

uorc benchmarks

uorc 0.5.0, 0 search steps per file, linux/x64. Measurement only; no target is set.

Machine ceiling: 4 cores (AMD EPYC 7763 64-Core Processor), 15.6 GiB memory; memory copy 20202 MB/s on one core and 22474 MB/s on all 4. A decoder cannot write its output faster than the machine copies memory, so the last column is decompress throughput as a share of the all-core copy rate. uorc uses every core (8 MiB frames in parallel); the other tools run with their default thread settings.

canterbury

compressor files archive bytes ratio compress MB/s decompress MB/s decompress / copy
uorc (0 steps) 11 350645 8.016 0.07 0.39 0.0018%
gzip -9 11 730625 3.847 4.22 61.92 0.28%
bzip2 -9 11 542710 5.179 15.73 26.62 0.12%
xz -9e 11 493080 5.700 1.67 39.79 0.18%
zstd -19 --long 11 516216 5.445 1.37 57.55 0.26%
brotli -q 11 11 490662 5.729 0.56 58.66 0.26%
zpaq -m5 11 361718 7.771 0.38 0.38 0.0017%
7z PPMd -mx9 11 575170 4.887 7.26 7.35 0.033%
Per file (archive bytes)
file bytes uorc (0 steps) gzip -9 bzip2 -9 xz -9e zstd -19 --long brotli -q 11 zpaq -m5 7z PPMd -mx9
alice29.txt 152089 37158 54179 43202 48528 49212 46487 37591 38956
asyoulik.txt 125179 34559 48816 39569 44592 45138 42712 35369 38434
cp.html 24603 6351 7973 7624 7652 7716 6894 7418 7507
fields.c 11150 2402 3127 3039 3032 3017 2717 3469 3110
grammar.lsp 3721 1064 1234 1283 1292 1214 1124 2069 1326
kennedy.xls 1029744 14093 209721 130280 51868 64826 61498 16813 182238
lcet10.txt 426754 89314 144418 107706 119488 121337 113416 90013 102503
plrabn12.txt 481861 127122 194264 145577 165456 167439 163267 127568 138560
ptt5 513216 28280 52377 49759 39860 43610 40939 28561 48463
sum 38240 8943 12768 12909 9500 10978 10144 10324 12417
xargs.1 4227 1359 1748 1762 1812 1729 1464 2523 1656

silesia

compressor files archive bytes ratio compress MB/s decompress MB/s decompress / copy
uorc (0 steps) 12 40925775 5.179 0.10 0.62 0.0027%
gzip -9 12 67631918 3.134 10.77 145.02 0.65%
bzip2 -9 12 54506769 3.888 13.09 28.83 0.13%
xz -9e 12 48456004 4.374 1.94 63.93 0.28%
zstd -19 --long 12 52646619 4.026 2.24 321.88 1.4%
brotli -q 11 12 49564563 4.276 0.49 214.91 0.96%
zpaq -m5 12 39112562 5.419 0.40 0.40 0.0018%
7z PPMd -mx9 12 47684624 4.445 6.49 6.05 0.027%
Per file (archive bytes)
file bytes uorc (0 steps) gzip -9 bzip2 -9 xz -9e zstd -19 --long brotli -q 11 zpaq -m5 7z PPMd -mx9
dickens 10192446 2240481 3851815 2799520 2831212 2849320 2827777 2094743 2280117
mozilla 51220480 12587898 18994134 17914392 13376240 14978836 13872265 12041055 15911894
mr 9970564 2158855 3673937 2441280 2751892 3105810 2823136 2181310 2320767
nci 33553445 1387073 2987529 1812734 1449272 1615898 1519768 1251108 1484569
ooffice 6152192 1794476 3090434 2862526 2427224 2595308 2478855 1766550 2496131
osdb 10085684 2225316 3716337 2802792 2844556 3099934 2816278 2204741 2384175
reymont 6627202 1007520 1820826 1246230 1315592 1348541 1332158 956499 1054291
samba 21606400 3169580 5408266 4549759 3739524 3876656 3766340 3053820 3815315
sao 7251944 3979521 5327037 4940524 4425664 5000573 4586092 3899258 4783825
webster 41458703 6392659 12061616 8644714 8368672 8592754 8428575 5666831 6942883
x-ray 8474240 3653386 6037707 4051112 4491264 5129825 4682753 3669701 3850893
xml 5345280 329010 662280 441186 434892 453164 430566 326946 359764

Ratio is input bytes over archive bytes, and throughput is input MB per second, both over the files that compressor round-tripped. ✗ marks a failed round trip. zpaq and 7z archive bytes include their container headers. Tool versions:

  • gzip -9: gzip 1.12
  • bzip2 -9: bzip2, a block-sorting file compressor. Version 1.0.8, 13-Jul-2019.
  • xz -9e: xz (XZ Utils) 5.4.5
  • zstd -19 --long: *** Zstandard CLI (64-bit) v1.5.7, by Yann Collet ***
  • brotli -q 11: brotli 1.1.0
  • zpaq -m5: zpaq v7.15 journaling archiver, compiled Jan 5 2021
  • 7z PPMd -mx9: 7-Zip 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20