Skip to content

uorc v0.6.0

Latest

Choose a tag to compare

@github-actions github-actions released this 14 Nov 11:26
· 10 commits to main since this release
e713d1c

uorc v0.6.0

Client: https://uor-foundation.github.io/compression/

Closed claims

Codec point eeee0a2f6b3c26d5 (export 99c7079bc1aa5fd3, plan portfolio, 1 lane). Sizes and decode steps are exact: each was decided by running this codec over the κ-named corpus and is recorded with its decision in the release's frontier.toml. A step is one model input evaluated for one coded bit.

corpus corpus κ input bytes archive bytes decode steps steps per byte status decided by
uorc-v0.3.0 9b781bfc9840 1442315 575285 198707536 137.77 proved factory
canterbury c97b3c539721 2810784 350393 474920872 168.96 proved factory
silesia 45094a500116 211938580 40900503 40067382768 189.05 proved factory

UC-PERF-01 target

On every public corpus, the archive is no larger than each pinned comparator's. A refuted row is a comparator this codec does not yet reach.

corpus comparator comparator bytes uorc bytes status
uorc-v0.3.0 gzip -9 650271 575285 proved
uorc-v0.3.0 bzip2 -9 606478 575285 proved
uorc-v0.3.0 xz -9e 595432 575285 proved
canterbury gzip -9 730625 350393 proved
canterbury bzip2 -9 542710 350393 proved
canterbury xz -9e 493080 350393 proved
canterbury zstd -19 --long 516216 350393 proved
canterbury brotli -q 11 490662 350393 proved
canterbury zpaq -m5 361718 350393 proved
canterbury 7z PPMd -mx9 575170 350393 proved
silesia gzip -9 67631918 40900503 proved
silesia bzip2 -9 54506769 40900503 proved
silesia xz -9e 48456004 40900503 proved
silesia zstd -19 --long 52646619 40900503 proved
silesia brotli -q 11 49564563 40900503 proved
silesia zpaq -m5 39112562 40900503 refuted
silesia 7z PPMd -mx9 47684624 40900503 proved

Distance to each requirement

Each requirement of Uorc.Performance derives a bound for this codec on each corpus; the row is what the run observed, the bound, and how far the observation is from it on the side the requirement forbids. A distance of 0 is the requirement met. Bytes and steps are exact and decided; rates are measured on the host named by its κ. leastArchive: the archive against the information the chosen model assigns to the input (what is left is the archive's form and the coder's rounding). wholeHost: per input, the longest chain of dependent decode steps against the total divided by the threads. hostCeiling: throughput against the input over the host's ceiling (Uorc.Host::ceilingNanos): the slower of the steps the run took (decode: the archive's exact steps; compress: the model steps every trial coded, uorc::ncm::model_steps) over the host's threads at the per-core step rate of the codec's own mixer contraction and the cache lines its tables move over the host's memory rate. The compute term covers the contraction only, so the distance includes the rest of each step; both sides count the archive's moved lines, a floor on what the compress trials move.

requirement quantity corpus at observed bound distance unit status
leastArchive archive bytes over the information the chosen model assigns uorc-v0.3.0 - 575285 574444 841 bytes proved
wholeHost longest chain of dependent decode steps over the total divided by the threads uorc-v0.3.0 4 threads (met up to 1) 198707536 49676884 149030652 steps proved
hostCeiling decompressed bytes per second under the host's ceiling (Uorc.Host::ceilingNanos) uorc-v0.3.0 host ce7702cbee43 (AMD EPYC 9V74 80-Core Processor, 4 threads, avx2) 826314 11401593 10575279 bytes/s measured
hostCeiling compressed input bytes per second under the host's ceiling (Uorc.Host::ceilingNanos) uorc-v0.3.0 host ce7702cbee43 (AMD EPYC 9V74 80-Core Processor, 4 threads, avx2) 75497 2153064 2077567 bytes/s measured
leastArchive archive bytes over the information the chosen model assigns canterbury - 350393 349228 1165 bytes proved
wholeHost longest chain of dependent decode steps over the total divided by the threads canterbury 4 threads (met up to 1) 474920872 118730218 356190654 steps proved
hostCeiling decompressed bytes per second under the host's ceiling (Uorc.Host::ceilingNanos) canterbury host ce7702cbee43 (AMD EPYC 9V74 80-Core Processor, 4 threads, avx2) 827217 9190916 8363699 bytes/s measured
hostCeiling compressed input bytes per second under the host's ceiling (Uorc.Host::ceilingNanos) canterbury host ce7702cbee43 (AMD EPYC 9V74 80-Core Processor, 4 threads, avx2) 103909 1995499 1891590 bytes/s measured
leastArchive archive bytes over the information the chosen model assigns silesia - 40900503 40896951 3552 bytes proved
wholeHost longest chain of dependent decode steps over the total divided by the threads silesia 4 threads (met up to 1) 35179213488 10016845692 25162367796 steps proved
hostCeiling decompressed bytes per second under the host's ceiling (Uorc.Host::ceilingNanos) silesia host ce7702cbee43 (AMD EPYC 9V74 80-Core Processor, 4 threads, avx2) 800801 7790846 6990045 bytes/s measured
hostCeiling compressed input bytes per second under the host's ceiling (Uorc.Host::ceilingNanos) silesia host ce7702cbee43 (AMD EPYC 9V74 80-Core Processor, 4 threads, avx2) 143624 2218440 2074816 bytes/s measured
Per family (first claim's corpus)
family members archive bytes decode steps
tiny 2 1475 631296
small 2 2479 6441216
text 3 259504 162957848
other 1 311827 28677176

The register holds 42 claims over 3 corpora and 3 codec points; 10 concern other points (earlier releases, engine candidates) and are kept by κ, never rerun.

The model on the page

Performance rows: UC-PERF-01 (benchmark_target_unmet), UC-PERF-02 (kernel_theorem), UC-PERF-03 (kernel_theorem), UC-PERF-04 (kernel_theorem), UC-PERF-05 (kernel_theorem), UC-PERF-06 (kernel_theorem), UC-PERF-07 (kernel_theorem), UC-PERF-08 (kernel_theorem), UC-PERF-09 (kernel_theorem), UC-PERF-10 (kernel_theorem), UC-PERF-11 (build_evidence), UC-PERF-16 (build_evidence), UC-PERF-17 (build_evidence).

Claims: 42 recorded, 35 proved, 1 refuted (listed above).

Engine: tick 0, rule30 register 0x9e3779b97f4a7c15, 10 strategies in the portfolio, 70 weights.

The full register is model.json, on the Pages site and attached to this release.

uorc benchmarks

uorc 0.6.0, 0 search steps per file, linux/x64. Measurement only; no target is set.

Machine ceiling: 4 cores (Intel(R) Xeon(R) 6975P-C), 15.3 GiB memory; memory copy 14966 MB/s on one core and 18694 MB/s on all 4. A decoder cannot write its output faster than the machine copies memory, so the last column is decompress throughput as a share of the all-core copy rate. uorc uses every core (8 MiB frames in parallel); the other tools run with their default thread settings.

canterbury

compressor files archive bytes ratio compress MB/s decompress MB/s decompress / copy
uorc (0 steps) 11 350393 8.022 0.12 0.64 0.0034%
gzip -9 11 730625 3.847 5.30 81.15 0.43%
bzip2 -9 11 542710 5.179 20.39 39.16 0.21%
xz -9e 11 493080 5.700 1.84 57.50 0.31%
zstd -19 --long 11 516216 5.445 1.92 86.40 0.46%
brotli -q 11 11 490662 5.729 0.73 83.33 0.45%
zpaq -m5 11 361718 7.771 0.61 0.60 0.0032%
7z PPMd -mx9 11 575170 4.887 10.44 10.06 0.054%
Per file (archive bytes)
file bytes uorc (0 steps) gzip -9 bzip2 -9 xz -9e zstd -19 --long brotli -q 11 zpaq -m5 7z PPMd -mx9
alice29.txt 152089 37151 54179 43202 48528 49212 46487 37591 38956
asyoulik.txt 125179 34553 48816 39569 44592 45138 42712 35369 38434
cp.html 24603 6351 7973 7624 7652 7716 6894 7418 7507
fields.c 11150 2402 3127 3039 3032 3017 2717 3469 3110
grammar.lsp 3721 1064 1234 1283 1292 1214 1124 2069 1326
kennedy.xls 1029744 13895 209721 130280 51868 64826 61498 16813 182238
lcet10.txt 426754 89314 144418 107706 119488 121337 113416 90013 102503
plrabn12.txt 481861 127081 194264 145577 165456 167439 163267 127568 138560
ptt5 513216 28280 52377 49759 39860 43610 40939 28561 48463
sum 38240 8943 12768 12909 9500 10978 10144 10324 12417
xargs.1 4227 1359 1748 1762 1812 1729 1464 2523 1656

silesia

compressor files archive bytes ratio compress MB/s decompress MB/s decompress / copy
uorc (0 steps) 12 40900503 5.182 0.16 0.88 0.0047%
gzip -9 12 67631918 3.134 12.74 171.52 0.92%
bzip2 -9 12 54506769 3.888 16.03 34.36 0.18%
xz -9e 12 48456004 4.374 2.10 81.82 0.44%
zstd -19 --long 12 52646619 4.026 2.97 463.93 2.5%
brotli -q 11 12 49564563 4.276 0.60 246.53 1.3%
zpaq -m5 12 39112562 5.419 0.57 0.56 0.0030%
7z PPMd -mx9 12 47684624 4.445 7.82 7.24 0.039%
Per file (archive bytes)
file bytes uorc (0 steps) gzip -9 bzip2 -9 xz -9e zstd -19 --long brotli -q 11 zpaq -m5 7z PPMd -mx9
dickens 10192446 2239508 3851815 2799520 2831212 2849320 2827777 2094743 2280117
mozilla 51220480 12584355 18994134 17914392 13376240 14978836 13872265 12041055 15911894
mr 9970564 2157575 3673937 2441280 2751892 3105810 2823136 2181310 2320767
nci 33553445 1380245 2987529 1812734 1449272 1615898 1519768 1251108 1484569
ooffice 6152192 1794232 3090434 2862526 2427224 2595308 2478855 1766550 2496131
osdb 10085684 2223567 3716337 2802792 2844556 3099934 2816278 2204741 2384175
reymont 6627202 1006322 1820826 1246230 1315592 1348541 1332158 956499 1054291
samba 21606400 3167616 5408266 4549759 3739524 3876656 3766340 3053820 3815315
sao 7251944 3979025 5327037 4940524 4425664 5000573 4586092 3899258 4783825
webster 41458703 6386553 12061616 8644714 8368672 8592754 8428575 5666831 6942883
x-ray 8474240 3652495 6037707 4051112 4491264 5129825 4682753 3669701 3850893
xml 5345280 329010 662280 441186 434892 453164 430566 326946 359764

Ratio is input bytes over archive bytes, and throughput is input MB per second, both over the files that compressor round-tripped. ✗ marks a failed round trip. zpaq and 7z archive bytes include their container headers. Tool versions:

  • gzip -9: gzip 1.12
  • bzip2 -9: bzip2, a block-sorting file compressor. Version 1.0.8, 13-Jul-2019.
  • xz -9e: xz (XZ Utils) 5.4.5
  • zstd -19 --long: *** Zstandard CLI (64-bit) v1.5.7, by Yann Collet ***
  • brotli -q 11: brotli 1.1.0
  • zpaq -m5: zpaq v7.15 journaling archiver, compiled Jan 5 2021
  • 7z PPMd -mx9: 7-Zip 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20