cxop

cxop

Pre-compress files into sibling variants for HTTP content negotiation. For every eligible source, cxop writes .gz, .br, and .zst next to it, so the CDN / edge Lambda / nginx gzip_static / etc. can pick the encoding the client advertised in Accept-Encoding. The name is compress-xop, following imop's shape (batch producer with options.bash scaffolding).

Decompression pairs with arx — arx file.html.br handles the reverse.

Usage

cxop file.html                       # Compress one file to siblings
cxop public/                         # Walk a directory tree
cxop --only html,css,js public/      # Restrict to an allowlist
cxop --formats br,zst public/        # Skip gzip
cxop clean file.html                 # Remove siblings of a single file
cxop clean public/                   # Remove siblings tree-wide
cxop clean --no-recursive public/    # Remove siblings, top level only
cxop clean --orphans public/         # Include orphans (source missing)
cxop exts public/                    # Survey extensions + allowlist hint

Directory or file is inferred automatically — cxop <path> becomes batch for a directory, compress for a file. clean <path> follows the same convention: pass a file to clean just its three siblings, or a directory to clean tree-wide.

Commands

Command	Description
auto <path>	Infer: file → compress; directory → batch (default)
compress, c	Compress one file to sibling variants
batch, b	Walk a directory and compress eligible files
clean	Remove .gz/.br/.zst siblings for a file or under a directory
exts, extensions	Survey extensions in a directory: count, eligibility, and an --only allowlist suggestion

Options

Option	Description
--formats LIST	Codec subset: gz,br,zst (default: all available)
--only LIST	Restrict to these extensions (allowlist mode)
--exclude LIST	Skip these extensions in addition to the default blocklist
--include LIST	Force-include extensions normally on the blocklist
-q, --quality N	Quality 0–100 (default: 100, mapped per codec)
--best	Slow-but-best: zopfli for .gz, --ultra -22 for .zst
--min-size N	Skip sources smaller than N bytes (default: 150)
--min-ratio PCT	Drop variant unless at least PCT% smaller than source; 0 = no check (default: 0)
--force	Re-compress even when sibling mtime is fresh
--keep-larger	Don't delete existing siblings that fail the size check
--no-recursive	clean / exts: act on the top directory only, don't descend
--orphans	clean: also remove variants whose original source is missing (default: skip and report)
-j, --jobs N	Parallel jobs (default: 1, 0 = nproc)
-n, --dry-run	Show commands without writing

How files are selected

Negative list by default — every regular file is a compression candidate unless its extension is on a small blocklist of known-self-compressed or not-worth-it types:

• Images — jpg, jpeg, png, gif, webp, avif, heic, heif, bmp, tiff, tif, ico
• Video — mp4, m4v, mkv, webm, mov, avi, wmv, flv, 3gp, 3g2
• Audio — mp3, m4a, aac, ogg, opus, flac, wav
• Archives / already-compressed — gz, br, zst, xz, bz2, lz, lzma, lzo, 7z, zip, rar, arj, cab, tgz, tbz, tbz2, txz, tzst
• Self-compressed fonts — woff, woff2, eot
• Zip-based documents — pdf, epub, mobi, azw, azw3, kfx, docx, xlsx, pptx, odt, ods, odp, odg, odb
• Disk images / binaries — iso, dmg, img, vmdk, qcow2, jar, war, apk, aab
• Encrypted / near-random — age, gpg, pgp, asc, enc

Override the list with --only EXT,... (allowlist — nothing else is compressed), --exclude EXT,... (add to blocklist), or --include EXT,... (force-in blocklisted extensions).

Size-check safety net

By default (--min-ratio 0) every variant produced by the codec is kept, even if it's larger than the source — the compressor decides, not cxop. Pass --min-ratio PCT to drop variants that don't beat the source by at least PCT% (e.g. --min-ratio 5 keeps only variants at least 5% smaller; useful when you'd rather fall back to the uncompressed original than serve a marginal win).

When the check does drop a variant, any existing stale sibling is also deleted so the deploy state matches the current rules. Use --keep-larger to preserve them instead.

Codecs

Codec	Default tool	Slow-best tier (--best)	Notes
.gz	gzip -9 -c -n	zopfli --gzip --i15	Universally understood. zopfli is ~100× slower for ~5% smaller output.
.br	brotli -c -q 11	(already max)	Default for HTTPS clients; typically smallest for text.
.zst	zstd -q --long=27 -19	zstd --ultra -22	Near-brotli ratio, faster decompression, increasing HTTP support.

Missing tools are warned about once and the affected codec is skipped; the other codecs still run. Install:

brew install gzip zopfli brotli zstd
apt install gzip zopfli brotli zstd     # Debian/Ubuntu
dnf install gzip zopfli brotli zstd     # Fedora; zopfli is in EPEL on RHEL-likes

Freshness and re-runs

Re-running cxop on an already-compressed tree is fast: for each (source, codec) pair, if the sibling exists and its mtime is not older than the source's, the existing sibling is kept untouched. Touch the source (or use --force) to re-compress.

Cleaning variants

cxop clean <path> removes .gz/.br/.zst siblings — use it before re-running with different codec / quality settings, or as a final step in a deploy that doesn't want variants on disk anymore.

• cxop clean file.html — clean just one file's three siblings. Fastest path when you need to re-compress a single asset.
• cxop clean public/ — walk the tree, remove every sibling next to a still-present source.
• cxop clean --no-recursive public/ — top level only, don't descend into subdirectories.

Orphans: a sibling whose source file is gone is called an orphan. By default clean skips orphans and reports the count; pass --orphans to include them. The default exists because an orphaned abc.txt.gz may be the only remaining copy of that content; silently deleting it could be data loss. Inspect what you have (cxop exts helps) before opting in.

clean supports --dry-run / -n — the summary shows Would remove counts instead of Removed.

Surveying extensions

cxop exts <dir> walks the tree and aggregates by file extension — useful when preparing an --only allowlist, auditing where disk space is going, or checking whether the default blocklist catches what you expect:

$ cxop exts public/
Extensions under public/:
       SIZE    COUNT  EXT               STATUS
    12.3 MB       15  .png              skip (blocked)
     4.1 MB      210  .html             compress
   892.3 KB       45  .css              compress
   512.1 KB       22  .js               compress
     8.4 KB        2  .webmanifest      compress

294 file(s), 17.8 MB total, 5 unique extension(s).
Current filters would compress 279 file(s) / 5.5 MB (31% of size, 94% of count), skip 15 file(s) / 12.3 MB.

Allowlist for the compressible set:
  --only html,css,js,webmanifest

Rows are sorted by accumulated size (largest first) so the heaviest extensions are at the top — answers "where is the space going?" at a glance. The split between size and count in the summary line is often revealing: a handful of big PNGs can be 70% of disk while compressible text is 99% of files.

The output reflects whatever --only / --exclude / --include you pass alongside — so you can iterate: run exts, see what's classified how, tweak the flags, re-run. The allowlist suggestion at the bottom is paste-ready; drop it into a cxop --only … public/ invocation if you want exactly those extensions compressed.

exts honors --no-recursive if you only care about the top-level.

Blog deploy example

For a Hugo build in public/ fronted by CloudFront + the aws-website-cdn-cdk edge Lambda:

hugo
cxop --jobs 0 public/
aws s3 sync public/ s3://<bucket>/ --delete

aws s3 sync uploads both originals and siblings; the Lambda (lambda/index.mjs) reads <key> + .br|.zst|.gz from S3 based on Accept-Encoding. The Lambda's current content-type dispatch covers text/plain, text/html, text/css, application/javascript, application/json, application/xml — broadening it (e.g. to image/svg+xml, application/wasm, font/ttf) is a separate change on the Lambda side; cxop already produces variants for anything eligible.

Pairing with arx

cxop is compression-only. To undo a variant, use arx:

arx file.html.br        # decompress to file.html
arx list file.html.gz   # inspect without extracting

arx already handles single-file .gz, .br, .zst via its codec-lookup table, so there's no separate cxop --decompress mode.