Experiments

Reproducible command lines for the pipelines cuSZ ships. Each block compresses, then decompresses and reports quality against the original, so a run is self-checking.

Set up once:

export PATH=$(pwd)/bin:$PATH               ## specify the path temporarily
cd data && sh ./sh.get-sample-data         ## download sample data
export CESM=$(pwd)/cesm-CLDHGH-3600x1800
export EB=1e-4

${CESM} is 2D, 3600x1800, f32. Substitute your own file and -l extent throughout; nothing below is specific to this dataset.

Lorenzo, the default

cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600x1800 -z --report cr
cusz -i ${CESM}.cusza -x --compare ${CESM}

--predictor lrz is the default and may be stated explicitly. lrz-zz selects the zigzag variant. Error control is -m r2r (relative to value range) or -m abs; -e is the bound.

Composing a command line

Every block on this page is the same skeleton with different middle parts. The task flag has three spellings, and may instead be given as a subcommand in argv[1].

Compressing: -z, --zip, --compress

Subcommand form: cusz compress ..., cusz comp ..., cusz zip ...

part	flag	values
input	-i, --input	the file to compress
type	-t, --type, --dtype	f32/f4, f64/f8
extent	-l, --len, --xyz, --dim3	<x>, <x>x<y>, <x>x<y>x<z>
mode	-m, --mode	r2r (default), abs
bound	-e, --eb, --error-bound	e.g. 1e-4
predictor	-p, --pred, --predictor	lrz (default), lrz-zz, spl = spl-y25, spl-y24
primary codec	-c1, --codec1	hfr-v4 (default), hf, hf-rev2, hfr-v2, hfr-v3, hfr-pbkc, hfr-pbkgo, lc = tcms, fzgcodec
secondary codec	-c2, --codec2	lc, rtr, bitr, hi-cr, hi-tp
shorthand	-s, --scheme	tp = speed (sets codec1 lc), cr (sets codec1 hfr-v4)
report	-R, --report	cr
skip	-S, --skip, --exclude	huffman, write2disk

Decompressing: -x, --unzip, --decompress

Subcommand form: cusz decompress ..., cusz decomp ..., cusz unzip ...

part	flag	values
input	-i, --input	the .cusza archive
verify	--compare, --origin	the original file; prints PSNR and max error
skip	-S, --skip, --exclude	write2disk

Type, extent, mode, bound, predictor and codecs are all read back from the archive header, so none of them is repeated on -x.

Two things about reporting that are easy to trip over:

• --compare is what prints PSNR, not a report key. There is no quality key; --compare <original> alone produces the quality line.
• time is currently disabled. On -z it prints Reporting time is disabled/to be updated; on -x any report string containing it fails with psz_review_decompression is to be updated. Use an external timer until that returns.

Lorenzo, the default

cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600x1800 -z --report cr
cusz -i ${CESM}.cusza -x --compare ${CESM}

--predictor lrz is the default and may be stated explicitly. lrz-zz selects the zigzag variant. Error control is -m r2r (relative to value range) or -m abs; -e is the bound.

Composing a command line

Every block on this page is the same skeleton with different middle parts.

part	flag	values	applies to
task	-z / -x	compress / decompress
input	-i	the file; a .cusza on -x	both
type	-t, --type	f32/f4, f64/f8	-z
extent	-l, --len	<x>, <x>x<y>, <x>x<y>x<z>	-z
mode	-m, --mode	r2r (default), abs	-z
bound	-e	e.g. 1e-4	-z
predictor	--predictor, --pred	lrz (default), lrz-zz, spl = spl-y25, spl-y24	-z
primary codec	-c1, --codec1	hfr-v4 (default), hf, hf-rev2, hfr-v2, hfr-v3, hfr-pbkc, hfr-pbkgo, lc = tcms, fzgcodec	-z
secondary codec	-c2, --codec2	lc, rtr, bitr, hi-cr, hi-tp	-z
shorthand	-s, --scheme	tp = speed (sets codec1 lc), cr (sets codec1 hfr-v4)	-z
verify	--compare	the original file; prints PSNR and max error	-x
report	-R, --report	cr	-z
skip	-S, --skip	huffman, write2disk	both

On -x the type, extent, mode and bound come from the archive header, so they are not repeated.

Two things about reporting that are easy to trip over:

• --compare is what prints PSNR, not a report key. There is no quality key; --compare <original> alone produces the quality line.
• time is currently disabled. On -z it prints Reporting time is disabled/to be updated; on -x any report string containing it fails with psz_review_decompression is to be updated. Use an external timer until that returns.

Spline

cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600x1800 --predictor spl -z --report cr
cusz -i ${CESM}.cusza -x --compare ${CESM}

Two variants exist and the spelling picks one:

flag	variant	dimensions
--predictor spl, spl-y25, spline-y25	y25, the default spline	2D and 3D
--predictor spl-y24, spline-y24	y24	3D only

spl-y24 on a 2D field is not rejected: it compresses, reports a ratio, and reconstructs to the wrong values. On ${CESM} at -m r2r -e 1e-4 it returns PSNR=11.3 with max_error=9.2e-01 against a bound of 1e-4. Check the PSNR whenever you change predictor.

Spline is interpolation-based and generally reaches a higher ratio than Lorenzo at the same bound on smooth fields, at lower throughput. On a 3D field (any f32 volume; the extent is what matters):

export HURR=/path/to/CLOUDf48.f4
cusz -t f32 -m r2r -e ${EB} -i ${HURR} -l 500x500x100 --predictor spl -z --report cr
cusz -i ${HURR}.cusza -x --compare ${HURR}

The LC path

Requires a build carrying the LC codec fix. On a cuSZ without it, LC compresses cleanly and decompresses to noise -- the archive is written, the run reports success, and the reconstruction is wrong. Check with the round trip below before trusting an LC measurement: a correct run reports the same PSNR as the Lorenzo block above, not a negative one.

LC replaces the Huffman stage with the LC-framework codecs. Three shapes:

# TCMS-only: eq is TCMS-compressed, anchor and outliers stay raw
cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600x1800 --codec1 lc -z --report cr
cusz -i ${CESM}.cusza -x --compare ${CESM}

# HiTP: TCMS over eq, BITR over anchor+outliers
cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600x1800 --codec1 lc --codec2 lc -z --report cr

# HiCR: Huffman over eq, RTR over the whole encoded block
cusz -t f32 -m r2r -e ${EB} -i ${CESM} -l 3600x1800 --codec1 hf --codec2 lc -z --report cr

--scheme tp is shorthand for --codec1 lc, so it takes the same path. --codec1 tcms is accepted as a synonym for lc.

LC trades ratio for throughput: on ${CESM} at -m r2r -e 1e-3, TCMS-only reaches about CR 3.2 where the default Huffman path reaches about 13.5, at the same PSNR. Whether that trade is worth it depends on whether the bitstream stage is your bottleneck.

Checking a run is actually lossless-to-bound

--compare <original> prints PSNR and max error. Two habits worth keeping:

• Read the sign. A broken decode reports a negative PSNR. A regular expression like PSNR=[0-9.]+ silently matches nothing on those lines and reads as "no output" rather than as failure.
• Use a bound tight enough to produce outliers. At a loose bound a field may quantize to zero out-of-radius values, so a path that drops outliers entirely still round-trips correctly and looks healthy. -m abs -e 1e-4 on ${CESM} exercises them; -m r2r -e 1e-3 largely does not.

Skipping stages

cusz ... -z --skip huffman        # predictor only, no entropy coding
cusz ... -x --skip write2disk     # decompress and report without writing output

Useful for isolating where time goes: run once whole, once with the codec skipped, and the difference is the codec's share.