PJA 0.4.0

New in the zpaq-std plugin: five z codecs that let zpaq-std store once what several files of a block share.

• The z codecs. pjaz, pjalsz, pjaxz, pjallz and pjalsxz each work like the codec without the z: the same files and the same streams. The difference is in the block. zpaq-std passes the PJA streams of a block's files through zpaq's level 1 LZ77 (the -m1 method), and a match can point back into the files before it. Each z codec carries a ZPAQL decoder that expands that LZ77 and then decodes the files as usual. So whatever files share (one image saved progressive and baseline, with or without restarts, gives the same coefficients) is stored once. This is what LZMA2 does after PJA in 7-Zip.

• Smaller ZPAQL programs. zpaq-std's issue #4 led to three improvements in the compiler, only for programs that are not frozen yet:

  • each constant is built with the shortest sequence of instructions found;
  • in the start-up code, constants close to one another are built from the one before;
  • the most-used variables are placed where one instruction reaches them.

  The z decoders are 37 to 41% smaller than the decoders of their base codecs (pjaz: 39,458 bytes against pja's 62,294), and libzpaq extracts about 25 to 30% faster with them than with the first z versions. The five frozen decoders do not change, byte for byte.

• Nothing else changes. The pja command, libpja (7-Zip) and the five existing codecs are 0.3.0's.

Measured by zpaq-std with the z codecs, before the variable placement (-sat, a z codec only where it wins, every file decoded and compared before it is written), size of the .zpaq:

set	before	with the z codecs
244 arithmetic-coded	4,763,128	1,259,556 (18.8%)
640 generated	13,217,825	3,268,732 (17.9%)
6 MPO	33,092,189	33,086,536
78 JPEGs of every kind	23,505,647	23,385,551
151 photos	59,207,732	59,170,972

Every block in those archives now carries a decoder some 17 KB smaller. With libzpaq (zpaq 7.15's engine), each set in one block of its codec's segments came back byte for byte, with a window of 2^24 and with the smallest one that holds the largest segment. Out of 150 damaged segments per codec, none made the decoder hang. Details in doc/CODEC.md.

Downloads

platform	the pja command	the zpaq-std plugin	libpja (C library, 7-Zip plugin)
Linux x86-64	pja-0.4.0-linux-x86_64.tar.gz	pja-0.4.0-zpaqstd-plugin-linux-x86_64.tar.gz	pja-0.4.0-lib-linux-x86_64.tar.gz
Linux x86 (32-bit)	pja-0.4.0-linux-x86.tar.gz	pja-0.4.0-zpaqstd-plugin-linux-x86.tar.gz	pja-0.4.0-lib-linux-x86.tar.gz
Linux arm64	pja-0.4.0-linux-arm64.tar.gz	pja-0.4.0-zpaqstd-plugin-linux-arm64.tar.gz	pja-0.4.0-lib-linux-arm64.tar.gz
macOS arm64	pja-0.4.0-macos-arm64.tar.gz	pja-0.4.0-zpaqstd-plugin-macos-arm64.tar.gz	pja-0.4.0-lib-macos-arm64.tar.gz
Windows x64 (7 and later)	pja-0.4.0-windows-x64.zip	pja-0.4.0-zpaqstd-plugin-windows-x64.zip	pja-0.4.0-lib-windows-x64.zip
Windows x86 (XP and later)	pja-0.4.0-windows-x86.zip	pja-0.4.0-zpaqstd-plugin-windows-x86.zip	pja-0.4.0-lib-windows-x86.zip

pja-0.4.0-zpaqstd.tar.gz holds the plugin's sources as zpaq-std vendors them, with VENDOR.md. CI builds and checks every binary on its own platform (.github/workflows/release.yml); libpja is checked with its C test and, on Linux x86-64, through a real 7-Zip. SHA256SUMS has the checksums. The Linux builds need glibc 2.34 or later. The macOS ones are not signed: run xattr -d com.apple.quarantine after unpacking.