Skip to content

Safety and robustness

Jump to bottom
YadeWira edited this page Nov 7, 2026 · 3 revisions

Safety and robustness

PJA's first rule: the original comes back byte for byte, or nothing is written. This page measures what that means against the other JPEG recompressors: packJPG 5.0g, Lepton (Dropbox, f34c7f4), brunsli (Google, c18b962) and JPEG XL's lossless JPEG transcoding (cjxl/djxl 0.11.2), with PJA 0.9.0 (0.9.5 fixed the two weaknesses it found: see below). Measured on 05/11/2026; a second person (a skeptic pass) re-ran the strongest claims, and corrected two.

What PJA does

  • Verifies before writing. Compressing decompresses the result and compares it with the input; extracting checks the stored size and BLAKE3-128 hash. On a mismatch nothing is written (doc/CLI.md).
  • Writes atomically. An output goes to a new temporary in the same directory and takes its name only once complete and verified, without replacing anything (O_EXCL, link or renameat2(RENAME_NOREPLACE)); with -f the temporary is renamed over the old entry, never written through a link.
  • Refuses bombs before decoding. The .pja declares the original's size: at most 65,536 times the payload, and --max-output (256 MB by default) caps it.
  • Refuses what it does not know. A stream from a newer PJA is a clean "future version" error, never a misread (doc/STABILITY.md).
  • Fails safely inside. The Pascal code is compiled with range and overflow checks, mandatory; the C core is small, fuzzed, and covered by the verification: a wrong result is never written.

The comparison

test PJA packJPG Lepton brunsli JPEG XL
Single-bit flips in the compressed file: 4000 per tool (20 photos x 200). Counts are error / silent wrong output (exit 0) / crash / timeout 4000 / 0 / 0 / 0. I re-ran 500 new flips (half in the first 64 bytes): 500 errors, 0 silent 3996 / 4 / 0 / 0. I re-ran the 4 silent cases and all 4 came out the same: +20, +1878, +2869 and +100 bytes; djpeg decodes 2 without warnings and warns on 2. New run: 500 of 500 errors 287 / 3712 / 0 / 0, plus 1 identical. 3684 of the 3712 printed STREAM_INCONSISTENT or THREAD_PROTOCOL_ERROR but still exited 0. I re-ran 7 of them: all 7 exited 0, 4 with a 0-byte file and 3 with a truncated JPEG that djpeg rejects 3805 / 195 / 0 / 0. I re-ran 6 of the silent cases: all 6 exited 0 and gave valid JPEGs that djpeg decodes without warnings, so the damage can't be detected afterwards. New run: 44 silent out of 500 3639 / 361 / 0 / 0. I re-ran 5 of the silent cases: all 5 gave valid JPEGs (djpeg clean). 12 of the 361 fell back to re-encoding from pixels. New run: 49 silent out of 500
Truncation: 1000 per tool (error / silent / crash / timeout) 1000 / 0 / 0 / 0 1000 / 0 / 0 / 0 214 / 782 / 1 / 3 (timeout above 60 s). Correction: the 1 crash (rc -9 after a recoder.cc assert) did not reproduce. On my re-run it was rc 1 ASSERTION_FAILURE, leaving a 0-byte file. Lepton is not deterministic under damage 1000 / 0 / 0 / 0 1000 / 0 / 0 / 0
Output file left on disk after a reported error No (0 of 5000; 0 of 48 header-byte flips) No (0 of 4996) Yes, in all 501 error cases (checked in results.tsv) No (0 of 4805) No (0 of 4639)
Checksum or hash of the original stored in the file Yes: BLAKE3-128 and orig_size (doc/FORMAT.md lines 39-40). I flipped bits in the hash field and got 'does not verify; nothing written', rc 1, so the check works. In the random-flip runs the codec's own checks caught everything before the hash was reached No (source read: no crc or hash) No: only the size (u32), which the decoder does not enforce, and zlib Adler-32 over the header part No: only a weak ANS final-state check No (neither jbrd nor the codestream has a checksum)
Compressor decodes and compares before writing, by default Yes (--no-verify turns it off) No in single-thread mode (the default); -ver turns it on; forced on with multiple threads (source read) Yes (-skipvalidate turns it off) No, and there is no option for it No, and there is no option for it
Compressor fuzz (10 min, mutated JPEGs): compress exits 0 but the original cannot be restored 0 of 51,880 accepted (but only 53k runs, the fewest of the five) 199. I re-ran 8 of them: all 8 lost data (a=0, x=0, different bytes). With -ver, all 8 were refused at compress time with nothing written 0 (it validates by default) 0 17. I re-ran 6 of them: all 6 lost data. cjxl reported 'lossless transcode' and exited 0, djxl exited 0, the output bytes differ (e.g. 39857 -> 24986 B), and the file has a jbrd box
Decompressor fuzz (10 min): silent wrong output / signal crashes / hangs over 30 s 0 / 0 / 0 (148k runs, the fewest) 458 / 0 / 0 27,587 / 0 / 126 (165k runs ended with its own assertion exit code instead of a signal) 15,516 / 0 / 0 32,358 / 0 / 0
Compressor fuzz crashes 0 0 0 signals (387 exits carried assertion text) 0 6 SIGABRT from bad_alloc under a 4 GiB address-space limit (a forged huge SOF)
Disk full (tmpfs at about 50% of the output) rc 1, nothing left; an old file under the output name is kept intact. Reproduced: x and x -f, 'OLDCONTENT' kept rc 1, partial file deleted; with -o the old file is lost (not re-run by me) Decompress exits 0 with a TRUNCATED JPEG (4,300,800 of 8,465,129 B). Reproduced 3 of 3, including over an old file; it prints 'write error' and 'UNSUPPORTED_JPEG' but still returns 0 rc 1, but a truncated file stays under the final name. Reproduced with dbrunsli rc 1, truncated file left (not re-run by me)
Output name is an existing file or a symlink Refuses without -f. With -f, renames a temp file over the link: the link target is untouched and a dangling target is not created. Reproduced Without -o it silently writes in_.pjg and exits 0. With -o it writes through the symlink and the target is overwritten. Reproduced Silently writes through the symlink (target overwritten). Reproduced Writes through the symlink and creates a dangling link's target. Reproduced Writes through the symlink (target overwritten). Reproduced
Killed during the write (SIGKILL at the moment the output appears, 10 trials) Final name absent; 0.9.0 left a partial hidden .pja-tmpN behind. Since 0.9.5 the next run reports it and --clean-temps deletes it; Ctrl-C and other signals delete it at once 0-byte file under the final name Partial or empty file under the final name Partial file under the final name Partial file under the final name
fsync of the output No in 0.9.0; yes since 0.9.5 (the file before its rename, the directory after; since 0.9.7 on a thread of its own, beside the next file's work) No No No No
Format: version byte; a newer file read by an older decoder Header version and stream version; Codec/Format(FutureVersion), clean refusal, no output. Written freeze from 0.9.0 (doc/STABILITY.md) Version byte; old files from other main versions are refused (2.5j files are rejected by 5.0g); no format stability across main versions Version byte; refusal with a misleading 'v0.5' message that leaves a 0-byte file; project archived Version bits; generic 'Failed to parse'; no promise ISO standard; no version byte to bump; not measured

"Silent" means exit code 0 with output that is not the original. Lepton usually prints an error but still exits 0, so a script that checks exit codes accepts the damage; brunsli's and JPEG XL's silent outputs are valid JPEGs, so nothing downstream can tell.

How it was measured

I re-checked the claims myself in /mnt/IA_LAB/agentes/pja/safety/skeptic. Every command ran under taskset -c 32-63 (Python helpers ran with -I), with at most 12 parallel jobs. Binaries: PJA 0.9.0 from integrity/build/pja ('pja 0.9.0', a local build from the tag), plus packJPG, lepton, cbrunsli/dbrunsli and cjxl/djxl from RULES. The repos stayed unchanged (git status in /home/forum/git/pja is clean).

Checks:

  1. Re-tallied integrity/results.tsv with awk. Every count in the integrity table matches: 25,000 rows, and 501 Lepton errors that left output.
  2. rep.py re-ran individual cases through harness.py's own decompress(), comparing output length and identity and running djpeg on the result. All 4 packJPG silent cases reproduced exactly (+20, +1878, +2869, +100 B; djpeg clean on 2 and rc 2 on 2). 6 of 6 sampled brunsli and 5 of 5 sampled jxl silent cases reproduced as valid JPEGs that djpeg decodes cleanly. 7 of 7 Lepton silent cases reproduced with exit 0 and STREAM_INCONSISTENT text: 4 left 0-byte files and 3 truncated files that djpeg rejects. The single Lepton 'crash' did not reproduce: rc 1 with ASSERTION_FAILURE.
  3. fresh.py: 500 new flips per tool with a different seed, half of them in the first 64 bytes. pja 500 errors; packjpg 500 errors; brunsli 451 errors / 44 silent / 5 identical; jxl 390 / 49 / 61. No error case left output.
  4. Header test on one .pja: I flipped bit 0 of each byte in 0..47. All 48 were refused and none left output. Flips in the hash field gave 'does not verify; nothing written', which shows the BLAKE3 check fires.
  5. Disk full: unshare -rm with a 4200k tmpfs and an 8,465,129 B JPEG (dsc.jpg). Lepton decompress gave rc 0 and a 4,300,800 B truncated JPEG, 3 of 3 times, once over an old file. dbrunsli gave rc 1 and left a truncated file. pja x -f gave rc 1 'No space left on device (Io(28))' and the old file stayed intact. pja x gave rc 1 and left nothing.
  6. Symlink and dangling-symlink tests on photo.jpg. Lepton, cbrunsli, cjxl and packJPG -o wrote through the link and replaced the victim's content. packJPG without -o wrote in_.pjg with rc 0. pja refused (rc 1). pja -f replaced the link with a regular file and left the victim intact. cbrunsli created the dangling link's target; pja -f did not.
  7. Compressor losses: 8 random packJPG LOSS samples from robust/runs/enc_packjpg/found all compressed and decompressed with rc 0 and differing output, and -ver refused all 8 (rc 1, no .pjg). 6 cjxl LOSS samples: cjxl 0 'lossless transcode' and djxl 0, with differing bytes; checked one .jxl and it has a jbrd box.

Not re-run by me: the kill-timing trials, the bomb and RSS numbers, the fuzz execution counts, the cross-version tests and the desk research (taken from the sub-reports).

Limits and caveats

  • Corrected: the Lepton truncation 'crash' (rc -9 after an assert in recoder.cc) did not reproduce. The same input gave rc 1 ASSERTION_FAILURE with a 0-byte file. Report it as 1 non-deterministic event, not a reproducible crash.

  • Corrected: the robustness table credits PJA's 0 silent outputs to the BLAKE3 check alone. In the random-flip runs the codec's own checks rejected every case before the hash was reached. The hash does work when it is hit (flips in the hash field gave 'does not verify; nothing written'), so the 0 comes from both layers together.

  • Fuzz effort was not equal. PJA ran the fewest executions (148k decompress, 53k compress), so its zeros come from the smallest sample. The fuzzer was dumb (not coverage-guided), ran 10 min per tool and had no ASan.

  • PJA 0.9.0 is a local build from the v0.9.0 tag. The v0.9.0 GitHub release asset did not exist ('release not found'; the latest release is v0.8.0).

  • 'Silent' means exit 0 with output that is not byte-identical. Lepton usually prints an error yet still exits 0, so a script that checks exit codes accepts the damage. brunsli and jxl silent outputs are valid JPEGs, so nothing downstream detects them.

  • Only 20 photos for the integrity test and 7 seeds for the fuzzer. No JPEG-LS or MPO inputs in the damage tests.

  • The kill-mid-write and power-loss results are for PJA 0.9.0. Since 0.9.5, PJA deletes its temporaries on SIGINT, SIGTERM, SIGHUP, and SIGPIPE. It reports what a SIGKILL leaves on the next run and deletes it only with --clean-temps. It flushes each output and (on Unix) its directory before reporting success. Of the tools compared, PJA 0.9.5 is the only one that calls fsync. Behaviour after a real power loss was still not measured: the test suite checks the order of system calls with strace (fsync, then rename, then fsync of the directory), not an actual power cut.

    Measured cost of flushing, 151 photos (76 MB), -th 8, medians of 5 runs, two series: SATA SSD a +3.7% / +4.8%, x +7.1% / +4.4% compared with 0.9.0; tmpfs within 1%; --no-sync within 1% of 0.9.0.

  • packJPG's compressor losses happen only with default options (-ver is off in single-thread mode). With -ver, the 8 I re-ran were refused.

  • Disk-full tests ran on tmpfs inside an unprivileged user namespace; ext4 was not tested.

  • On whether this is thanks to Pascal: no. Every advantage measured here comes from design choices that any language could implement: a stored hash and size of the original, decode-and-compare before writing, temp file plus rename with O_EXCL/RENAME_NOREPLACE, lstat checks, and refusing future versions. Language was not measured, and packJPG (C++) shows that a version byte alone does not give compatibility.

  • Artifacts: /mnt/IA_LAB/agentes/pja/safety/skeptic/{rep.py, fresh.py, cases.txt, lep0.txt, df/, sl/, loss/, hh/}.

What PJA does not do

  • Repair. A damaged .pja is detected, not repaired: keep backups, or store .pja files in an archiver with recovery records (RAR, par2).

  • Authenticate. The hash catches accidents (a disk, a network, a hand), not someone who rewrites a .pja and its hash on purpose. There is no signature and no encryption, by design.

  • Interrupted writes and power cuts (fixed in 0.9.5). Every output is written to a temporary file named .pja-tmp-PID-N in the same directory. The temporary is flushed to the disk (fsync, or FlushFileBuffers on Windows) before it takes the output's name, and on Unix the directory is flushed after the rename. Windows has no call to flush a directory, so there the name is left to the file system's journal: a power cut just after a run can lose a name there, but not leave a partial file under it. So after a power cut an output is either complete or missing, never empty or partial under its name.

    On Ctrl-C, kill, a closed terminal, or a closed pipe (SIGINT, SIGTERM, SIGHUP, SIGPIPE; on Windows Ctrl-C, Ctrl-Break, or closing the console), PJA deletes the temporaries it is still writing. It then exits as the signal would have (130 for SIGINT, 141 for SIGPIPE).

    A kill -9 or a crash cannot clean up, so a temporary can still be left behind. The next run that writes to that directory lists it on stderr. --clean-temps deletes it, under these conditions: the name matches PJA's own pattern, it is a regular file (not a link or a directory), it is older than an hour, and the process whose PID is in the name is no longer running. Nothing is deleted unless you ask. The process check only sees this machine's processes, so on a directory shared over NFS or with a container the hour of age is the only guard.

    Flushing costs 3.7-7.1% on a SATA SSD (151 photos, -th 8) and under 1% on tmpfs. --no-sync turns it off for outputs that can be recreated.

Home

Clone this wiki locally