ppgrid 0.5.0

ppgrid 0.5.0

The audit-hardened release: the 2026-10-09 hygiene sweep (80 findings, all areas), a CLI verification surface (compare / --check / completions), config-file support, and independent IDW + nearest-neighbour baselines for benchmarking work.

What's new

• Hygiene sweep — 51 P2 + 29 P3 findings from the 2026-10-09 audit closed across all areas: named error exits for corrupt inputs (#84 a2), scoped locking with byte-identical output (#84 a3), completed temp-file cleanup on kill paths (#84 a4), CPL zstd struct version guard (#84 a6), calibration LUT pairing + old-file compat (#84 a7). Behaviour changes are pinned in tests/test_audit_84.py; the committed melb anchors stayed byte-identical.
• CLI verification batch (#66 #67 #68 #69): ppgrid compare OUT1 OUT2 (band-level diff report, exit 0/1/2), --check (post-write verification against the run plan), ppgrid completions bash|zsh, token-wins subcommand dispatch.
• Config file (#63): pipeline options in ppgrid.toml; precedence CLI flag > config > built-in default.
• Run fingerprint sidecar (#64): every successful run writes <out>/ppgrid_run.json (commit, env, resolved plan) for reproducibility.
• --log-file PATH (#65): mirrors the run log to a file.
• Baselines (#107): new ppgrid/baselines.py — independent IDW + exact k-NN (scipy, [baselines] optional extra) with a methods-section renderer for benchmark papers; core install unchanged.
• macOS fix: unlink(dir) EPERM now maps to the named ValueError (was a raw PermissionError on BSD, #84 a6 pin).

Performance claims

None this release with new numbers. The audit sweep is behaviour-preserving by design (byte-identical anchors at all six melb resolutions); internal threading changes (a5 grid_index_mt) carry no published benchmark. Last scoped numbers: the v0.3.0 "Turbo benchmarks (full AU)" README table.

Support

Python 3.11-3.14; Linux and macOS supported and CI-covered; Windows unsupported (match the current SUPPORT.md). Full matrix + feedback path (issues / discussions / email):
SUPPORT.md.

Changelog

Full detail: CHANGELOG.md
([0.5.0] entry).

Standalone binaries

Single-file builds: no Python, no compiler, no GDAL install required — the C interp ships prebuilt.

ppgrid-0.5.0-linux-x86_64.xz
sha256: 3840181c350516d8066e705e69c0edfa68e4ecdf23b191fbe82acdba392976a

macOS (arm64 / x86_64) builds to follow via the binary.yml CI matrix (PR #137).

ppgrid 0.4.1

CLI help command + leveled logging (#51). Changelog: https://github.com/marzukia/ppgrid/blob/main/CHANGELOG.md

0.4.0 — low-RAM sparse skip + write-path hardening

0.4.0 — low-RAM speed, write-path hardening, publish guard

• Low-RAM runs (regime C / per_box) are faster: output tiles whose source footprint contains no task data skip the GDAL warp entirely (conservative check — only truly-empty tiles; a no-op when nothing is skippable), and descent row-bands run in parallel (--max-band-parallel, auto-sized min(workers, nbands, 8), per-level /proc/meminfo gate with serial fallback). Output is bit-identical to the serial path, guarded end-to-end for vertically asymmetric tasks.
• Failed runs no longer leave half-written outputs: finals publish via atomic rename (.tif.tmp -> final), and a non-zero exit over pre-existing rasters warns the outputs are stale.
• Release pipeline guard: a tag that doesn't match the packaged version fails before upload; workflow_dispatch dry-run available.
• CI: 4-leg Python 3.11-3.14 matrix on PRs, main leg on pushes.
• 190 tests green, ruff clean.

v0.3.0

ppgrid 0.3.0

Turbo mode: continent-scale grids in minutes, not hours.

What's new

• --turbo — opt-in multithreaded pipeline sized to your RAM cap.
  Presets --turbo 16|32|64|128 (GB); bare --turbo = auto
  min(physical, cgroup) - 8 GB. Also --max-ram <GB> / --ram-gb <GB>
  (imply turbo) and --turbo-strict (error + exit 3 instead of warn when
  the shared path is infeasible).
• RAM budget + regimes — planning budget Bt = 0.85 * C - 4 GB
  (C = effective cap) picks a regime per run: A full in-RAM shared,
  B input in RAM + banded val/sup memmap, C shared all-memmap
  banded descent (input + val/sup file-backed memmap, 8 GB page-cache
  floor in its peak accounting), per_box per-box warp/write fallback
  taken when regime C has < 2 in-flight units (the only genuine low-RAM
  mode). A pre-check after gridding prints the cap/budget/regime actually
  used. Note --ram-gb / --max-ram cap the shared-field budget, not
  process RSS (measured: 39.4 GB RSS under a 20 GB budget).
• Byte-exact parallel writer — parallel tile reproject + parallel
  per-tile ZSTD write, gated by a compression oracle; any mismatch falls
  back to the byte-exact stock-codec parallel write (the serial
  per-band write is the last resort only if that writer's layout guards
  fail — on the reference GDAL 3.12.4 / libtiff 6.2 stack the oracle
  mismatches, so turbo runs take the stock parallel write). Output is
  sha256-identical to the serial path.
• MT read side — Pipeline(n_threads=N) (default 1 = the old serial
  path): threaded 2048px warp tiles + parallel CSV ingest for large files.
• cgroup cap fix — auto cap now walks the process cgroup hierarchy to
  the smallest finite memory.max, so capped systemd/user slices no longer
  plan a regime that OOMs the slice.

Benchmark

Full-AU, 16M points, --res 100 on marzuki-hydrogen (CPU-only):

pre-speed-pass --turbo 64 (regime A, same code): 30:45 (quiet) to 50:46 (loaded)
post --turbo 64 (regime A):                       2:33

~12-20x wall (the pre-PR number is load-dependent; best-case pre-PR is
~12x), peak RSS 43 GB under a 64 GB cap (50.4 GB sizing budget), and the
outputs are sha256-identical to the pre-PR anchors (value.tif and
support_km.tif). Also measured: per-box 3:20 at 8.5 GB RSS (the only
genuine low-RAM mode); regime C under --ram-gb 20: 6:25 pre-PR, 3:08
post, RSS 39.4 GB (--ram-gb caps the shared-field budget, not RSS).

Install

pip install ppgrid

Breaking / flag notes

• No breaking changes to the serial default, with one exception: a
  correctness fix in the banded multi-level descent. The support blend
  computed (1-a)*pv*ps instead of (1-a)*ps (buffer reuse).
  support_km.tif for multi-level grids may change vs 0.2.2 (now
  correct); value.tif is unchanged.
• New CLI flags: --turbo [16|32|64|128], --max-ram <GB>,
  --ram-gb <GB>, --turbo-strict.
• New kwarg: Pipeline(n_threads=...) (default 1, serial).
• Turbo infeasibility: default is [warn] + continue on the per-box path
  (exit 0, correct output); --turbo-strict makes it an error (exit 3).

v0.2.0

v0.2.0 - Polish release.

New:

• Added version to package.
• Moved pyarrow to optional [parquet] dependency.
• Added CHANGELOG.md, SECURITY.md, SUPPORT.md.
• Added CODEOWNERS and pre-commit config.
• Added examples/run_melb.py reproducible example script.
• Added CI and PyPI badges to README.
• Stated benchmark hardware in README.

Fixed:

• Fixed README project structure and CLI table.
• Added python keyword and Python 3.14 classifier.
• Fixed emdash in pyproject.toml description.
• Regenerated uv.lock.

Note: Previous versions (0.1.0-0.1.4) were metadata-only fixes and are now superseded by this release.