ncarnate 2.2.0 — Audit & Convert

ncarnate 2.2.0 — Audit & Convert

This release adds an archive-modernization workflow: a read-only audit that plans a migration, and a converter that executes exactly what the audit blessed — with integrity checks and non-destructive defaults throughout. It is the first PyPI release to include the audit subcommand and the migration-manifest contract (developed as the internal 2.1.0 milestone), released together with the converter.

The existing single-file/single-directory form (ncarnate <path>) is unchanged.

Highlights

🔍 ncarnate audit — plan a migration without touching your data

Walk an archive and classify every file without reading its science arrays or modifying anything — a cheap, read-only pass you can run against a terabyte before deciding what to convert.

ncarnate audit /data/archive --output manifest.jsonl --checksum sha256

• Detects formats, inspects metadata only, and sorts each file into a stable status taxonomy: ready, ready_no_geolocation, already_modern, unsupported, malformed, unsafe, unknown.
• Writes a versioned migration manifest — a per-file JSONL record (path, checksum, status, issues, conversion plan) with a checked-in JSON Schema and an append-only issue-code registry. A flat CSV projection is available for triage.
• Prints a readiness summary by files and bytes. Never writes to audited files; never touches the network.
• Opt-in --checksum sha256 records per-file hashes so the manifest can later be executed safely.

✅ ncarnate convert --manifest — execute the plan, safely

Hand the converter that manifest and it modernizes exactly the granules the audit blessed — into a mirrored output tree, leaving your originals untouched by default.

ncarnate convert --manifest manifest.jsonl --out-dir ./modern --root /data/archive

• Integrity by construction: re-verifies each file's recorded sha256 before touching it, so a file that changed since the audit is skipped with an error rather than silently converted. A null hash is refused without --allow-unverified; a blocker is never converted.
• Non-destructive by default: writes to --out-dir (HDF4/HDF-EOS2 → .nc, netCDF names kept); --in-place is a separate opt-in, and --skip-existing makes an interrupted run resumable.
• Resilient at scale: per-record isolation means one bad file never aborts the run; an end-of-run summary counts converted / skipped / failed with reasons, and the exit code is non-zero iff a selected record failed.
• Trusted read base: because a manifest is untrusted input, manifest mode requires an operator-controlled base — --root DIR (anchor reads to a directory you control; the archive's current location) or --allow-manifest-root to explicitly trust the manifest's recorded root.

Security & robustness

The audit + converter were put through an independent security-and-correctness review and hardened before release, including: allocation-bomb (declared-oversize) files are correctly refused as unsafe; untrusted manifest/metadata input can no longer surface as an uncaught traceback; and the converter never trusts an attacker-controllable read base by default.

Public API

• audit_path, AuditOptions — the read-only audit.
• convert_manifest, ConvertOptions — the manifest-driven converter.
• Both are exported from the top-level ncarnate package and documented in the API reference.

Compatibility

• No breaking changes to existing usage: ncarnate <path> and ncarnate convert <path> behave exactly as before.
• New: manifest-mode conversion (convert --manifest) requires --root or --allow-manifest-root (see above).
• Python 3.10–3.13, Linux and macOS. (On Windows, the HDF4 path needs conda-forge pyhdf or WSL; the netCDF path works from PyPI wheels.)

Install

pip install --upgrade ncarnate
# or
conda install -c conda-forge ncarnate

Full changelog: see CHANGELOG.md.