v0.7.0

What has changed

Added

• Repository format v2: New repository format with compact binary serialization for index files, replacing JSON from v1. Tree blobs use JSON (same as v1) for forward-compatible extensibility. v1 is deprecated and will be removed in a future release.
• mapache migrate: New command to convert v1 repositories to v2 format. Re-encrypts packs, re-indexes files, and rebuilds the binary index. Supports --dry-run for preview. The command can now also be interrupted cleanly with Ctrl+C, exiting with the conventional interrupt code.
• --format: New flag in init to select repository format version.
• --compression none: Per-blob compression marker in v2 enables storing already-compressed content (video, photos, archives) without zstd overhead.
• Zero blob deduplication: New BlobType::Zero deduplicates zero-filled regions across snapshots. Zero blobs are stored in pack footers with length=0, consuming no pack space.
• ECC (Reed-Solomon error correction): Optional parity-only sidecars (.ecc files) for pack files, configured via --ecc <PERCENT> during init. Striped encoding handles multi-gigabyte packs without loading the entire file into memory.
• ECC in bundle format v2: Bundle files now support inline Reed-Solomon ECC protecting the blob data section. Enabled with --ecc <PERCENT> during bundle creation. Corrupted blobs are automatically repaired during extraction.
• mapache ecc: New command to manage ECC sidecars after repository creation. Subcommands: enable, disable, set-percent, and regenerate. Allows enabling ECC on existing repositories, changing the overhead percentage, or regenerating all sidecars.
• Verify metadata: mapache verify can now check metadata files (index, snapshot, etc.) and repair them if ECC is enabled.
• Lazy index loading: Index files are now loaded on demand, reducing RAM usage for commands that don't need the full index.
• KDF calibration: Added --calibrate-kdf to allow calibrating the Argon2id parameters for a target run time in the running hardware. Memory is auto-detected (10% of RAM, clamped to 32–64 MiB).
• mapache verify --dump-pack-blobs <FILE>: Writes every pack blob descriptor as one line per blob: <blob_id_hex> <blob_type> <pack_id_hex>. Streams pack by pack with bounded memory, so it is safe on large repositories. Useful for auditing what each pack physically contains.
• mapache stats unreferenced blobs: The Snapshots section reports blobs tracked in the index but not referenced by any snapshot, with the reclaimable encoded bytes. The row is only shown when the count is non-zero. The metric lands reliably at zero after clean now that GC detects cross-pack phantom blob descriptors and stats walk cold indices too.
• copy --all: Added an option to mapache copy to copy all snapshots to a new repository in one go.
• mapache repack: New command that re-encodes every reachable blob using the current settings (e.g. --compression none, --pack-size), rebuilding the packs and dropping packs that are no longer referenced by any snapshot.

Changed

• GC repack memory: pack segments are downloaded, decoded and immediately re-encoded in a streaming pipeline with a bounded window of concurrent segments ([runtime] gc-repack-concurrency), instead of collecting whole decoded batches. Decoding is lazy — blobs are decrypted/decompressed one at a time on the write path — so neither the number of blobs nor a segment's decoded size drives peak RSS; it stays flat regardless of how much data is being repacked. [runtime] gc-decoded-budget has been removed: configs that still set it will now fail to parse (a clear unknown-key error) and the key should be deleted. Packer pool buffers are also no longer pre-allocated to the full pack size, removing a fixed RSS cost proportional to --pack-size. [runtime] gc-repack-concurrency (default 4) tunes the segment pipeline width: higher overlaps more downloads for throughput, lower trims the in-flight encoded/decoded bytes.
• copy requires a selector: mapache copy now errors unless one of --all, --snapshot, --host or --tags is given. Previously a bare copy silently copied every snapshot in the source; use --all for that.
• Restorer memory: Reduced restore memory usage by compacting the per-blob planning data and eliminating redundant allocations in the pack download pipeline.
• Lazy index eviction by blob budget: In lazy mode, the resident (hot) index pool is bounded by a soft blob budget instead of a fixed index count. The least-recently-used resident index is evicted and fully unloaded from RAM (only its lightweight metadata is kept) each time the pool exceeds the budget, and at least one index is always kept resident even when a single index is larger than the budget. Evicted indices are re-loaded from disk on the next access. Configurable via [runtime] lru-max-blobs in the TOML config (default: 1,000,000).
• Config key alignment: TOML config keys now match the corresponding CLI flag names. snapshot.num-readers and snapshot.num-packers are now snapshot.readers and snapshot.packers; the old keys are no longer accepted.
• AES-GCM-SIV nonce position: In v2, encrypted blobs place the nonce at the end ([ct | tag | nonce]) instead of the start. Eliminates an extra allocation and memory copy during encryption.
• v1 deprecation warning: snapshot, restore, and other commands now warn when operating on v1 repositories. Consider migrating with mapache migrate.
• TUI repository format indicator: The dashboard top bar now shows the repository format version.
• Bundle performance: Speed up bundle writer and refactor archiver pipeline.
• Small-file performance: The archiver now chunks small files inline instead of round-tripping them through the chunker pool, and the chunker's stream buffer grows geometrically instead of allocating the full maximum size upfront. Small/medium sources use less memory and back up faster.
• Bundle open retry: Opening a bundle with the wrong password now prompts again instead of failing immediately.
• Copy and sync format enforcement: mapache copy and mapache sync now require source and destination repositories to use the same format version. Transferring between v1 and v2 repositories could silently mix incompatible storage layouts (sync) or leak v2-only features into v1 (copy).
• Reduced lock hold time in GC referenced blob scanning.
• mapache stats performance: File sizes are now derived from directory listings instead of one lstat per file, the objects directory is scanned once instead of twice, and the repository directories are listed concurrently. Snapshot analysis and --full pack footer parsing now run with bounded concurrency.
• mapache forget combines explicit ids with retention rules: Naming snapshots on the command line no longer discards the --keep-* rules (including those coming from the config file). Both are now applied together, with keep rules taking priority: a snapshot named for removal that is still covered by a retention rule is kept, and a warning reports which rule saved it. The policy line lists the explicit selection as forget (N snapshots) alongside the retention rules. Previously the policy argument group was declared with a single member and therefore enforced nothing, so forget <id> --keep-last 1 was accepted and silently ignored the retention rules.
• mapache forget output: The keep and remove tables now include a Reason column naming the retention rules that kept each snapshot (last, daily, tags, keep-min, ...) or why it is removed. The active policy with its parameters is echoed above the tables, section headers show counts, dry runs are marked with a [DRY RUN] banner, and the summary points at mapache recall and mapache clean. The JSON output gains a top-level policy array and a reason field per entry.
• Snapshot tables: log and forget tables are rendered without extra cell padding, making them noticeably narrower.
• mapache amend accepts multiple snapshot IDs: Pass one or more snapshot IDs to amend them in a single invocation (e.g. mapache amend <ID_1> <ID_2>). Each selected snapshot is rewritten and reported with a progress counter when more than one is selected. The original latest default and --all flag are unchanged.
• Keyfile format: Key files now use a nested kdf object with an algorithm discriminator (e.g. {"algorithm": "argon2id", "m": ..., "t": ..., "p": ...}) instead of flat top-level m, t, p fields. Old v1 keyfiles are read transparently by constructing the kdf object from the flat fields.
• CLI output polish: mapache log aligns snapshot metadata labels, dims the description, and ends with a styled active/dropped summary. mapache find prints a search header, per-snapshot match headers with short ID and timestamp, and a match summary (or No matches found). mapache cache list gains a table header with Files/Modified/Size columns and a summary row, and cache clean shows per-package progress and a success summary. mapache recall now confirms the recalled snapshot with its ID and timestamp.

Fixed

• rechunk snapshot loss on no-op rewrites: Running rechunk when the rewrite produces unchanged content (e.g. a second run with the same settings) deleted the snapshot it had just saved. Because snapshot IDs are content-addressed, the rewritten snapshot gets the same ID as the original, so saving the new file and then deleting the old one removed the only copy, leaving the repository with zero snapshots. The old snapshot is now only deleted when its ID differs from the newly saved one.
• mapache stats accounting: Pack counts and sizes no longer include .ecc, .tmp and .dropped files, and ECC bytes are no longer added twice to the reported total repository size.
• Config precedence: CLI flags now correctly take precedence over config file values. Previously any value present in the config silently overrode the explicit command-line flag (e.g. config keep_last=1 won over --keep-last 30).
• Restore metadata safety: With --strategy skip or --strategy newer, files kept locally no longer get their metadata (mtime, permissions, xattrs) silently overwritten by the snapshot's metadata. Newly restored files and directories still receive the snapshot's metadata.
• Copy integrity: mapache copy now aborts (non-zero exit) when a source tree cannot be read, instead of writing a destination snapshot that silently references data which was never copied.
• rebuild-index safety: When any pack fails to parse, rebuild-index aborts and leaves the existing index untouched instead of persisting a partial index and deleting the original (which would orphan blobs). It also now holds an exclusive lock.
• Blob verification: Every blob read path (restore, cat/mount, diff, copy, GC, bundle extraction) now verifies that the decoded content matches its content ID, catching same-length blob swaps that pass AEAD decryption.
• Keyfile hardening: Argon2id parameters read from keyfiles are validated against sane bounds before any derivation, and keyfile decompression is capped at 64 MiB, preventing memory/CPU DoS via a planted malicious keyfile. Bundle headers get the same Argon2 parameter bounds check.
• Data safety: Skip files with missing blobs instead of silently corrupting them via offset drift. Validate decoded blob lengths against the index.
• sync lock correctness: mapache sync now aborts when the destination is an existing repository whose lock cannot be acquired, instead of silently continuing without a destination lock and degrading the copy. The unlocked fallback is only used for fresh (bootstrapping) destinations.
• copy destination lock: mapache copy now acquires a lock on the destination repository by default (honoring --no-lock).
• Restore symlink strategy: Restore now honors --strategy for symlinks: skip keeps a conflicting symlink with a warning, overwrite replaces it, and fail aborts — instead of always replacing or always failing regardless of the chosen strategy.
• restore --delete with --include: The delete pass now strips the same leading path prefix as the restore pass. Previously, deleting with --include looked for local directories under the full snapshot path.
• Lock staleness: A lock left by a process that is still running on the same host is no longer treated as stale merely because it exceeded the lock age. Locks are only reclaimed by age when the owning process cannot be confirmed alive (other or unknown host).
• Skipped-file snapshot completeness: Snapshots no longer abort with "root tree ID not set" when files are skipped mid-run (e.g. an unreadable file or a diff-resolution failure). After the stream ends, remaining trees are force-finalized so the snapshot completes and omits the skipped items with a warning.
• Deleted nodes in snapshots: The per-node progress counters count deleted nodes again; they had been silently dropped from the item counts.
• Stat-failure visibility: Files that fail to stat during directory scanning are now surfaced as visible warnings in snapshot and bundle output instead of being silently omitted with only a debug log, so data loss from permission errors is not silent.
• verify error handling: mapache verify now errors out on failures while listing index/snapshot metadata files and no longer discards per-snapshot corruption-impact traversal results (reported, and honored by --fail-early), instead of silently continuing as if everything was verified.
• migrate / footer robustness: Pack footer parsing and v1→v2 migration now detect out-of-bounds blob descriptors and u32 offset overflow and return an error instead of panicking or wrapping on a malformed footer or a data section over 4 GiB.
• Migration preserves dropped snapshots: migrate now moves superseded objects to the .dropped trash instead of deleting them directly, so dropped snapshots — and their ECC sidecars — survive the rewrite. GC reclaims the trash afterwards.
• Bundle writer bounds: The ECC, index, and manifest section lengths written to the bundle trailer are now validated with u32::try_from instead of silently truncating any section larger than 4 GiB.
• ECC crash-safety: ecc set-percent regenerates sidecars before updating the manifest so the repository stays protected throughout, and disable resets the config before deleting obsolete sidecars.
• Bundle ECC repair safety: Repairing a corrupted bundle writes the repaired output to a .repaired file instead of overwriting the original, which is left untouched.
• ECC repair integrity: After repairing a corrupted bundle, the repaired shards are CRC-validated and the repair is refused if the check fails, instead of writing repaired (possibly wrong) data anyway. Partial-shard CRC mismatches are now caught.
• Deterministic blob nonce: Blob encryption nonces are now derived from the content hash instead of an independent random value, so the 96-bit GCM-SIV nonce can only repeat for identical content (which is safe for GCM-SIV), eliminating random collision risk at scale.
• key destructive confirmation: key delete now requires --yes (or an interactive confirmation) before proceeding.
• forget explicit-target filters: An explicitly named --forget snapshot that is excluded by the --host/--tags filters now errors out instead of silently forgetting nothing.
• amend informative errors: amend now reports that the snapshot was not found instead of claiming the operation was "interrupted by user".
• FUSE read performance: Mounted file reads now resolve blob lengths from the index and binary-search the first intersecting blob, instead of loading and decrypting every blob from the start of the file on each syscall.
• Packer worker pool: Workers that stop early after another worker reports an error now return their packer to the empty pool instead of dropping it, keeping the pool stable and preventing the pack-saver loop from hanging on a starved pool when a pack fails.
• gc delete failures: gc now counts per-object deletion failures, logs and emits user-visible errors for each one, and fails the run when any object could not be deleted instead of logging "Deleted N" as if everything succeeded.
• Incremental metadata: Record chmod/chown/xattr/flag changes that were silently lost when the Unchanged path overwrote fresh metadata.
• Crash-safe renames: Fsync the parent directory after every rename so directory entries survive power loss. Temp files now append .tmp instead of replacing the extension, preventing sibling collisions.
• Pre-1970 timestamps: Preserve negative timestamps instead of clamping to epoch, fixing silent metadata loss on old files.
• Bundle validation: Reject blob sizes and trailer fields that overflow u32 instead of silently truncating.
• Robust index/bundle parsing: Parser reads no longer preallocate capacity from untrusted header counts before validating them, preventing crafted corrupt files from triggering giant allocations or corrupting headers.
• Bundle cleanup on interrupt/error: Delete the output file when a bundle creation is interrupted by Ctrl+C or fails after the file has been created. Also delay opening the bundle writer until after all upfront checks pass.
• Verify exit codes: Detect corrupt metadata files (exit code 23) and preserve per-error exit codes in clean.
• Distinct exit codes: Every error variant now maps to its own exit code instead of collapsing to generic 1, so scripts can distinguish failure categories (e.g. snapshot not found, repository error, I/O error).
• Symlink restore safety: Symlink subtrees no longer emit during tree walk, preventing writes through a symlink from escaping the target root.
• Cache race: Fix missed wakeups in download coalescing; failed downloads now fail fast for all waiters.
• SFTP/S3: Non-blocking connection acquisition; stop retrying permanent HTTP 4xx errors. Rate limiter now throttles reads in chunks.
• Windows: expand ~ when HOME is not set.
• GC small-pack repack: clean no longer merges a single small data pack with a single small tree pack. The pack saver keeps data and tree blobs in separate packers, so repacking one of each type produces the same number of packs with zero savings. Only small packs of the same blob type are merged when there are 2 or more. The "Action" line no longer claims to repack packs that will not be repacked, and the "Packs:" line shows 0 small when no merge is possible.
• find interrupt handling: Pressing Ctrl+C during mapache find now stops the search and exits with code 130 instead of continuing to scan.
• stats interrupt handling: Pressing Ctrl+C during mapache stats now stops snapshot analysis (and --full pack-footer parsing) and exits with code 130 instead of continuing to scan.

---

Full Changelog: v0.6.0...v0.7.0