Cram v1.3.0

Changed

• The vendored sevenz-rust2 fork is gone. Cram depended on sevenz-rust2-cram, a published
  copy of upstream 0.21.3 carrying two additions it needed: preparing a solid block off the writer
  thread, and an accessor for a coder's properties. Both were offered upstream, reviewed, and
  merged; upstream shipped them in 0.22.0. Cram now depends on plain sevenz-rust2 0.22 and the
  fork is retired.

  Dropping four upstream releases in one step also picks up fixes the fork never had. The one worth
  naming: coder stream counts read from a 7z block header are attacker-controlled and were summed
  without an overflow check, so a malformed archive could wrap past the bound that limits them.
  Cram reads archives it did not create, so this was reachable. Upstream fixed it in 0.21.4;
  the fork was pinned below that. Also included are a PPMd memory-limit check that compared bytes
  against a kilobyte limit (0.22.1), and batched AES-CBC decryption for encrypted .7z (0.21.5).

• lzma-rust2 0.16 → 0.20 and lz4_flex 0.13 → 0.14, which is what keeps one copy of each in the
  build now that sevenz-rust2 0.22 has moved. cram-extract was pinned to lzma-rust2 0.16
  separately and is bumped with it, so the standalone decoder and cram no longer ship different
  versions of the same XZ decoder.

No flag changed, no .cram archive reads differently, and nothing in the CLI moved.

Full Changelog: v1.2.0...v1.3.0

Cram v1.2.0

Full Changelog: v1.1.0...v1.2.0

Cram v1.1.0

Mostly work on finding duplicates, and on what you can see and do with the result. The engine gains
progress reporting and a verification pass; Cram Studio gains a gallery, a full-size viewer and the
ability to delete. As with every release here, the Cram Studio installer ships as an asset and is a
separate proprietary product under its own EULA.

Added

• A duplicate scan reports what it has found while it is still running. A scan over a whole
  drive spends minutes walking before it can group anything, and until now it said nothing for all
  of it, which is indistinguishable from being hung. Progress::on_scan_progress(files, dirs) is
  called at most every 250 ms, and Studio shows the running counts.

• Photos and video open in their own tab in Cram Studio, as a gallery. A duplicate set of
  images cannot be judged from a list of paths, so image and video sets now go to a separate tab
  with thumbnails, per-file or whole-row selection, and identical and look-alike kept on separate
  panes. Everything that is not an image or a video stays where it was: a thumbnail of a .dll
  tells you nothing.

• A full-size preview behind every thumbnail, from the magnifier on hover or a double-click.
  The arrows step through the set with both neighbours prefetched, and the image is contained in a
  fixed stage so two near-identical shots land in the same place and only the difference moves.
  Where the file is small enough and the format is one the view can draw directly, the original
  bytes are shown rather than a re-encode, so nothing you are judging is an artefact of the
  preview; when it does have to downscale, it says so.

• Duplicates can be deleted from Cram Studio, to the Recycle Bin. Deliberately not a button you
  can hit in passing: it has to be held, it says how many and how much, and everything goes to the
  Recycle Bin so a mistake is recoverable from Explorer. Selecting every copy in a set raises a
  warning rather than being forbidden.

• The hard-link option explains itself, behind a ? beside it, and a duplicate can be shown in
  its folder rather than only named.

Fixed

• A look-alike group could swallow hundreds of unrelated images. One scan put 936 different
  terminal screenshots in a single group. Grouping was by perceptual hash alone, unioned, which is
  single-linkage clustering: A joins B and B joins C, so one bridging pair merges two sets that
  resemble each other not at all. Tightening the hash threshold could not have fixed it, because
  single-linkage always finds a bridge. A candidate pair is now verified against the pixels — same
  aspect ratio within 10%, and a mean absolute difference no greater than 0.007 over a 64-pixel
  colour render. Colour matters here: discarding chroma is what makes a hash robust, and it is
  also what makes two unrelated dark terminals look identical. The threshold sits between the
  noisiest thing that must stay together (a photo resized and re-encoded at JPEG q40, 0.0037) and
  the closest thing that must separate (two different terminals, 0.0132) — about 1.9x from each. A
  retake of the same terminal with one word changed scores 0.0009 and stays grouped. Those are
  synthetic images, so treat the margin as real but not generous.

• A cancelled extraction took back more than it wrote. It now removes only files and directories
  that did not exist when it created them, so cancelling an extraction into a folder that already
  held a file of the same name leaves that file alone.

• A directory cycle walked forever. Past 1,000 levels the walk checks file identity and stops
  when it reaches somewhere it has already been. Legitimately deep trees are unaffected, which has
  a test.

• Thumbnails were drawn at roughly a quarter of the resolution they were displayed at. The
  request was sized in CSS pixels rather than device pixels, the result was then cropped to fill and
  upscaled again, the fast integer resampler aliased text, and JPEG's default quality rang around
  every glyph edge. Now: sized from the display's pixel ratio, Lanczos3, quality 90, and the whole
  image shown rather than a crop. Against a perfect render at the display size, 12.4x closer for
  4.1x the bytes.

• Installing a new version over an old one failed with Cram Studio closed. cram shell install
  registers cram_shell.dll as an in-process context-menu handler, so the first right-click after
  an install maps it into explorer.exe, which does not let go; the installer then could not open
  it for writing. Explorer is not something an installer can close, and the check for a running app
  looks for Studio rather than Explorer. A mapped file can still be renamed, so the installer now
  moves the old one aside and writes the new one over the name it vacated. Explorer keeps serving
  right-clicks from the previous version until it next restarts, which is a better outcome than an
  install that will not proceed.

• The browser hand-off left Firefox's own copy of every download on disk. The add-on called
  downloads.erase(), which removes the entry from Firefox's history and never touches the file.
  Deleting the bytes is downloads.removeFile(), which was not called at all. The leftover file
  also explains the file(1).type names: Cram found its own destination taken and uniquified around
  it. The add-on updates separately from Cram itself: install cram-handoff-v1.1.0.xpi below. It
  will not update on its own, because the add-on is unlisted and carries no update URL.

Cram v1.0.2

Skips 1.0.1, which the CLI already used for a crates.io-only release. cram --version and Studio's
updater both compare their own version against the release tag, so the two have to agree, and every
crate here is on 1.0.2 rather than leaving the CLI a version ahead of everything else.

Fixed

• A directory tree deeper than about 640 levels killed the process. Both tree walks recursed.
  Each frame of the duplicate-scan walk costs 3,264 bytes on the shipped Windows binary, so a scan
  running on a 2 MiB worker thread ran out of stack and died with 0xc00000fd. It died silently: a
  stack overflow is a hardware exception rather than a Rust panic, so nothing unwinds, no error is
  reported and no diagnostic is written — cram dedup and Cram Studio both simply disappeared.
  Found by scanning a drive that happened to hold a 14,566-level tree. Both walks now carry their
  own stack, so depth is bounded by memory rather than by the thread it runs on. cram a had the
  same defect on the create side and is fixed with it; archive member order is unchanged, and now
  has a test that says so.

Added

• A checkpoint that outlives a run which dies without unwinding. The event log lives in memory
  and dies with the process, so a crash left nothing at all to read, and the duplicate-scan engine
  recorded nothing even with detailed diagnostics switched on. A running operation now mirrors its
  operation, phase, item count and current depth to a file once a second, and deletes it on a clean
  finish — so a file left behind is itself evidence that the run did not finish, and says where it
  had got to. cram diag report adopts it into the next report. Cost is one atomic load per item,
  and a checkpoint another process is still rewriting is left alone.

Cram v1.0.0

The first public release of the Cram engine and command line. Everything below is new, so there is
nothing to list as changed or fixed.

The release will also carry the Cram Studio installer as an asset. Studio is a separate,
proprietary product under its own EULA; the MIT OR Apache-2.0 licence covers the engine and CLI in
this repository and not that installer.

Added

The cram CLI, one command for the whole lifecycle: l (list), x (extract), a (create),
t (test), conv (convert), dedup (find duplicate files), mount, rec (recovery sidecar),
sign / verify / keygen, make-sfx, and dl (segmented download, behind the opt-in download
feature). Free and open
source under MIT OR Apache-2.0.

Formats. Reads ZIP, 7z, tar (+ gzip / xz / zstd / bz2 / lz4 / brotli), ISO 9660, RAR, bare
single-stream compressed files (foo.gz, foo.xz, …), and Cram's own .cram. Writes ZIP, 7z, tar
(+ the same codecs), and .cram. RAR is read-only, creating RAR is forbidden by the UnRAR
licence and never will be supported.

The .cram format. Content-defined chunking (FastCDC) → BLAKE3-keyed global
dedup with no dictionary-window limit → compressed packs → a footer index. An archive is v1 unless
it uses a per-entry transform (see JPEG recompression below), in which case it declares v2 and a
v1-only reader refuses it rather than misreading it. Optional Argon2id +
AES-256-GCM encryption, when a password is set the footer index is sealed along with the packs, so
the file listing is hidden as well as the contents, and byte-for-byte reproducible when unencrypted.
Specified normatively in docs/CRAM_FORMAT.md.

Three effort levels, --fast, --auto (the default) and --small, plus --store for an
uncompressed archive that is still deduplicated. --small is the far end: the widest pack the format
allows, LZMA's extreme match search, and a per-pack search over pre-filters and coder parameters,
keeping whichever came out smallest. That search is worth its cost because the answer is
content-dependent, the x86 BCJ filter takes Silesia's ooffice down 14.1% and makes mozilla 0.9%
larger, so it can only ever be a candidate. Measured on Silesia, --small is 2.5% smaller than the
same archive built without the search, and smaller than xz -9e with a 256 MiB dictionary. Nothing
about it reaches the reader: an xz block header carries its own filter chain, so a --small archive
is read by any Cram build.

--store is not the fast option, despite compressing nothing. Measured on a 94,778-file tree it
ties --fast on create while writing 3.4x the bytes, then extracts 2.6x slower carrying them back.
What it is for is reading part of an archive without decompressing anything.

cram dedup, find duplicate files across folders and drives, without archiving anything. A file
whose size is unique cannot have a byte-identical twin, so it is never read; same-size files are
separated by a partial hash of their first and last 64 KiB; only what survives both is read in full
and confirmed with BLAKE3. Reads are scheduled per drive, every volume at once, but one sequential
reader on a spinning disk and several on an SSD, since parallel reads make an HDD slower. Hard links
are counted as one physical file, so reclaimable space is not overstated.

By default it only reports. --link replaces duplicates with hard links (every filename and folder
stays where it is), --quarantine <dir> moves them aside instead, and both preview unless --apply
is given. Nothing is ever deleted. Each pair is re-hashed at the moment of action, so a plan made
earlier cannot act on a file that has changed since.

--similar additionally flags images that look alike without being byte-identical (a resize, a
re-save). These are reported separately, are never counted as reclaimable, and no action can consume
them: a perceptual hash cannot tell a redundant re-encode from two different frames of a burst.
Needs the phash feature.

Lossless JPEG recompression in .cram, on by default. A photo is already entropy-coded, so
general-purpose compressors gain roughly nothing on one; redoing that coding with a stronger coder
(Lepton) is worth about 23% while extraction reconstructs the original file byte-for-byte. Measured
on one folder of 34 phone photos (26.1 MB): ZIP and 7z both produced output fractionally larger
than the originals, tar.xz managed 2.7%, and .cram was 23.6% smaller with all 34 files
extracting byte-identical. That is a single sample rather than a benchmark. Every candidate is verified to round-trip before it is stored, and anything that
fails verification is stored untouched. cram a --no-recompress turns it off.

Linux and macOS support (x86_64-unknown-linux-gnu, aarch64-apple-darwin), built and tested
alongside Windows, plus an install.sh that fetches the right binary for either. See Known
limitations for what still differs between the three.

A second, independent .cram decoder. cram-extract.exe implements the same spec from the
document alone, shares no code with the engine, and takes five direct pure-Rust dependencies
(lzma-rust2, ruzstd, aes-gcm, argon2, lepton_jpeg). It contains no C or C++ code,
so unlike cram.exe it needs no DLL beside it beyond the OS's own. Your data stays recoverable even
if the main build is not available. It doubles as the make-sfx self-extractor stub.

Parallel extraction for the formats with a random-access interface (ZIP, ISO, .cram). The worker
count is derived from the destination drive, hardware auto-detect plus a one-shot calibration
cached in the per-user config directory (%APPDATA%\cram\profile.toml on Windows,
~/.config/cram/ on Linux, ~/Library/Application Support/cram/ on macOS). --skip leaves a destination file alone only when a
per-entry CRC proves it identical, so it helps on ZIP and 7z entries that store a CRC, and does
nothing on .cram, tar, RAR or ISO, or on a WinZip AES entry written in AE-2 form, which stores no
CRC and is proven by its AES authentication instead.

Damage is contained per entry. A damaged or truncated archive does not abort the job: intact
entries are extracted, every failure is reported by name, and the command exits non-zero. A partial
extraction can never report itself as a clean one.

Encryption on create and extract: AES-256 for ZIP and 7z, AES-256-GCM for .cram. Hiding the
file listing as well as the contents: .cram always does it when a password is set, 7z does it on
--encrypt-names, and ZIP cannot; ZIP encrypts contents but leaves the central-directory names in
the clear, so --encrypt-names on a ZIP is refused rather than silently ignored.

Integrity and repair, on any file; not just archives.

• cram sign / cram verify, detached ed25519 signatures (.cramsig), with --key to pin a
  required signer. The hash is streamed, so file size does not matter.
• cram rec, Reed-Solomon parity sidecars (.cramrec) that verify and repair bit-rot or
  truncation. This one works on the file in memory: creating a sidecar reads the whole file in,
  and verifying or repairing reads in both the file and its sidecar, so allow for roughly twice the
  file size in RAM. Files above about 200 GiB are refused outright.

cram mount, browse an archive as a virtual folder through Windows ProjFS. ZIP, ISO and .cram
are served by byte range straight from disk; tar, 7z, RAR and bare compressed streams are decoded
into RAM up front and capped at 2 GiB. ProjFS is an optional Windows feature (Client-ProjFS, off by
default); the DLL is bound lazily at run time, so every other command works whether or not it is
enabled.

--writable makes the mount folder a persistent layer over the archive. ProjFS makes a mount
writable whether or not anyone asks — a modified placeholder becomes a full file and a deleted one a
tombstone, both on disk — so a read-only mount never prevented writes, it only discarded them along
with the folder. With --writable they are kept: the archive is the immutable base, the folder is
everything that diverged, and re-mounting resumes over it. A modified file wins over the archive's
copy, an untouched one still comes from the archive, and the .cram is never written to. Deleting
the folder resets to a pristine archive and is the only way, since ProjFS cannot un-tag a
virtualization root. Without the flag, a folder that has picked up files not in the archive is now
kept rather than deleted, which previously lost them silently.

--remember adds a mount to a list that cram mount --restore brings back after a reboot, in one
process holding all of them. --list shows it, --forget drops an entry without touching the
folder. Nothing is remembered unless asked for: the list starts empty and there is no setting that
turns auto-remount on for everything. Encrypted archives are refused, their password not being
something Cram will store. Cram Studio, when set to start with Windows, runs
--restore at boot and only at boot, in a detached cram.exe so the mounts outlive the Studio
window; opening Studio by hand re-mounts nothing.

cram shell, Cram on the Windows Explorer right-click menu. Extract here, extract to a
subfolder and test on an archive; add to a .cram or a .zip on anything else. A container
document (.docx, .jar, .epub) gets both sets, since it is legitimately both. Where
cram-studio.exe sits beside cram.exe, two more entries appear, "Open in Cram Studio" and
"Add to archive…", which open Studio rather than running a cram command. A COM
IContextMenu handler, the same mechanism WinRAR and 7-Zip use, registered under HKCU only so it
needs no elevation and changes nothing for other accounts. On Windows 11 it appears under "Show more
options". cram shell uninstall removes it and cram shell status reports what is registered.

cram update, replace the installation with ...