Release LowPack 0.2.1 with safe archive migration

CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.2.1 - 2026-07-29
+
+- Add a read-only `lowpack compatibility` probe plus an authenticated, atomic
+  `lowpack migrate` command and typed Python APIs for format 1.0 archives
+  created by LowPack 0.1.x.
+- Preserve legacy chunk payloads while converting embedded dictionaries,
+  selection goals, and codec decisions into strict format 1.1 manifest schema
+  2.0 records.
+- Fully reconstruct and verify migrated archives before publishing the output,
+  leaving the source untouched and requiring explicit destination overwrite.
+- Separate end-user wheel installation from contributor setup, with a
+  cross-platform getting-started guide, checksum guidance, upgrade steps, and
+  compatibility reference.
+
 ## 0.2.0 - 2026-07-29
 
 - Harden input collection against duplicate names and identities, symlink

README.md

@@ -13,8 +13,8 @@ it will actually be stored and used, then hands it to proven lossless codecs.
 It is a terminal tool and Python library that stays completely on your laptop.
 
 > **A quick, honest alpha note:** the `.lpk` format is still experimental.
-> Archives from alpha releases may need migration, so please do not make
-> LowPack the only copy of important data.
+> LowPack 0.2.1 can migrate format 1.0 archives made by the 0.1 releases, but
+> please do not make any alpha archive the only copy of important data.
 
 Oh! One point I care about being clear on: LowPack does **not** universally
 outperform Zstandard, gzip, ZIP, Brotli, LZ4, or anything else. Results depend
@@ -24,16 +24,28 @@ indexing, deduplication, explainable selection, and reversible preparation.
 
 ## Install
 
-Python 3.9 through 3.14 is supported. For development, I usually start with a
-fresh environment so the checks describe the project instead of my machine:
+Python 3.9 through 3.14 is supported. To use LowPack today, install the wheel
+attached to the GitHub release:
 
 ```powershell
 python -m venv .venv
 .venv\Scripts\Activate.ps1
-python -m pip install -e ".[dev]"
+python -m pip install "https://github.com/devkyato/Lowpack/releases/download/v0.2.1/lowpack-0.2.1-py3-none-any.whl"
 lowpack --version
+lowpack doctor
 ```
 
+On macOS or Linux, activation is `source .venv/bin/activate`; the install
+command is otherwise the same. If I want the terminal command isolated from a
+project, I use `pipx install` with that wheel URL instead. Each release also
+includes `SHA256SUMS`, the source archive, and exact verification notes. The
+[getting-started guide](docs/getting-started.md) covers installation,
+upgrades, a first round trip, and common command-not-found fixes.
+
+For development, clone the repository and use
+`python -m pip install -e ".[dev]"` in a fresh environment. That keeps the
+published install path separate from contributor tooling.
+
 That is it. LowPack needs no account, website, cloud service, daemon, database,
 analytics, or background network access.
 
@@ -54,6 +66,20 @@ Selective extraction is `lowpack extract project.lpk project/src -o selected`.
 I made `--overwrite` explicit on purpose, and `lowpack doctor` is there when
 you want a quick check of the local environment.
 
+If an archive came from LowPack 0.1, migrate it without touching the original:
+
+```powershell
+lowpack compatibility old-project.lpk
+lowpack migrate old-project.lpk -o project-1.1.lpk
+lowpack verify project-1.1.lpk --full
+```
+
+Oh! On this part I thought the safest upgrade was the least surprising one:
+LowPack authenticates the old archive, rewrites only the framing and manifest,
+fully reconstructs and verifies the migrated temporary archive, and moves it
+into place only after all of that succeeds. See the
+[compatibility guide](docs/compatibility.md).
+
 Codec selection uses deterministic policy names: `balanced`, `smallest`,
 `prefer-store`, `prefer-zstd-low`, and `avoid-zlib`. They describe stable
 preferences rather than claiming to measure whole-machine speed or memory.
@@ -146,21 +172,33 @@ the identified corpus and environment.
 ## Python API
 
 ```python
-from lowpack import inspect_archive, pack, unpack, verify_archive
+from lowpack import (
+    inspect_archive,
+    migrate_archive,
+    pack,
+    probe_compatibility,
+    unpack,
+    verify_archive,
+)
 
 pack(["project"], "project.lpk", profile="source", goal="balanced")
 info = inspect_archive("project.lpk")
 assert verify_archive("project.lpk", full=True).valid
 unpack("project.lpk", output="restored")
+
+# For a 0.1 archive:
+probe_compatibility("old-project.lpk")
+migrate_archive("old-project.lpk", "project-1.1.lpk")
 ```
 
 Functions return typed frozen result models.
 
 ## Limitations
 
 This is where I would rather be specific than sound finished too early. The
-format has no pre-1.0 forward-compatibility promise. Version 0.2 deliberately
-introduces manifest schema 2 after the 0.1 security review. Source dictionaries
+format has no forward-compatibility promise during alpha. Version 0.2
+deliberately introduced manifest schema 2 after the 0.1 security review;
+0.2.1 provides a checked migration from format 1.0. Source dictionaries
 use bounded deterministic samples and only apply to Zstandard chunks.
 Telemetry canonical mode stores exact IEEE-754 values, but only exact mode
 preserves the original decimal spelling. Canonical transforms have a 64 MiB

docs/architecture.md

@@ -9,6 +9,10 @@ review.
 extraction. Its individual stages are deliberately isolated as helpers with
 one-way data flow; `format.py` handles fixed binary framing and `manifest.py`
 handles executable-free canonical JSON plus total JSON limits.
+`migration.py` is a deliberately narrow bridge from format 1.0 to 1.1: it
+authenticates the source, normalizes its manifest into schema 2, preserves the
+chunk payload area, and accepts the output only after current full
+verification.
 `codecs` exposes the experimental `Codec` protocol. `profiles` detects or
 reversibly prepares application data without owning compression. `selection`
 measures a bounded sample and stores all candidate measurements. The CLI is a

docs/compatibility.md

@@ -0,0 +1,61 @@
+# Archive compatibility and migration
+
+LowPack 0.2 writes binary format 1.1 with manifest schema 2.0. LowPack 0.1.x
+wrote format 1.0. I did not want a security-driven schema change to leave the
+earliest test archives stranded, so 0.2.1 adds one explicit bridge:
+
+```powershell
+lowpack compatibility old-project.lpk
+lowpack migrate old-project.lpk -o project-1.1.lpk
+lowpack verify project-1.1.lpk --full
+```
+
+The command never edits `old-project.lpk`. An existing output is refused unless
+you pass `--overwrite`.
+
+`lowpack compatibility` is the read-only half of this workflow. It checks
+framing, canonical manifest structure, the manifest hash, the complete body
+hash, and current schema relationships, then reports either `current` or
+`migration-available`. It does not decompress chunks, extract files, or write
+an output:
+
+```powershell
+lowpack compatibility old-project.lpk --json
+```
+
+## What the migration does
+
+Oh! On this part, “migration” does not mean extracting files somewhere and
+packing them again with a possibly different policy. LowPack:
+
+1. checks the 1.0 header, footer, canonical manifest hash, and complete body
+   hash;
+2. converts embedded compression dictionaries into the authenticated schema 2
+   catalog;
+3. maps the old selection goal names to their deterministic policy names and
+   records preferred-versus-actual chunk decisions;
+4. validates paths, sizes, offsets, dictionaries, transforms, permissions,
+   references, and payload boundaries against the current strict schema;
+5. preserves the original compressed chunk payload area;
+6. writes a sibling temporary 1.1 archive and fully decompresses,
+   reconstructs, and hashes every file; and
+7. atomically publishes the destination only after all checks pass.
+
+Stored permission values are reduced to ordinary rwx bits during migration.
+Restoring even those bits remains opt-in during extraction.
+
+## What it deliberately does not do
+
+Migration supports format 1.0 only. It does not guess at unknown future
+formats, repair a corrupt body, bypass current safety limits, or overwrite the
+source. If a 0.1 archive contains a relationship that the current validator
+cannot prove safe, migration fails without publishing a partial destination.
+
+Use `--json` when another local tool needs a stable result record:
+
+```powershell
+lowpack migrate old.lpk -o migrated.lpk --json
+```
+
+For the precise framing and schema, see the [format reference](format.md). For
+the trust model, see [extraction security](security.md).

docs/format.md

@@ -58,6 +58,12 @@ Every file records a preferred codec policy plus one `chunk_decisions` entry
 per reference. Those entries name the actual stored codec/dictionary and say
 whether content-addressed deduplication reused an earlier representation.
 
+Format 1.0, written by LowPack 0.1.x, used embedded per-chunk dictionary bytes
+and a less explicit codec-decision record. `lowpack migrate` is the supported
+bridge to format 1.1 and manifest schema 2.0. The
+[compatibility guide](compatibility.md) describes exactly what is preserved
+and checked.
+
 A zero-byte file has an empty chunk list and SHA-256
 `e3b0c44298fc1c149afbf4c8996fb924...`.
 

docs/getting-started.md

@@ -0,0 +1,81 @@
+# Getting started
+
+I wanted the first LowPack run to answer two questions quickly: “Did I install
+the real release?” and “Can I restore exactly what I packed?” This page does
+both without needing an account or network connection after installation.
+
+## Install the release
+
+LowPack supports Python 3.9 through 3.14. On Windows PowerShell:
+
+```powershell
+python -m venv .venv
+.venv\Scripts\Activate.ps1
+python -m pip install "https://github.com/devkyato/Lowpack/releases/download/v0.2.1/lowpack-0.2.1-py3-none-any.whl"
+lowpack --version
+lowpack doctor
+```
+
+On macOS or Linux:
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install "https://github.com/devkyato/Lowpack/releases/download/v0.2.1/lowpack-0.2.1-py3-none-any.whl"
+lowpack --version
+lowpack doctor
+```
+
+The release page includes the wheel, source archive, and `SHA256SUMS`. Download
+the wheel first if you want to compare its SHA-256 before installing it. I
+publish the exact expected digests in the release description as well.
+
+For a globally available but isolated terminal command, replace the virtual
+environment steps with `pipx install <wheel-url>`. LowPack itself does not
+contact GitHub or any other service while packing, inspecting, verifying, or
+extracting.
+
+## Make and restore an archive
+
+```powershell
+lowpack pack my-project -o my-project.lpk --profile source
+lowpack inspect my-project.lpk
+lowpack verify my-project.lpk --full
+lowpack unpack my-project.lpk -o restored
+```
+
+I thought too on the point that a successful pack should not be treated as a
+successful restore. `verify --full` decompresses each stored representation
+and checks reconstructed file hashes. For important data, still test the
+restored directory and keep another copy outside the archive.
+
+The `source` profile excludes documented cache/build paths. LowPack prints and
+records every exclusion; use `--include-all` when the directory should be
+literal. Use the `general` profile for an ordinary tree with no profile
+exclusions.
+
+## Upgrade or remove
+
+Upgrade to a newer release by activating the same environment and installing
+its wheel URL with `--upgrade`. Remove LowPack with:
+
+```powershell
+python -m pip uninstall lowpack
+```
+
+Uninstalling never removes `.lpk` files or extracted data.
+
+If you have a 0.1.x archive, install 0.2.1 and follow the
+[compatibility guide](compatibility.md). Installing a new LowPack version does
+not silently rewrite existing archives.
+
+## If `lowpack` is not found
+
+First run `python -m lowpack --version`. If that works, LowPack is installed
+for that interpreter and the environment's scripts directory is simply not on
+the current shell path. Activate the virtual environment again, or use
+`python -m lowpack` in place of `lowpack`.
+
+If neither command works, compare `python -m pip --version` and
+`python --version`; they should point to the environment where you intended to
+install LowPack.

docs/limitations.md

@@ -3,7 +3,9 @@
 I would rather keep this list direct than hide unfinished edges behind an
 “alpha” label. These are the boundaries I know about today:
 
-- Format compatibility is not promised across alpha releases.
+- Forward compatibility is not promised across alpha releases. LowPack 0.2.1
+  provides a checked migration for format 1.0 archives created by 0.1.x; it
+  does not guess at unknown formats or repair corrupted archives.
 - Chunk boundaries are fixed-size, not content-defined.
 - Source dictionary training is active, bounded, deterministic, and
   authenticated in the manifest, but it currently applies only to Zstandard
@@ -23,5 +25,5 @@ I would rather keep this list direct than hide unfinished edges behind an
 - General safety limits are constants in the 0.2 library API.
 
 I am tracking the larger follow-ups in the
-[v0.2.0 milestone](https://github.com/devkyato/Lowpack/milestone/1), so this
-page and the issue tracker should tell the same story.
+[issue tracker](https://github.com/devkyato/Lowpack/issues), so this page and
+the roadmap should tell the same story.

docs/security.md

@@ -34,3 +34,10 @@ privately owned output directory plus OS quotas or sandboxing.
 Archived permissions are ignored by default. `--restore-permissions` opts in
 to ordinary user/group/other rwx bits; setuid, setgid, and sticky bits are
 masked during packing and extraction.
+
+Migration uses the same trust boundary. `lowpack migrate` first authenticates
+the complete format 1.0 body, converts its manifest into the strict current
+schema, rejects unsafe paths and inconsistent relationships, and writes to a
+sibling temporary file. It then performs a full decompression and
+reconstruction check before atomic replacement. The source archive is never
+modified, and an existing destination still requires `--overwrite`.

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "lowpack"
-version = "0.2.0"
+version = "0.2.1"
 description = "Application-aware, lossless local-first packing built on proven codecs."
 readme = "README.md"
 requires-python = ">=3.9"

src/lowpack/__init__.py

@@ -1,16 +1,27 @@
 """LowPack public API."""
 
-__version__ = "0.2.0"
+__version__ = "0.2.1"
 
 from .archive import inspect_archive, pack, unpack, verify_archive
-from .models import ArchiveInfo, PackResult, VerificationResult
+from .migration import migrate_archive, probe_compatibility
+from .models import (
+    ArchiveInfo,
+    CompatibilityResult,
+    MigrationResult,
+    PackResult,
+    VerificationResult,
+)
 
 __all__ = [
     "ArchiveInfo",
+    "CompatibilityResult",
+    "MigrationResult",
     "PackResult",
     "VerificationResult",
     "inspect_archive",
+    "migrate_archive",
     "pack",
+    "probe_compatibility",
     "unpack",
     "verify_archive",
 ]