🧊 Coldkeep v1.10.3 — Packed Storage Metadata Integrity

[franchoy]

Coldkeep v1.10.3 — Packed Storage Metadata Integrity

Release Type

Stabilization / correctness and integrity hardening release.

Summary

Coldkeep v1.10.3 is a fixes-only release in the v1.10 Reliability Freeze, CI Hardening & Correctness Burn-down train.

This release hardens packed-storage metadata integrity and verification behavior. It ensures that packed-storage metadata is explicit, internally consistent, bounded, validated before trust, and covered by a deliberate corruption regression matrix.

This release does not introduce new product features. It does not start engine extraction, catalog abstraction, daemon/API/UI work, GC reachability corrections, restore/recovery rewrites, storage format redesign, CI workflow changes, or dependency changes.

---

Highlights

• Added packed-storage metadata invariant documentation.
• Added packed manifest/index relational validation.
• Added packed metadata offset/length/bounds validation.
• Added overflow-safe range validation.
• Hardened packed payload hash/checksum validation.
• Added malformed/truncated packed-container read-safety coverage.
• Documented and regression-protected packed compatibility/version guardrails.
• Consolidated the packed-storage corruption regression matrix.
• Closed v1.10.3 tracker/matrix rows with proof.
• Completed local validation, pre-release checklist execution, CI, and Codacy review before release.

---

Packed Storage Integrity Improvements

Manifest / Index Validation

v1.10.3 adds centralized relational validation for packed manifest/index metadata.

Validation now rejects:

• storage blocks without chunk-block references,
• conflicting per-block offsets,
• conflicting duplicate block/chunk index entries.

This validation runs before payload-dependent verification in both:

• VerifyRepository
• VerifyRepositoryFast

Offset / Length / Bounds Validation

v1.10.3 adds overflow-safe packed range validation.

The new validation checks:

• non-negative offsets,
• non-negative lengths,
• overflow-safe range bounds,
• chunk-reference ranges extending beyond block plaintext size.

This prevents packed metadata from driving unsafe reads, invalid ranges, or wrapped integer arithmetic.

Hash / Checksum Consistency

v1.10.3 hardens packed payload integrity metadata.

Validation now covers:

• compressed_hash byte length when present,
• physical_hash byte length when present,
• decoded chunk_hash requirements,
• required non-empty SHA-256 digest metadata,
• exact 64-character SHA-256 hex format,
• valid hex decoding,
• deterministic failure for malformed digest metadata,
• deterministic payload mismatch failures.

Malformed Container Read Safety

v1.10.3 adds integration coverage proving malformed packed container states fail safely.

Covered cases include:

• missing physical packed container files,
• truncated packed container files,
• no-silent-skip behavior,
• valid packed-container sanity coverage.

No production change was required for this phase because existing FileContainer.ReadAt short-read enforcement and VerifyStoredBlock physical_missing wrapping were confirmed sufficient after earlier validation hardening.

Compatibility / Migration Guardrails

v1.10.3 documents and regression-protects current packed compatibility behavior.

Findings:

• format_version != 1 is rejected deterministically as metadata_invalid,
• invalid storage block codecs are blocked by schema-level CHECK constraints,
• current packed format verifies successfully,
• no migration engine was introduced,
• no automatic metadata upgrade path was introduced,
• no new version field was introduced.

Corruption Regression Matrix

v1.10.3 adds a consolidated packed-storage corruption regression matrix.

The matrix maps concrete tests/evidence to:

• manifest/index corruption,
• bounds corruption,
• hash/checksum corruption,
• malformed/truncated read safety,
• compatibility guardrails,
• valid current packed behavior.

---

Tracker / Matrix Closure

v1.10.3 closed the packed-storage tracker/matrix scope with explicit decisions.

Final matrix decisions:

• CK-110-M035 → accepted
• CK-110-M060 → deferred
• CK-110-M061 → deferred

All 19 linked issue rows were updated with aligned status, decision, decision rationale, and closure proof inherited from the parent matrix row.

Closure artifacts:

• docs/release/v1.10/v1.10.3-closure-report.md
• docs/release/v1.10/v1.10.3-closure-summary.csv

---

Pre-PR Codacy / Static-Analysis Risk Pass

Before opening the pull request, an additional Codacy-risk/static-analysis pass was executed.

Resolved before PR:

• high-confidence unchecked integer-conversion findings in packed verify paths,
• uint64 -> int64 chunk-id conversion risks in verify_repository.go,
• uint64 -> int conversion risk in verify_block_pipeline.go,
• duplicate Markdown heading in the Phase 6 release notes.

Validation after these fixes included:

• gofmt,
• go vet ./...,
• golangci-lint run ./...,
• gosec -quiet ./internal/verify/...,
• go test ./internal/verify -count=1,
• integration test reruns,
• serial full-suite validation.

Known residual note:

• parallel go test ./... showed non-deterministic integration instability in the local environment during the pre-PR risk pass;
• serial go test -p 1 ./... passed;
• PR CI completed successfully before merge.

---

Validation

Phase-Level Validation

Across the v1.10.3 implementation phases, validation included:

• targeted manifest/index tests,
• targeted offset/length/bounds tests,
• targeted hash/checksum tests,
• targeted malformed/truncated read-safety tests,
• targeted compatibility guardrail tests,
• consolidated corruption regression matrix tests,
• go vet ./...,
• go test ./...,
• go test -race ./....

Local Validation

Phase 9 recorded final local validation evidence.

Passed:

• packed-storage targeted validation,
• package-local packed validation,
• go vet ./...,
• go test ./...,
• go test -race ./...,
• govulncheck with no reachable vulnerabilities,
• closure artifact validation,
• applicable CI-equivalent local checks.

Documented deviation:

• scripts/audit_ci.py was requested as a likely candidate but is not present in this repository, so it was recorded as skipped with rationale.

Pre-Release Checklist

The full active pre-release checklist flow completed green locally.

Executed and passed:

• Steps 1–11 active release-gate flow,
• Step 4 optional umbrella integration suite,
• Steps 15–17 snapshot/retention release gates,
• Step 18-style final sanity checks.

Coverage included:

• DB bring-up,
• CI-parity quality checks,
• full matrix simulation,
• bootstrap on/off checks,
• doctor/verify/batch contracts,
• physical-file contract drift/repair checks,
• integration/adversarial/smoke flows,
• snapshot lifecycle smoke checks,
• adversarial snapshot/retention gates G14–G17,
• manual snapshot lifecycle gate,
• doctor --output json,
• validation matrix audit.

Resolved during pre-release execution:

• Step 2 formatting drift was remediated and rerun to green.
• Step 3 strict v1.7 compatibility gate passed after building a v1.7.0 binary from tag v1.7.0 and running the gate on isolated DB coldkeep_v17_gate.
• Step 17 restore destination collision was resolved by rerunning restore with fresh destination out_step17, then completing diff/delete/GC checks.

Final pre-release state:

• active gates passed,
• historical Steps 12–14 treated as archived/non-gating,
• temporary restore artifacts removed,
• working tree clean.

Pull Request Validation

The pull request completed with:

• CI green,
• Codacy review completed with no unresolved new-actionable issue before merge,
• merge completed into main.

---

Compatibility Notes

This release intentionally makes verification stricter for invalid or corrupted packed-storage states.

Potentially affected cases:

• packed metadata with inconsistent block/chunk relationships,
• chunk-block ranges that exceed plaintext block size,
• malformed or wrong-length hash metadata,
• corrupted packed payload bytes,
• truncated or missing physical packed container files,
• unsupported packed format versions.

Valid current packed storage remains supported.

---

Documentation Added or Updated

v1.10.3 added or updated:

• docs/release/v1.10/v1.10.3-phase0-baseline.md
• docs/release/v1.10/v1.10.3-phase1-packed-metadata-invariants.md
• docs/release/v1.10/v1.10.3-phase2-manifest-index-validation.md
• docs/release/v1.10/v1.10.3-phase3-offset-length-bounds.md
• docs/release/v1.10/v1.10.3-phase4-hash-checksum-consistency.md
• docs/release/v1.10/v1.10.3-phase5-malformed-container-read-safety.md
• docs/release/v1.10/v1.10.3-phase6-compatibility-guardrails.md
• docs/release/v1.10/v1.10.3-phase7-corruption-regression-matrix.md
• docs/release/v1.10/v1.10.3-closure-report.md
• docs/release/v1.10/v1.10.3-closure-summary.csv
• docs/release/v1.10/v1.10.3-local-validation.md
• docs/release/v1.10/v1.10.3-pr-notes.md
• docs/release/v1.10/v1.10.3-release-notes.md
• docs/release/v1.10/v1.10.3-changelog.md
• docs/release/v1.10/v1.10.3-phase-status.md
• docs/release/v1.10/v1.10.3-checklist.md
• root CHANGELOG.md

---

Out of Scope

The following remain intentionally deferred to later v1.10.x releases:

• GC reachability corrections,
• full restore/recovery rewrite,
• Codacy/CI policy implementation,
• critical-path coverage gates,
• full filesystem abstraction,
• full filesystem fault injection framework,
• engine extraction,
• catalog abstraction,
• default database backend changes,
• NAS/network/cloud expansion,
• new product features.

---

Recommended Tagging Commands

Run from clean, updated main after merge:

git checkout main
git pull --ff-only
git status --short
git tag -a v1.10.3 -m "v1.10.3 — Packed Storage Metadata Integrity"
git push origin v1.10.3

Then publish the GitHub release using this document as the release body.

---

Release-Train Context

v1.10.3 belongs to the v1.10 stabilization train, whose purpose is to turn the v1.9 functionally complete system into a trust-complete baseline before engine extraction begins in v1.11.

The v1.10 train remains fixes-only: correctness, validation, recovery safety, scanner triage, CI improvement, and regression hardening only.

---

This discussion was created from the release 🧊 Coldkeep v1.10.3 — Packed Storage Metadata Integrity.