Skip to content

🧊 Coldkeep v1.10.3 — Packed Storage Metadata Integrity

Choose a tag to compare

View all tags
@franchoy franchoy released this 17 May 20:40
· 696 commits to main since this release
v1.10.3
175d6ef
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Coldkeep v1.10.3 — Packed Storage Metadata Integrity

Release Type

Stabilization / correctness and integrity hardening release.

Summary

Coldkeep v1.10.3 is a fixes-only release in the v1.10 Reliability Freeze, CI Hardening & Correctness Burn-down train.

This release hardens packed-storage metadata integrity and verification behavior. It ensures that packed-storage metadata is explicit, internally consistent, bounded, validated before trust, and covered by a deliberate corruption regression matrix.

This release does not introduce new product features. It does not start engine extraction, catalog abstraction, daemon/API/UI work, GC reachability corrections, restore/recovery rewrites, storage format redesign, CI workflow changes, or dependency changes.

Highlights

  • Added packed-storage metadata invariant documentation.
  • Added packed manifest/index relational validation.
  • Added packed metadata offset/length/bounds validation.
  • Added overflow-safe range validation.
  • Hardened packed payload hash/checksum validation.
  • Added malformed/truncated packed-container read-safety coverage.
  • Documented and regression-protected packed compatibility/version guardrails.
  • Consolidated the packed-storage corruption regression matrix.
  • Closed v1.10.3 tracker/matrix rows with proof.
  • Completed local validation, pre-release checklist execution, CI, and Codacy review before release.

Packed Storage Integrity Improvements

Manifest / Index Validation

v1.10.3 adds centralized relational validation for packed manifest/index metadata.

Validation now rejects:

  • storage blocks without chunk-block references,
  • conflicting per-block offsets,
  • conflicting duplicate block/chunk index entries.

This validation runs before payload-dependent verification in both:

  • VerifyRepository
  • VerifyRepositoryFast

Offset / Length / Bounds Validation

v1.10.3 adds overflow-safe packed range validation.

The new validation checks:

  • non-negative offsets,
  • non-negative lengths,
  • overflow-safe range bounds,
  • chunk-reference ranges extending beyond block plaintext size.

This prevents packed metadata from driving unsafe reads, invalid ranges, or wrapped integer arithmetic.

Hash / Checksum Consistency

v1.10.3 hardens packed payload integrity metadata.

Validation now covers:

  • compressed_hash byte length when present,
  • physical_hash byte length when present,
  • decoded chunk_hash requirements,
  • required non-empty SHA-256 digest metadata,
  • exact 64-character SHA-256 hex format,
  • valid hex decoding,
  • deterministic failure for malformed digest metadata,
  • deterministic payload mismatch failures.

Malformed Container Read Safety

v1.10.3 adds integration coverage proving malformed packed container states fail safely.

Covered cases include:

  • missing physical packed container files,
  • truncated packed container files,
  • no-silent-skip behavior,
  • valid packed-container sanity coverage.

No production change was required for this phase because existing FileContainer.ReadAt short-read enforcement and VerifyStoredBlock physical_missing wrapping were confirmed sufficient after earlier validation hardening.

Compatibility / Migration Guardrails

v1.10.3 documents and regression-protects current packed compatibility behavior.

Findings:

  • format_version != 1 is rejected deterministically as metadata_invalid,
  • invalid storage block codecs are blocked by schema-level CHECK constraints,
  • current packed format verifies successfully,
  • no migration engine was introduced,
  • no automatic metadata upgrade path was introduced,
  • no new version field was introduced.

Corruption Regression Matrix

v1.10.3 adds a consolidated packed-storage corruption regression matrix.

The matrix maps concrete tests/evidence to:

  • manifest/index corruption,
  • bounds corruption,
  • hash/checksum corruption,
  • malformed/truncated read safety,
  • compatibility guardrails,
  • valid current packed behavior.

Tracker / Matrix Closure

v1.10.3 closed the packed-storage tracker/matrix scope with explicit decisions.

Final matrix decisions:

  • CK-110-M035 → accepted
  • CK-110-M060 → deferred
  • CK-110-M061 → deferred

All 19 linked issue rows were updated with aligned status, decision, decision rationale, and closure proof inherited from the parent matrix row.

Closure artifacts:

  • docs/release/v1.10/v1.10.3-closure-report.md
  • docs/release/v1.10/v1.10.3-closure-summary.csv

Pre-PR Codacy / Static-Analysis Risk Pass

Before opening the pull request, an additional Codacy-risk/static-analysis pass was executed.

Resolved before PR:

  • high-confidence unchecked integer-conversion findings in packed verify paths,
  • uint64 -> int64 chunk-id conversion risks in verify_repository.go,
  • uint64 -> int conversion risk in verify_block_pipeline.go,
  • duplicate Markdown heading in the Phase 6 release notes.

Validation after these fixes included:

  • gofmt,
  • go vet ./...,
  • golangci-lint run ./...,
  • gosec -quiet ./internal/verify/...,
  • go test ./internal/verify -count=1,
  • integration test reruns,
  • serial full-suite validation.

Known residual note:

  • parallel go test ./... showed non-deterministic integration instability in the local environment during the pre-PR risk pass;
  • serial go test -p 1 ./... passed;
  • PR CI completed successfully before merge.

Validation

Phase-Level Validation

Across the v1.10.3 implementation phases, validation included:

  • targeted manifest/index tests,
  • targeted offset/length/bounds tests,
  • targeted hash/checksum tests,
  • targeted malformed/truncated read-safety tests,
  • targeted compatibility guardrail tests,
  • consolidated corruption regression matrix tests,
  • go vet ./...,
  • go test ./...,
  • go test -race ./....

Local Validation

Phase 9 recorded final local validation evidence.

Passed:

  • packed-storage targeted validation,
  • package-local packed validation,
  • go vet ./...,
  • go test ./...,
  • go test -race ./...,
  • govulncheck with no reachable vulnerabilities,
  • closure artifact validation,
  • applicable CI-equivalent local checks.

Documented deviation:

  • scripts/audit_ci.py was requested as a likely candidate but is not present in this repository, so it was recorded as skipped with rationale.

Pre-Release Checklist

The full active pre-release checklist flow completed green locally.

Executed and passed:

  • Steps 1–11 active release-gate flow,
  • Step 4 optional umbrella integration suite,
  • Steps 15–17 snapshot/retention release gates,
  • Step 18-style final sanity checks.

Coverage included:

  • DB bring-up,
  • CI-parity quality checks,
  • full matrix simulation,
  • bootstrap on/off checks,
  • doctor/verify/batch contracts,
  • physical-file contract drift/repair checks,
  • integration/adversarial/smoke flows,
  • snapshot lifecycle smoke checks,
  • adversarial snapshot/retention gates G14–G17,
  • manual snapshot lifecycle gate,
  • doctor --output json,
  • validation matrix audit.

Resolved during pre-release execution:

  • Step 2 formatting drift was remediated and rerun to green.
  • Step 3 strict v1.7 compatibility gate passed after building a v1.7.0 binary from tag v1.7.0 and running the gate on isolated DB coldkeep_v17_gate.
  • Step 17 restore destination collision was resolved by rerunning restore with fresh destination out_step17, then completing diff/delete/GC checks.

Final pre-release state:

  • active gates passed,
  • historical Steps 12–14 treated as archived/non-gating,
  • temporary restore artifacts removed,
  • working tree clean.

Pull Request Validation

The pull request completed with:

  • CI green,
  • Codacy review completed with no unresolved new-actionable issue before merge,
  • merge completed into main.

Compatibility Notes

This release intentionally makes verification stricter for invalid or corrupted packed-storage states.

Potentially affected cases:

  • packed metadata with inconsistent block/chunk relationships,
  • chunk-block ranges that exceed plaintext block size,
  • malformed or wrong-length hash metadata,
  • corrupted packed payload bytes,
  • truncated or missing physical packed container files,
  • unsupported packed format versions.

Valid current packed storage remains supported.

Documentation Added or Updated

v1.10.3 added or updated:

  • docs/release/v1.10/v1.10.3-phase0-baseline.md
  • docs/release/v1.10/v1.10.3-phase1-packed-metadata-invariants.md
  • docs/release/v1.10/v1.10.3-phase2-manifest-index-validation.md
  • docs/release/v1.10/v1.10.3-phase3-offset-length-bounds.md
  • docs/release/v1.10/v1.10.3-phase4-hash-checksum-consistency.md
  • docs/release/v1.10/v1.10.3-phase5-malformed-container-read-safety.md
  • docs/release/v1.10/v1.10.3-phase6-compatibility-guardrails.md
  • docs/release/v1.10/v1.10.3-phase7-corruption-regression-matrix.md
  • docs/release/v1.10/v1.10.3-closure-report.md
  • docs/release/v1.10/v1.10.3-closure-summary.csv
  • docs/release/v1.10/v1.10.3-local-validation.md
  • docs/release/v1.10/v1.10.3-pr-notes.md
  • docs/release/v1.10/v1.10.3-release-notes.md
  • docs/release/v1.10/v1.10.3-changelog.md
  • docs/release/v1.10/v1.10.3-phase-status.md
  • docs/release/v1.10/v1.10.3-checklist.md
  • root CHANGELOG.md

Out of Scope

The following remain intentionally deferred to later v1.10.x releases:

  • GC reachability corrections,
  • full restore/recovery rewrite,
  • Codacy/CI policy implementation,
  • critical-path coverage gates,
  • full filesystem abstraction,
  • full filesystem fault injection framework,
  • engine extraction,
  • catalog abstraction,
  • default database backend changes,
  • NAS/network/cloud expansion,
  • new product features.

Recommended Tagging Commands

Run from clean, updated main after merge:

git checkout main
git pull --ff-only
git status --short
git tag -a v1.10.3 -m "v1.10.3 — Packed Storage Metadata Integrity"
git push origin v1.10.3

Then publish the GitHub release using this document as the release body.

Release-Train Context

v1.10.3 belongs to the v1.10 stabilization train, whose purpose is to turn the v1.9 functionally complete system into a trust-complete baseline before engine extraction begins in v1.11.

The v1.10 train remains fixes-only: correctness, validation, recovery safety, scanner triage, CI improvement, and regression hardening only.

Assets 2
Loading
0 Join discussion