v2.1.0

Release Overview for Hauler v2.1.0

Comparison base: v2.0.3 → v2.1.0

v2.1.0 is a minor release built on the containerd-native foundation laid down in v2.0.0. The headline is that hauler store sync and hauler store add are now fully concurrent, with signature verification pinned to digests and running in parallel too. Alongside that, this release adds an audit trail, a store integrity checker, store-to-manifest generation, private/insecure registry support across every pull path, and a set of fixes for containerd imports, chunked hauls, and Docker Hub reference handling.

---

⚡ Concurrent Pulls - PR #711

"feat: async pulls"

Previously every image, file, and chart was pulled one at a time, and signature verification ran as a serial pass before any bytes moved. v2.1.0 fetches concurrently end to end.

---

What Changed?

• The OCI layout store is now safe for concurrent writes. Blob writes are deduplicated and bounded by a semaphore, index.json fsyncs are coalesced instead of issued per descriptor, and pkg/retry is cancellable so one failure aborts the run instead of sleeping out its budget.
• Images, files, and charts fetch under a bounded worker pool. --concurrency / -j (default 5, or HAULER_CONCURRENCY) controls artifact-level parallelism; --blob-concurrency (or HAULER_BLOB_CONCURRENCY) bounds blob-write fan-out independently and is auto-derived when unset. Chart dependency trees parallelize per level, including images discovered inside charts.
• Live progress display. A per-artifact progress region on a TTY, automatically suppressed when output is redirected or at debug log level. --no-progress opts out.
• Verification is digest-pinned and concurrent. Each image resolves to a digest once; that digest is verified and that digest is stored, closing the window where a tag could move between check and pull. pkg/cosign moved from cosign's CLI entrypoint to its library API, so verification no longer serializes on a process-global stdout swap (~1.1s per image, ~16 minutes on an 880-image manifest before the first byte moved).

---

Why It Matters?

• Large hauls are dramatically faster. Multi-hundred-image product manifests are bottlenecked by network and disk now, not by a serial loop.
• What you verified is what you stored. Pinning the digest through resolve → verify → store makes the stored bytes provably the verified bytes.

---

✨ New Features

PR	Feature	What it does
#632 / #727	Audit logging	Every store add, store remove, store save, and store load writes an audit entry to ~/.hauler/audit.log, plus a portable subset to <store>/audit.log (no hostname, IP, or paths) that travels safely inside a haul. Adds a persistent store-id (<store>/hauler.json), a per-invocation audit-id, a ~/.hauler/stores.json inventory, and --audit-level (none / standard / verbose, or HAULER_AUDIT_LEVEL). --store / -s now also accepts a store-id or unambiguous prefix in place of a path.
#699	Store integrity check	hauler store info --check walks every artifact's descriptor graph and validates each blob's existence, size, and digest, reporting missing, size-mismatch, digest-mismatch, or unreadable in a failure table (or a problems field with -o json). A corrupt manifest no longer hides the rest of the store.
#695 / #694 / #757	Generate a manifest from a store	hauler store create manifest writes a hauler-manifest.yaml that replicates the contents of an existing store, honoring --platform and --rewrite. Backed by a new hauler.dev/original-ref annotation that preserves each artifact's original reference regardless of rewrites (refs are normalized to containerd's canonical form when compared, so a cosmetic Docker Hub difference like index.docker.io vs docker.io isn't treated as a rewrite - #757).
#714	Private and insecure registries & fileservers everywhere	--ca-file and --insecure-skip-tls-verify (plus the matching manifest fields and hauler.dev/* annotations) are now supported for pulling images, charts, and files from registries & fileservers with self-signed or untrusted certificates.
#723	Per-manifest store and retries	New hauler.dev/store annotation routes a manifest document's content to a different store than --store, and hauler.dev/retries overrides the retry count. Adds HAULER_RETRIES, and centralizes --tempdir / HAULER_TEMP_DIR resolution.
#657	Platform-aware chart image discovery	hauler store add chart --add-images / Charts manifests now honor --platform, so images discovered inside a chart are pulled for a single platform instead of every architecture.
#702	Remote chunked hauls	hauler store load can load chunked hauls from remote URLs, passing each chunk as its own -f.
#737	--work-dir for sync	hauler store sync --products extracts product manifests into --work-dir (or HAULER_WORK_DIR) instead of the current directory, so it works from read-only or unwritable CWDs.

Dev tooling: #731 re-enabled the store serve TLS integration tests and added store copy integration tests; #733 tidied --help example formatting.

---

🐛 Bug Fixes

PR	Fix
#746 / #744	Digest-pinned multi-arch images survive a containerd import. Restores the oci-layout marker in saved hauls and v1's docker.io/... name normalization (both v2 regressions), and derives manifest.json synthetic tags from the parent index digest. --containerd hauls now ship an images-only index.json alongside a full hauler-index.json sidecar that store load prefers. store save no longer modifies the store directory while packaging, and archives are byte-reproducible again.
#761	Same-repo digest adds no longer overwrite each other. Digest-form refs were keyed by repo alone, so adding repo@sha256:AAA then repo@sha256:BBB silently replaced the first entry in index.json — sigs, atts, and SBOMs included. Digest refs are now keyed by their full repo@digest, and repo:tag@sha256:... keeps the tag as the entry name instead of dropping it (--rewrite inherits it). See the downgrade caveat in Breaking Changes.
#702	Chunked haul naming. Chunks are now <haul>.tar.zst.001 instead of <haul>_N.tar.zst, which mis-detected names like airgapped_docs_0.1.8_... as chunks and misplaced the index in names with dots. Old-format chunks still load when unambiguous; collisions and trailing empty chunks are handled.
#734	Consistent flag precedence in store sync. CLI flag > per-item field > manifest annotation, everywhere. Also fixes store add image --insecure-skip-tls-verify (was declared but never bound, so it did nothing) and the dead chart hauler.dev/ca-file annotation.
#755	Consistent reference parsing. All reference parsing now goes through a single reference.ParseReference helper that keeps registry-less references registry-less instead of silently defaulting them to index.docker.io/library/... - so charts, rewrites, and store create manifest no longer pick up a Docker Hub prefix the user never specified.
#719	--rewrite on Docker Hub library images. store info no longer re-normalizes index.docker.io/library/... over a rewritten name, and library/ is kept when explicitly given in the rewrite string (images and charts).
#722	store serve registry --config honors http.debug. The pprof / expvar debug server and Prometheus metrics endpoint were silently ignored.
#727	Audit logs survive store save / store load - non-OCI files inside the .tar.zst were not being read back.
#737	Path-traversal guard rejected every filename when extracting to /.
#732	Fixed store add image logging output at --log-level debug.
#718	Container images. hauler base bumped to bci-base:16.1, hauler-debug moved from alpine:latest to bci-base:16.1, and fixed both images sharing the same name label.

---

⚠️ Breaking Changes ⚠️

• #711 - A signature verification failure now aborts the run. Previously the artifact was skipped and the run still succeeded. With --ignore-errors, Hauler warns and stores the unverified artifact anyway. --retries does not retry a failed verification, since the digest is pinned and every attempt would check identical bytes. Combining --key with any --certificate-* constraint is now rejected.
• #632 - hauler store info --output json changed shape from a bare array to {"store-path": ..., "store-id": ..., "artifacts": [...]}.
• #734 - Flag precedence changed. String flags (--platform, --key, --certificate-*) flipped from per-item-first to CLI-first, and --store / --retries now beat the corresponding per-item field or annotation. insecureSkipTLSVerify in manifests is now a plain bool. --ca-file and skipping TLS verification are mutually exclusive.
• #746 - Synthetic tags for digest-pinned multi-arch images in manifest.json changed from child-digest to parent-digest form (repo:sha256-<parent>-<os>-<arch>); scripts matching the old tags need updating. Default hauls now carry oci-layout, so ctr images import takes the OCI path - mixed-content hauls destined for direct containerd import should use --containerd.
• #702 - Chunk file naming changed to <haul>.tar.zst.NNN (see Bug Fixes).
• #761 - Stores containing multiple same-repo digest entries (now possible with the same-repo digest fix — see Bug Fixes) are not safely writable by older hauler versions — their loader still collapses such entries at read time, and a subsequent write would persist the loss.

---

🔧 Maintenance & Dependencies 🔧

• Quality / CI: pkg/artifacts cleanup (#728), integration test coverage for store copy and TLS store serve (#731), vulnerability fixes (#738), CLI help formatting (#733).
• Notable dependency bumps: Go → 1.26.6; containerd/v2 → 2.3.4; sigstore/cosign/v3 → 3.1.3; sigstore/sigstore → 1.10.9; go-containerregistry → 0.22.0; helm/v4 → 4.2.4; k8s.io libs (apimachinery, api, client-go) → 0.37.0; logrus → 1.10.2; docker/go-metrics → 0.1.0; sigstore/rekor → 1.5.4 (CVE resolution); plus x/mod and go-isatty.

📒 Full Changelog 📒

• v2.0.3...v2.1.0

Contributors

@zackbradys
@amartin120
@CamrynCarter
@atoy3731
@eklatzer
@dependabot[bot]