Skip to content

v0.3.163 - Object-Storage Upload Adapter (Stage 1)

Choose a tag to compare

@mow-coding mow-coding released this 03 Jul 17:16

v0.3.163 - Object-Storage Upload Adapter (Stage 1)

v0.3.163 lands Stage 1 of the WOM #11 live object-storage upload adapter as an
approval-gated WOM-kit CLI surface (not a provider MCP the AI drives). Stage 1
ships everything provable in a temp dir — the upload plan, digest-aware
idempotency, local RAW-byte verification, the execution-receipt and resume-ledger
shapes, the manifest wom_uploaded transition, and the load-bearing secret and
manifest-safety controls — while shipping NO live transport. The wire step
itself is a later, human-gated stage.

The no-network boundary (the load-bearing property)

This release is structurally incapable of a real provider call, and a reviewer
can confirm that in seconds:

  • No transport that performs a socket operation ships. The upload spine calls a
    provider only through an injected ObjectStorageTransport. Stage 1 ships the
    abstract interface plus a NullTransport whose every method raises, and no
    class that opens a socket. No boto3/httpx/requests import sits on the
    upload path, and wom-kit/pyproject.toml gains no dependency (still PyYAML).
  • object_storage_resolve_transport(provider_kind) returns null for every
    provider. There is no env var, flag, or branch in this release that resolves a
    live client.
  • archive object-storage-upload --approve fails closed with
    live_transport_not_implemented before any credential read and before any
    byte read.
  • live_object_upload_adapter_implemented and provider_api_call_implemented
    are both false in command output and the capability matrix.

The guarantee is compositional: even if a reviewer distrusts the fail-closed
blocker, there is literally no client object to call. Making a real PUT requires
a Stage-2 code change that adds an import and rewires the resolver — a diff that
cannot land silently.

Command surface

  • archive object-storage-upload-plan <archive-root> --provider-kind <kind> --store-ref <label> --access-key-id-ref env:<NAME> --secret-access-key-ref env:<NAME> [--only sha256:<hex>] [--max-objects N] [--skip-uploaded] --dry-run — read-only, writes nothing,
    reads no secret. Emits a per-object plan (object_id, size_bytes, the
    content-addressed key shape sha256/<first2>/<64hex>, and a digest-aware
    would_upload/already_uploaded verdict) and REFUSES at the service layer if
    the resolved plan exceeds --max-objects.
  • archive object-storage-upload-verify <archive-root> --dry-run — hashes each
    planned object's local RAW bytes with sha256 and asserts equality with the
    object id. It never uses the BOM/newline normalizer; upload verification is
    byte-exact.
  • archive object-storage-upload <archive-root> ... (--dry-run | --approve)
    the mutating command. --dry-run previews the plan and the execution-receipt
    shape with no provider call, no byte read, and no secret read. --approve
    fails closed in Stage 1. The three-way gate (reject both modes, reject neither,
    reject --approve without a safe --reviewed-by) is enforced at the CLI and
    re-enforced in the service layer.

Each command keeps the established object-storage-* family naming with the
objet-* aliases.

Idempotency and verification (digest-aware)

--skip-uploaded treats an object as already_uploaded only when a
provider-confirmed wom_uploaded manifest location exists whose key_hint
digest equals the object id — never on an external declared_uploaded evidence
record and never on a manifest-only hit without a remote HEAD. Layer A gates
cost; only a remote HEAD gates correctness. The audit and doctor enforce the
same digest invariant on every object_storage location, so even a hand-edited
manifest is caught. When a transport is injected (tests only), the per-object
spine runs a remote HEAD before upload, a single PUT below the 5 GiB multipart
threshold or a multipart upload above it (full-object sha256 verified — ETag is
never treated as the content hash), a remote HEAD after upload, and a crash-safe
append-only resume ledger that is authoritative for "this object's PUT already
succeeded."

Reliability: manifest-write hardening

The shared object-storage manifest writer now holds the same manifest lock the
objet-capture append path uses and writes via a temp+fsync+os.replace atomic
writer. Previously it ended in a bare write_text, so a crash mid-write or a
concurrent capture could corrupt the whole manifest. This fix is additive and
also hardens the existing external-upload-evidence command.

Secret and leak discipline

Both resolved key values are read only under --approve after every gate, held
in locals, and cleared in a finally block. Before any write, and on every exit
path, the fully-serialized output (receipt + ledger delta + manifest delta +
return payload) is checked by a direct-value containment guard against both key
values — the sound control that catches a bare 64-hex/40-char secret the named
regex scanners miss — backed by those scanners plus the forbidden-location
scanner as a backstop. Execution receipts and ledger rows are built from a fixed
scalar allowlist; no request headers, Authorization, StringToSign,
CanonicalRequest, or provider error body is ever recorded.

Also in this release

  • New doctor check _check_object_storage_execution_receipts validating
    object-storage-upload-receipt.schema.json, dry_run:false + non-empty
    reviewed_by on applied receipts, self-referential receipt_path, and the
    digest invariant.
  • Read-only MCP tools object_storage_upload_plan and
    object_storage_upload_verify. No upload write tool is exposed.

What is deliberately NOT in this release

The live PUT, real provider HEAD/checksum behavior, SigV4 signing, and cost
accounting. Those are a later, tiny-first, human-gated stage. This release
cannot upload to a provider.