Skip to content

v1.4.1 — Marketplace E2E findings closed

Choose a tag to compare

View all tags
@masumi-ryugo masumi-ryugo released this 23 Jun 16:37
· 35 commits to main since this release
v1.4.1
3579e29

v1.4.1 closes four critical regressions surfaced running v1.4.0 on a live Marketplace AMI with AWS CLI 2.33 + real AWS S3. Every fresh-install S3 client was hitting one of them.

Fixed

  • Bug 1 (auth, critical) — external clients no longer get 403 NotSignedUp. Upstream s3s::auth::SimpleAuth verified every SigV4 signature against the single AKID/secret pair the gateway happened to have loaded (the IRSA / instance-role STS access key, rotated by the SDK every ~1 h). Any external client whose access key didn't match was rejected by the gateway itself. v1.4.1 vendors s3s 0.13.0 as crates/s3s/ (v0.13.0-s4.1), adds a NoVerify mode (S3ServiceBuilder::set_skip_signature_verification + auth::AcceptAnyAuth), and the gateway now accepts any SigV4-signed request — decoding the streaming body framing but skipping the HMAC compare — then re-signs to the real backend with its own SDK credentials. Trusted-network gateway posture: keep the gateway behind a security group.
  • Bug 2 (multipart, critical) — aws s3 cp of files ≥ 8 MiB no longer fails with Checksum Type mismatch. AWS CLI 2.30+ defaults to CRC64NVME on multipart and sends x-amz-checksum-algorithm: CRC64NVME on CreateMultipartUpload, pinning the session. The gateway's per-part frame strips client per-part checksums before forwarding, but the backend SDK then reapplied its own default (CRC32) on UploadPart. v1.4.1 strips checksum_algorithm/checksum_type at CreateMultipartUpload time too, so both sides converge on the SDK default.
  • Bug 3 (listings) — ListObjectsV2 is now client-transparent by default. HEAD/GET already returned MD5(original) + original size, but listings leaked the compressed bytes' size + backend ETag, so aws s3 sync and rclone declared the local copy newer/larger and re-transferred every run. v1.4.0 documented this as an opt-in (--accurate-list-size); v1.4.1 flips the default. Pass --physical-listings to opt back into the v1.4.0 fast-but-inconsistent listings (saves one bounded-concurrency backend HEAD per listed key). The deprecated --accurate-list-size is kept as a hidden no-op alias.
  • Bug 4 (multipart GET, critical) — multipart GET no longer fails --checksum-mode ENABLED with Expected full object checksum (crc64nvme) X did not match combined checksum: Y. AWS S3 auto-computes a CRC64NVME of the bytes it received (the compressed parts) and stamps it on the object at Complete. v1.4.0 echoed that compressed-bytes checksum to clients, who then computed crc64nvme over the decompressed body the gateway streamed back and failed verification. v1.4.1 strips ChecksumCRC* and ChecksumType from HEAD/GET responses for s4-framed objects in the buffered/multi-frame full-GET path too (the single-PUT and partial-Range paths already stripped them). ETag (logical = MD5(original)) remains the validator.

Changed

  • crates/s3s/ and crates/s3s-aws/ are vendored from upstream s3s 0.13.0 as 0.13.0-s4.1 (both publish = false). The fork is source-compatible at the call-site level — only additive set_skip_signature_verification + AwsChunkedStream::new skip_verify parameter — so an external embedder pinning upstream s3s = "0.13" is unaffected; only this workspace consumes the fork. Patch will be offered upstream separately.
  • crates/s4-server is publish = false: the gateway's distribution channels are Docker (ghcr.io/abyo-software/s4), the Marketplace AMIs (CPU + GPU), and the Marketplace Container chart; the vendored s3s fork can't ship to crates.io. s4-codec, s4-codec-py (PyPI), and s4-config are unaffected.
  • Helm chart bumped to 0.3.7 (appVersion: 1.4.1); chart shape unchanged so existing helm upgrade is a drop-in.
  • quinn-proto bumped 0.11.14 → 0.11.15 for RUSTSEC-2026-0185 (remote memory exhaustion).

Marketplace

CPU AMI (prod-4opohg7jaqo24), GPU AMI (prod-l5my73chs43y6), and Helm chart container (prod-nimrbd77e4xfs) all rebuilt against v1.4.1 with new AddDeliveryOptions pushed (CVE / AMI scan in flight ~40–60 min); v1.4.0 versions of all three will be restricted via RestrictDeliveryOptions once v1.4.1 is SUCCEEDED so only the fixed build is subscribable.

Validation

12/12 E2E checks PASS against a fresh CPU AMI launched from ghcr.io/abyo-software/s4:1.4.1 with the as profile + a dummy AKID — Bug 1-4 all confirmed closed, in-instance smoke regression unaffected.

🤖 Generated with Claude Code

Assets 2
Loading