Skip to content

feat(ecr): Batch 2 — image + layer ops, content-addressed blob storage#718

Merged
vieiralucas merged 2 commits intomainfrom
worktree-worktree-ecr-batch2
Apr 23, 2026
Merged

feat(ecr): Batch 2 — image + layer ops, content-addressed blob storage#718
vieiralucas merged 2 commits intomainfrom
worktree-worktree-ecr-batch2

Conversation

@vieiralucas
Copy link
Copy Markdown
Member

@vieiralucas vieiralucas commented Apr 23, 2026

Summary

  • Batch 2 of 4 in the ECR buildout, building on feat(ecr): add ECR Batch 1 — repository CRUD, tags, policies #717.
  • Ships 10 ops: PutImage, BatchGetImage, BatchDeleteImage, BatchCheckLayerAvailability, DescribeImages, ListImages, GetDownloadUrlForLayer, InitiateLayerUpload, UploadLayerPart, CompleteLayerUpload.
  • Adds the content-addressed sha256 blob storage that Batch 3's OCI v2 Distribution protocol (the thing that makes real docker push / docker pull work) will expose over HTTP.
  • 21/58 ECR operations now implemented, all 21 at 100% variant coverage.
  • Baseline bumped: 60,621 / 61,743 variants passing (98.2%). No regressions on the 23 pre-existing services.

Key behaviours

  • Layer upload state machine. InitiateLayerUpload returns a UUID + 10 MiB partSize (matching AWS). UploadLayerPart requires contiguous byte ranges and part-size consistency, bumping last_byte_received. CompleteLayerUpload recomputes the sha256 of the received bytes and rejects mismatched digests with LayerDigestMismatchException.
  • Tag mutability. PutImage on an IMMUTABLE repo returns ImageAlreadyExistsException when a tag already points at a different digest; MUTABLE repos allow tag reassignment.
  • BatchDeleteImage semantics. Deleting by tag removes only the tag when other tags still reference the digest; deleting by digest or removing the last tag removes the underlying image.
  • Pagination. DescribeImages / ListImages use AWS's documented default page size of 100, enforce maxResults 1..=1000, and return InvalidContinuationTokenException on garbage nextToken input.
  • Persistence. Snapshot schema v1 -> v2, with every new field serde(default) so v1 snapshots load cleanly. All mutating ops trigger a snapshot save.

Test plan

  • cargo build --workspace
  • cargo fmt --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test -p fakecloud-e2e --test ecr --test ecr_persistence --test ecr_images — 13 passing
  • cargo test -p fakecloud-conformance --test ecr — 21 passing
  • cargo run -p fakecloud-conformance --release -- run --services ecr — 21/58 ops implemented, all 100%
  • cargo run -p fakecloud-conformance --release -- check — no regressions on other services

Follow-up

  • Batch 3: OCI v2 Distribution HTTP endpoints (/v2/...) + GetAuthorizationToken + Basic-Auth — the differentiator that makes docker push work against localhost:4566.
  • Batch 4: lifecycle + scanning + registry + CloudFormation/Lambda integration + website landing + marketing.

Summary by cubic

Adds ECR image and layer operations with content‑addressed sha256 blob storage. Enables full image management via the JSON control plane and sets up OCI v2 push/pull next.

  • New Features
    • Implemented: PutImage, BatchGetImage, BatchDeleteImage, BatchCheckLayerAvailability, DescribeImages, ListImages, GetDownloadUrlForLayer, InitiateLayerUpload, UploadLayerPart, CompleteLayerUpload.
    • Layer upload state machine: Initiate returns UUID + 10 MiB partSize; UploadLayerPart enforces contiguous ranges; CompleteLayerUpload validates digest before finalizing so mismatches return LayerDigestMismatchException without dropping upload state (retryable).
    • PutImage: computes manifest digest; if a supplied imageDigest mismatches, returns ImageDigestDoesNotMatchException. Tag rules: IMMUTABLE repos block tag reassignment (ImageAlreadyExistsException); MUTABLE repos allow it.
    • Delete behavior: deleting by tag removes only the tag; deleting by digest or the last tag removes the image.
    • Pagination: default 100 items; maxResults 1..=1000; invalid nextToken → InvalidContinuationTokenException.
    • Blob storage: per‑repo, sha256‑addressed layers; GetDownloadUrlForLayer returns /v2/<repo>/blobs/<digest> URL.
    • Persistence: snapshot schema v2 with serde defaults (v1 snapshots still load); all mutating ops save snapshots.
    • Conformance: 21/58 ECR ops at 100% variant coverage; baseline 60,621/61,743 (98.2%); no regressions.

Written for commit cc6b6b1. Summary will update on new commits.

Batch 2 of 4. Adds the 10 ops that make real image pushes work via
the JSON control plane, and gives us the content-addressed layer
storage that Batch 3's OCI v2 Distribution protocol will expose over
HTTP.

Operations:
- PutImage, BatchGetImage, BatchDeleteImage
- BatchCheckLayerAvailability
- DescribeImages, ListImages
- GetDownloadUrlForLayer
- InitiateLayerUpload, UploadLayerPart, CompleteLayerUpload

Implementation:
- sha256-addressed layer blobs stored per repo (snapshot as base64).
- Layer upload state machine keyed by uploadId; UploadLayerPart
  validates contiguous byte ranges and part-size consistency.
- CompleteLayerUpload recomputes the sha256 of received bytes and
  rejects mismatched digests with LayerDigestMismatchException.
- PutImage computes the manifest digest, supports tag reassignment
  on MUTABLE repos, and returns ImageAlreadyExistsException on
  IMMUTABLE repos.
- BatchDeleteImage removes tag-only when other tags still point at
  the digest; removes the image when the last tag/digest is gone.
- DescribeImages / ListImages paginate with the AWS default page size
  of 100, validate maxResults 1..=1000, and reject invalid
  continuation tokens with InvalidContinuationTokenException.
- Snapshot schema bumped to v2; state grows `images`, `image_tags`,
  `layers`, and `layer_uploads` fields, all serde(default) so an
  existing v1 snapshot loads cleanly.

Tests:
- 6 E2E round-trips (layer upload, digest mismatch, put/describe,
  batch-get, tag-only delete, immutable-tag guard).
- 10 new conformance tests (one per new op) — total 21/58 ops
  implemented, all at 100% variant coverage.

Baseline bumped: 60621 / 61743 variants pass (98.2%).
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 23, 2026

Codecov Report

❌ Patch coverage is 0.17391% with 574 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/fakecloud-ecr/src/service.rs 0.00% 570 Missing ⚠️
crates/fakecloud-ecr/src/state.rs 20.00% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 7 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/fakecloud-ecr/src/service.rs">

<violation number="1" location="crates/fakecloud-ecr/src/service.rs:791">
P1: Validate `imageDigest` against the uploaded manifest bytes before accepting the push.</violation>

<violation number="2" location="crates/fakecloud-ecr/src/service.rs:1305">
P1: A digest mismatch currently deletes the in-progress upload state, preventing retry of the same upload.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread crates/fakecloud-ecr/src/service.rs
Comment thread crates/fakecloud-ecr/src/service.rs
- CompleteLayerUpload now validates the received-bytes digest BEFORE
  removing upload state, so a mismatched digest no longer forces the
  caller to re-upload the whole blob; retrying with the correct
  digest reuses the in-flight upload.
- PutImage now validates the optional supplied imageDigest against
  the sha256 of the actual manifest bytes and rejects mismatches
  with ImageDigestDoesNotMatchException instead of silently storing
  the user-supplied digest.
@vieiralucas vieiralucas merged commit 59c667f into main Apr 23, 2026
48 checks passed
@vieiralucas vieiralucas deleted the worktree-worktree-ecr-batch2 branch April 23, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant