feat(batch): real ECS-backed SubmitJob execution (batch 2)#1986
Merged
Conversation
The headline differentiator: SubmitJob now runs a REAL container, not a fake. - Wire the ECS state + runtime into BatchService (with_ecs); SubmitJob resolves the job definition's containerProperties (+ this submit's containerOverrides), ensures a `fakecloud-batch` ECS cluster, RegisterTaskDefinition mapping vcpus/memory (legacy fields or resourceRequirements) -> ECS cpu/memory, and RunTask — reusing all of ECS's real container / portability / k8s handling through its public handle() API (mirrors how autoscaling drives EC2). - A background poller maps the ECS task's real lifecycle + container exit code onto the Batch job: SUBMITTED -> STARTING -> RUNNING -> SUCCEEDED (exit 0) or FAILED (carrying the real container.exitCode). The launch is non-blocking (ECS RunTask backgrounds the container pull/start, so no client timeout). - NO auto-success: with no container runtime wired the job stays SUBMITTED honestly — exactly the rival anti-pattern Batch beats (MiniStack jumps to SUCCEEDED with no compute). Tests: unit (resource mapping from both shapes, job-def latest-revision + override resolution, no-runtime stays SUBMITTED); Docker-gated e2e (real alpine container: exit 0 -> SUCCEEDED/exitCode 0, exit 7 -> FAILED/exitCode 7). Docs page updated to reflect real execution.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The headline differentiator for the cycle-5 winner: AWS Batch SubmitJob runs a REAL container, not a fake. Every other free emulator fakes Batch compute (MiniStack jumps straight to SUCCEEDED with no container; LocalStack paywalls it).
BatchService(with_ecs). SubmitJob resolves the job definition'scontainerProperties(+ this submit'scontainerOverrides), ensures afakecloud-batchECS cluster, RegisterTaskDefinition (mapping vcpus/memory — legacy fields orresourceRequirements— to ECS cpu/memory), and RunTask — reusing all of ECS's real container / portability / k8s handling through its publichandle()API (mirrors how autoscaling drives EC2).SUBMITTED → STARTING → RUNNING → SUCCEEDED(exit 0) orFAILED(carrying the realcontainer.exitCode). Launch is non-blocking (ECS RunTask backgrounds the container pull/start → no client timeout).SUBMITTEDhonestly — exactly the rival anti-pattern Batch beats.Test plan
batch_real_execution.rs: real alpine container —exit 0 → SUCCEEDED/container.exitCode 0,exit 7 → FAILED/exitCode 7. Both PASS locally with Docker.batch.rsstill passes;cargo clippy -p fakecloud-batch --bin fakecloud --all-targets -- -D warningsclean; doc-counts passes.Surface
Docs page updated to reflect real execution. The
batch_real_executione2e mirrors the EC2/ECS Docker-gated pattern (panics in CI without Docker, skips locally).Next batches
Array jobs +
dependsOn+ retry/timeout, then a CloudFormationAWS::Batch::*provisioner + tfacc.Summary by cubic
SubmitJob now runs real containers on ECS via
fakecloud-ecs, advancing jobs through STARTING/RUNNING and finishing as SUCCEEDED or FAILED from the container’s exit code; without a container runtime they remain SUBMITTED.New Features
fakecloud-batchECS cluster, register a task definition (map legacyvcpus/memoryorresourceRequirements→ ECS cpu/memory), then run a task.container.exitCode; launch is non-blocking.Refactors
resourceRequirementsinstead of deprecatedvcpus/memory; docs updated.Written for commit 97ca700. Summary will update on new commits.