feat(batch): job dependencies (dependsOn) — gated launch (batch 4)#1988
Merged
Conversation
- SubmitJob with `dependsOn` parks the job at PENDING and spawns a background
waiter that launches it only once every dependency has SUCCEEDED; if any
dependency FAILED the dependent job fails ("Dependent job failed"). The wait
never blocks the SubmitJob call.
- Refactored the launch path (launch / launch_ecs_task / spawn_status_sync) into
standalone free functions taking a `LaunchCtx` bundle, so the spawned waiter
can launch with no `&self` exactly like the inline SubmitJob path. The
`&self launch_job` is now a thin wrapper.
Tests: unit (dependsOn parks at PENDING without a runtime); Docker-gated e2e
(B dependsOn A: B stays PENDING while A runs, both SUCCEEDED after). Existing
real-execution + array e2e still pass against the refactored launch path. Docs
updated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AWS Batch batch 4 — job dependencies (
dependsOn), the orchestration feature every rival is weak on (ministack waitForTaskToken no-ops #959).SubmitJobwithdependsOnparks the job atPENDINGand spawns a background waiter that launches it only once every dependency hasSUCCEEDED; if any dependencyFAILEDthe dependent job fails ("Dependent job failed"). The wait never blocks theSubmitJobcall.launch/launch_ecs_task/spawn_status_sync) is now standalone free functions taking aLaunchCtxbundle, so the spawned waiter launches with no&self, exactly like the inline SubmitJob path.launch_jobis a thin&selfwrapper.Test plan
depends_on_parks_at_pending(no runtime → stays PENDING), plus existing.depends_on_job_waits_for_its_dependency— BdependsOnA: B is PENDING while A runs (3s), bothSUCCEEDEDafter. The existing real-execution / array / fail e2e still pass against the refactored launch path (all 4 green locally).cargo clippy -p fakecloud-batch --bin fakecloud --all-targets -- -D warningsclean. Docs updated.Next
Retry strategies + timeouts, then a CloudFormation
AWS::Batch::*provisioner + tfacc.Summary by cubic
Adds Batch job dependencies via
dependsOn. Dependent jobs park at PENDING and launch only after all parents succeed; they fail if any parent fails. The wait is non-blocking and works with real ECS-backed execution.New Features
SubmitJobwithdependsOnsets job toPENDINGand spawns a background waiter.SUCCEEDED; fails with "Dependent job failed" if any isFAILED.SubmitJobcall; unchanged behavior when no runtime is present.Refactors
launch,launch_ecs_task,spawn_status_sync, driven by aLaunchCtx.launch_jobis now a thin wrapper. Enables launching from the dependency waiter without&self.Written for commit 4819688. Summary will update on new commits.