[codex] consolidate terraform bulk execution on scheduler#2462
Closed
Mikhail Shirkov (shirkevich) wants to merge 15 commits into
Closed
Conversation
…ight noise Prevent terraform execution from falling back to default CI identity when users pass --identity/-i, skip CI hook setup on non-CI local runs unless forced, and propagate AuthManager through component processing so nested terraform state/output resolution reuses the authenticated identity. Made-with: Cursor
- Added tests to verify the behavior of the `-i` flag as an optional-value flag, ensuring that trailing native flags are preserved during pass-through stripping. - Implemented a test to confirm that explicitly passing an empty identity flag (`--identity=`) is treated as an interactive-selection sentinel, preventing it from being overridden by environment variables. - Updated the `TestProcessComponentConfig_PropagatesAuthManager` to ensure the correct propagation of the AuthContext with AWS credentials. These changes improve the robustness of identity handling in the Atmos CLI and ensure accurate propagation of authentication contexts.
…ity-ci-preflight-noise
…ity-ci-preflight-noise # Conflicts: # cmd/auth.go # cmd/auth_exec.go # cmd/auth_shell.go # cmd/terraform/flags.go # internal/exec/cli_utils_identity_test.go # internal/exec/cli_utils_test.go # pkg/component/ansible/executor.go # pkg/flags/global_registry.go # pkg/hooks/hooks_test.go
…ght-noise' into codex/dag-terraform-graph-bulk-path
Contributor
|
Tip Atmos Pro
No affected stacks workflow was detected for this pull request. 
|
Mikhail Shirkov (shirkevich)
force-pushed
the
codex/dag-terraform-graph-bulk-path
branch
from
ea62d6f to
9e83b56
Compare
Mikhail Shirkov (shirkevich)
changed the base branch from
main
to
codex/dag-scheduler-core
Collaborator
Author
|
Superseded by same-repo draft PR #2466, which uses cloudposse/atmos as the head repository and is based on codex/dag-scheduler-core. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacking
Branch history is stacked on #2461 and includes #2348. GitHub cannot use #2461 as the visible base because #2461's head branch currently lives in the fork, and pushing mirror branches to cloudposse/atmos is not permitted from this account.
Draft note
This is intentionally draft current state. The branch currently includes a temporary ATMOS_EXPERIMENTAL_DAG_MAX_CONCURRENCY override used for local timing experiments; the PR3-ready version should force effective concurrency back to 1 before review.
This branch now includes two concurrency-safety prerequisites discovered by timing experiments:
Validation
Local timing findings
Follow-up discussion
The short-term fix keeps current Atmos/Terraform behavior debuggable: commands still run in the normal component directory, and aliases sharing a physical folder are serialized. The longer-term way to unlock true parallelism for aliases sharing a folder would be per-node isolated workdirs plus isolated TF_DATA_DIR and generated files. That needs repo-owner discussion because it changes the operator debugging model: Atmos would need to decide whether and how to retain those per-node copies for inspection, how atmos terraform shell maps to them, and how cleanup/debug artifacts are managed.
Output remains heavily interleaved when concurrency is greater than 1, so user-visible concurrency still needs output orchestration before shipping.