Feature: Priority Directories, Priority Plans

[natalysheinin]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

• I'd be willing to implement this feature (contributing guide)

Describe the user story
As a developer, I want to be able to create a PR that bootstraps a bunch of required infrastructure & targets changes across various directories (1+ statefiles). For a good UX, I want to guarantee that the atlantis workflow will help me succeed without the complexities of order of operations.

As an atlantis admin, I want to configure certain statefiles/directories to have priority such that in a single PR, atlantis ensures that a priority plan/apply workflow gets completed before others.

Describe the solution you'd like

• I want to be able to identify which "dir" are priority in atlantis.yaml.
• I want this to be achievable within 1 PR.
• I want this to be solved with staged plans (don't generate an atlantis plan for any directories until a plan/apply is complete for the priority directory that is modified). This will be important to stay consistent with mergeable requirements.

Describe the drawbacks of your solution

Describe alternatives you've considered

1. Do this in 2 separate PRs - annoying UX
2. Consolidate state and use "depends_on"- breaks intended code organization of state file per team function / infrastructure function

[jamengual]

have you seen this? #2146

[natalysheinin]

Related to: #391

[jamengual]

@natalysheinin do you think that this : #3292 could work for you?

[vincentgna]

As I'm working on this feature currently, I would like to propose the following functionality expanding #3292 depends_on functionality with a configurable behaviour:

1. During plan, do not plan projects which have unapplied dependencies, update the commit status for these projects with a new state flag indicating "plan is pending"
2. During apply, only apply planned projects
3. (potentially automatically after apply?) Autoplan any dependant projects in "pending" state after apply run is complete
4. Repeat until all projects impacted by a PR are in applied state (automerge PR when all planned projects are successfully applied)

I see a few concerns:

• Should Atlantis “re-request” PR review after first apply to ensure the approval is requested for each phase? (this could be configurable, initially we won’t implement this and behaviour would be to not re-request approval(s))

• This may result in more long lived PRs with partially applied plans (Atlantis locks exist for this reason, once the PR is partially applied, the projects applied should be locked to the PR).

  When issuing the atlantis unlock comment, in future implementations, we could warn that already applied projects could cause significant issues if the PR lock is discarded)

Without this built-in functionality within Atlantis, it is the "executor" that is responsible to orchestrate the plan/apply steps.

For example:

• with terragrunt, you may use mock_outputs to ensure plan passes (altho with "mock" values...)
• with native Terraform, you may use toggle variables which are disabled on the initial PR and the onus is on the user to create multiple PRs to toggle variables on as dependencies are resolved (layer-by-layer, no atlantis automation). Anything else with Dynamic resolution of dependencies runs counter to Terraform design goals

Refs:

• Add support for stages of plans #972 (comment)
• Support prerequisite apply of projects for apply_requirements #391 (comment)