Terragrunt should enforce best practices around version control

[brikis98]

Terragrunt provides locking for state files, but you can still have "concurrency" issues with the following two scenarios:

1. You forget to run git pull before running terragrunt apply, so you end up applying out-of-date templates.
2. You forget to run git commit and git push after running terragrunt apply, so even if your other teammates remember to run git pull, they will end up applying out-of-date templates.

Perhaps we should add a new setting in .terragrunt:

enforce_version_control = {
  backend = "git"
  config = {
    branch = "master"
    remote = "origin"
  }
}

If you have that config in your file, then every time you run terragrunt apply, Terragrunt will:

1. Obtain a lock (if one is specified).
2. Check if you are on master and if not, exit with an error.
3. Check if you have any uncommitted or unpushed files in master, and if you do, exit with an error.
4. Run git pull origin master.
5. Then run your actual Terraform command.

Note that you'd probably only want this config in "shared" environments such as production. For test environments, you will likely make many changes with Terraform that are not committed to version control, as a way to test those changes before committing.

[DrewEaster]

This proposal wouldn't solve the 2nd problem described above:

You forget to run git commit and git push after running terragrunt apply, so even if your other teammates remember to run git pull, they will end up applying out-of-date templates.

What about introducing an additional wrapper step if the actual Terraform command is apply that does a git commit and a git push? Trying to play out in my mind what the downsides of this would be...

[brikis98]

This proposal wouldn't solve the 2nd problem described above:

This would be handled by step 3:

3. Check if you have any uncommitted or unpushed files in master, and if you do, exit with an error.

I suppose that step could also do the git commit and git push for you, but that seems riskier, as every team has its own process for that (e.g. submit a PR).

[DrewEaster]

Whoops, my attention to detail had a fail moment there :-)

[brikis98]

Heh, no worries.

Do you think this approach would solve the issues with out of date .tf files?

[DrewEaster]

I guess there's still some issues:

1. Forcing changes to be pushed probably only makes sense if the command is apply? Why would one expect changes to be pushed unless an attempt was being made to actual mutate infrastructure?

2. It's feasible that the Terraform apply command might fail, in which case changes have been pushed to git and infrastructure not mutated.

3. If there's a merge conflict when pulling master, again, there's going to be a window during which infrastructure is not mutated to reflect the changes that have been pushed to git.

Not sure a simple command line tool would ever be able to close every gap, though.

[brikis98]

Forcing changes to be pushed probably only makes sense if the command is apply?

True. I updated the description to make that more clear.

It's feasible that the Terraform apply command might fail, in which case changes have been pushed to git and infrastructure not mutated.

True, but I'd argue those changes should still be in Git in that case, as they likely partially applied, and that code is still the best representation of your infrastructure that you have.

If there's a merge conflict when pulling master, again, there's going to be a window during which infrastructure is not mutated to reflect the changes that have been pushed to git.

If git pull fails, then terragrunt apply will release the lock and exit without applying any changes. Once you fix the conflict and commit and push, then you can run it again.

Not sure a simple command line tool would ever be able to close every gap, though.

No, but it seems like we can get 95% of what we need here.

[tamsky]

What if I don't use git ?

[brikis98]

@tamsky Support for other version control systems could be added too. That's why there is a backend setting in the proposal above.

[tamsky]

Just sayin,
We do everything you propose (checking for upstream and local unpushed changes), in our Makefile.

[brikis98]

We are unlikely to implement this, as we've seen that people use Terragrunt and their VCS in a variety of ways. Closing.