Allow one terragrunt file to reference another terragrunt file

[robkinyon]

Per #26, I have a lot of commonalities between .terragrunt files in different directories. I'd really like to be able to do something like:

# devops/infrastructure/.terragrunt
dynamoDbLock = { stateFileId = "my_project_terraform_db_lock" }

remoteState = {
  backend = "s3"
  backendConfigs = {
    encrypt = "true"
    bucket = "my-bucket"
    region = "us-east-1"
  }
}

Then, in each directory, I could do something like:

# devops/infrastructure/application-qa/.terragrunt
import("../.terragrunt")
remoteState = {
  backendConfigs = {
    key = "terragrunt/application-qa.tfstate"
  }
}

The idea being that if I do an include, anything I do from that point onwards would override what was imported.

[josh-padnick]

@robkinyon I've experienced this myself, and it was indeed easy to forget to change my .terragrunt file.

In #26, you suggested an interpolation variable like ${dirname()}, but most users probably won't want that to resolve to an absolute path, so we need some way to indicate the "root" folder of the repo. I'm going to think out loud here and would love to hear input from @brikis98.

Option 1: Root Property

We could add a rootDirectory property to explicitly indicate the root directory. It could use absolute or relative paths. If it uses absolute paths, have we now gone from replacing the key to replacing rootDirectory which hasn't bought us much?

rootDirectory  = "../../"
...
remoteState = {
  backend = "s3"
  backendConfigs = {
    ...
    key = "${relativePath(var.rootDirectory)}"
  }
}

Option 2: Import an empty .terragrunt.root file

The file .terragrunt.root could be an empty marker file. This way, if you get your import path wrong, terragrunt will throw an error versus interpreting some other .terragrunt file.

import("../../.terragrunt.root")
...
remoteState = {
  backend = "s3"
  backendConfigs = {
    ...
    key = "${relativePath()}"
  }
}

Option 3: Import a non-empty .terragrunt.root file

This is what @robkinyon suggested. My concern here is what happens when the .terragrunt.root values are changed? We have no way of knowing which "child" .terragrunt files have been "migrated" to the latest settings.

Opinions welcome on which of these works best, or if there are other ideas.

[brikis98]

The syntax we're using for .terragrunt files is HCL, the same syntax used for Terraform, which is equivalent to JSON. So I suppose an import or include statement would be roughly equivalent to starting with the imported JSON code and then merging in the JSON code in the current file. So the implementation is pretty straight forward and would even work for multiple levels of import.

I don't think you need any dirname magic here either, as using relative import paths would work fine, and terragrunt could exit with an error if a path doesn't resolve to an actual .terragrunt file.

My concern, however, is that moving away from purely declarative, WYSIWYG experience increases the complexity. To make that worthwhile, we need to be clear on what problems we're solving:

1. My .terragrunt files are not DRY. This doesn't bother me. They don't change often and they are declarative, so this alone wouldn't be worth changing anything.
2. I make manual errors when copying/pasting settings from an old .terragrunt file to a new one. This is a legitimate concern, as the whole point of terragrunt is to reduce manual error. See Interpolation inside .terragrunt file #26 (comment) for my thoughts on ways that can be mitigated and their trade offs.

[brikis98]

We are discussing more or less the same ideas in #26, so I'm going to close this issue in favor of consolidating the conversation in one place.