Option to prefix *-all terraform output with working-dir label

[lorengordon]

Right now, all terraform output is unmodified, which makes a lot of sense for regular terraform commands because they are only working on one config at a time and the output can be captured/used for other purposes.

But with the *-all commands, the terraform output of parallel configs just blends together and it is very difficult to determine what line of output is associated with which config.

For example, running validate-all, I don't know which config succeeded and which failed.

Success! The configuration is valid.

Error: Missing required argument

  on main.tf line 39, in output "repos":
  39: output "repos" {

The argument "value" is required, but no definition was found.

I'm suggesting that, only for the *-all commands, prefix terraform output the same way the terragrunt output is handled, so we get something like this...

[configs/dev/foo] Success! The configuration is valid.

[configs/dev/bar] Error: Missing required argument
[configs/dev/bar] 
[configs/dev/bar]   on main.tf line 39, in output "repos":
[configs/dev/bar]   39: output "repos" {
[configs/dev/bar] 
[configs/dev/bar] The argument "value" is required, but no definition was found.

This way we can filter the log very easily to see all the messages for a given config.

[lorengordon]

Linking to a related issue, #74. I think both options would be useful.

[brikis98]

This seems like a good idea. However, it needs a little more thought, as for some commands (e.g., output), we don't want to mess with stdout or stderr. That said, a PR to add this functionality, with some flag to disable it (e.g., --terragrunt-no-log-prefix) is very welcome!

[ITJamie]

+1 i want to see this as a feature. even if its not enabled by default. something like --terragrunt-enable-log-prefix

[nickmaccarthy]

+1 for this. Even something at the top of the plan just saying Plan for directory: /foo/bar/baz would be enough for us.

[lorengordon]

I've found I can only use run-all commands with --terragrunt-parallelism 1. It's incredibly slow, but it's the only way the output still makes sense.

[max-rocket-internet]

We are also plagued by this problem.

I had these ideas:

1. Prepend all lines with the module name/path
2. Buffer and group the output per module and print later with a header or something
3. Colourise the output to identify each module

prefix terraform output the same way the terragrunt output is handled

Where does this "working-dir label" come from exactly?

[meyerbro]

I really want this fixed but while it's not there's a possible workaround...

terraform {
  extra_arguments "plan_file" {
    commands = ["plan"]
    arguments = ["-out=${get_terragrunt_dir()}/tfplan.binary"]
  }
}

You then do terragrunt run-all plan.

Then for each tfplan.binary you have now you can do a terragrunt show tfplan.binary

[rmammadli]

Hi there,

just would like to share our workaround for this case:
`
using before_hook on root level to get more info about running module / config-dir right at the beginning

terraform {
  before_hook "before_hook" {
    commands     = ["apply", "plan", "import", "push"]
    execute      = ["echo", "Running Terraform"]
    run_on_error = true
  }
}
....

Result:

Group 1
- Module /home/myname/terraform/env/test/local-cluster/config

time=2022-04-27T00:47:33+02:00 level=info msg=Executing hook: before_hook prefix=[/home/myname/terraform/env/test/local-cluster/config]
Running Terraform
...

[lorengordon]

@rmammadli I feel like that's only a partial solution, and unfortunately does not address the primary use case in the OP. Configs run in parallel and their output is interleaved. Simply outputting the path before/after any given config runs is most useful when run one at a time. This feature request is primarily about running multiple configs at once in parallel, where it is unknown which line of output is associated to any particular config.

[max-rocket-internet]

Sometime the text output of run-all commands is actually totally scrambled, like this for example:

      + id                                    = (known after apply)
  # module.cache.aws_cloudwatch_metric_alarm.high_database_memory_usage[0] will be created
      + insufficient_data_actions             = [
  + resource "aws_cloudwatch_metric_alarm" "high_database_memory_usage" {
          + "arn:aws:sns:eu-west-1:6666666666:xxxxxx-yy-xxxxxx-xxxx-xxxx",
        ]

[denis256]

Fixed in https://github.com/gruntwork-io/terragrunt/releases/tag/v0.45.1