Using extra_args on init to pass -no-color results in duplicate -no-color (auto-init)

[lorengordon]

Terragrunt's auto-init is adding -no-color to its terraform command, so when extra_args is used to pass -no-color to terraform init, we end up with two -no-color args on the command. Terraform considers this invalid and will just print its help/usage.

in terraform.tfvars:

    extra_arguments "init-no-color" {
      commands = ["init"]

      arguments = ["-no-color"]
    }

[terragrunt] [foo] 2019/02/08 06:46:17 Running command: terraform init -backend-config=region=us-east-1 -backend-config=bucket=foo -backend-config=key=tfstate/foo/terraform.tfstate -backend-config=encrypt=true -backend-config=dynamodb_table=foo -no-color -get=false -get-plugins=false -backend=false -from-module=file://C:/foo -no-color C:/.terragrunt-cache/-3SoM0iu9-_V75oTCkZXPsRZumo/k_ll3vc3K67aPC-9_M_dd1bo2Zo
Usage: terraform init [options] [DIR]

  Initialize a new or existing Terraform working directory by creating
  initial files, loading any remote state, downloading modules, etc.

  This is the first command that should be run for any new or existing
  Terraform configuration per machine. This sets up all the local data
  necessary to run Terraform that is typically not committed to version
  control.

  This command is always safe to run multiple times. Though subsequent runs
  may give errors, this command will never delete your configuration or
  state. Even so, if you have important information, please back it up prior
  to running this command, just in case.

  If no arguments are given, the configuration in this working directory
  is initialized.

[brikis98]

True, it doesn't look like we de-dupe args there. PR to fix it is welcome. Workaround is, of course, not to pass -no-color to init.

[lorengordon]

Do you happen to remember why -no-color was added as part of this commit? Is the fix as simple as just removing the argument?

[brikis98]

Because we're reading the stdout from Terraform to check for certain error messages and don't want that text polluted with color settings. Removing -no-color would not be a good fix. Best bet is to de-dupe arguments: e.g., only set our default value for -foo if the user hasn't passed in their own value for -foo (or its opposite -no-foo).

[lorengordon]

I don't see a clear way to do that at the moment. At least for now, I'll have to take a pass on trying to patch it myself.

[jasonarewhy]

Just adding some additional (no)color to this issue: we're using Atlantis in conjunction with Terragrunt, and Atlantis's output in Github PRs is dramatically improved by setting -no-color. This is something we used to set globally by setting TF_CLI_ARGS=-no-color in the Atlantis environment. I'm assuming due to recent Terraform changes that disallow duplicate args, Terragrunt's auto-init's addition of -no-color caused the auto-init to fail seemingly for no reason (since the TF_ARGS value isn't in the output).

I updated my Atlantis workflow to remove TF_CLI_ARGS=-no-color from the env and instead am directly running terragrunt -out .plan -no-color, which resolved my init issues, but oddly the auto-init now prints with colors, resulting in less readable output:

[terragrunt] [/home/atlantis/.atlantis/repos/mode/terraform-state/79/default/aws/aws_config] 2019/04/01 18:48:34 Running command: terraform init -backend-config=region=us-west-2 -backend-config=encrypt=true -backend-config=dynamodb_table=mode-terraform-state-lock-table -backend-config=key=aws/aws_config/terraform.tfstate -backend-config=bucket=mode-terraform-state

�[0m�[1mInitializing the backend...�[0m
�[0m�[32m
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.�[0m

�[0m�[1mInitializing provider plugins...�[0m
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "aws" (2.4.0)...

It's strange that I was clearly running into an issue with -no-color being set twice, yet when removing one instance of it, I now see color gunk (but only on the auto-init - the plan does not include colors as expected).

[mmclane]

I am experiencing the same thing.

[ilkelma]

I am also experiencing the auto init problem as @jasonarewhy (which might be a separate issue?) - It seems to stem from this file

terragrunt/cli/cli_app.go

Lines 756 to 767 in 2523967

	func prepareInitOptions(terragruntOptions *options.TerragruntOptions, terraformSource *TerraformSource) (*options.TerragruntOptions, error) {
	// Need to clone the terragruntOptions, so the TerraformCliArgs can be configured to run the init command
	initOptions := terragruntOptions.Clone(terragruntOptions.TerragruntConfigPath)
	initOptions.TerraformCliArgs = []string{CMD_INIT}
	initOptions.WorkingDir = terragruntOptions.WorkingDir
	initOptions.TerraformCommand = CMD_INIT
	// Don't pollute stdout with the stdout from Auto Init
	initOptions.Writer = initOptions.ErrWriter
	return initOptions, nil
	}

where the parent terragruntOptions are stripped out and replaced with a smaller set.

What it seems like is that the auto-init is run differently from when you run terragrunt init because if you run the actual command it runs prepareInitCommand defined here:

terragrunt/cli/cli_app.go

Lines 616 to 633 in 2523967

	func prepareInitCommand(terragruntOptions *options.TerragruntOptions, terragruntConfig *config.TerragruntConfig, allowSourceDownload bool) error {
	if terragruntConfig.RemoteState != nil {
	// Initialize the remote state if necessary (e.g. create S3 bucket and DynamoDB table)
	remoteStateNeedsInit, err := remoteStateNeedsInit(terragruntConfig.RemoteState, terragruntOptions)
	if err != nil {
	return err
	}
	if remoteStateNeedsInit {
	if err := terragruntConfig.RemoteState.Initialize(terragruntOptions); err != nil {
	return err
	}
	}
	// Add backend config arguments to the command
	terragruntOptions.InsertTerraformCliArgs(terragruntConfig.RemoteState.ToTerraformInitArgs()...)
	}
	return nil
	}

but auto-init happens here after we've already branched off from prepareInitCommand and are in prepareNonInitCommand:

terragrunt/cli/cli_app.go

Lines 689 to 701 in 2523967

	func prepareNonInitCommand(terragruntOptions *options.TerragruntOptions, terragruntConfig *config.TerragruntConfig) error {
	needsInit, err := needsInit(terragruntOptions, terragruntConfig)
	if err != nil {
	return err
	}
	if needsInit {
	if err := runTerraformInit(terragruntOptions, terragruntConfig, nil); err != nil {
	return err
	}
	}
	return nil
	}

Unless I'm way off base I'll try to get some free cycles to push up a PR to fix this (I think it's as simple as calling prepareInitCommand first/instead but i'll have to dig a little deeper).

[minhdanh]

@jasonarewhy Meanwhile I'm working around this issue when using Terragrunt with Atlantis by adding env var TF_CLI_ARGS=-no-color and in the plan command I run this: terragrunt plan -out=$PLANFILE 2>&1 | grep -v THIS_HELPS_GREP_REMOVE_COLORS

[dhirschfeld]

Just ran into this when trying to override, from the command line, a -var specified in extra_arguments. Rather than overriding the variable terragrunt just passes both on to terraform.

In my case terraform seems to pick the first occurrence and so ignores the -var specified on the command line.