-chdir is not honored when attempting to use terraform.d/plugins directory in the current working directory

[fnickels]

Terraform Version

$ terraform --version
Terraform v1.2.5
on linux_amd64

Terraform Configuration Files

https://github.com/fnickels/terraform-plugin-bug

Debug Output

Expected Behavior

Terraform provider versions saved under terraform.d/plugins directory should be picked up when the current working directory is specified with the command line option -chdir.

ie:

terraform -chdir=./app1 init

should behave the same as

cd ./app1 && terraform init

when there are valid provider files under ./app1/terraform.d/plugins

Actual Behavior

cd ./app1 && terraform init

returns the expected results

terraform -chdir=./app1 init

gets the most recent version on the public mirror and appears to ignore the local mirror

Steps to Reproduce

Create a file ./app1/main.tf with the contents such as:

terraform {
  required_providers {
    # standard form
    aws = {
      source  = "registry.terraform.io/hashicorp/aws"
      version = ">= 1.0"
    }
    external = {
      source  = "registry.terraform.io/hashicorp/external"
      version = ">= 1.0"
    }
  }
}

variable test {
  type = bool
  default = true
}

output mytest {
    value = var.test
}

Create a file ./providermirror1/mirrorsource.tf with the contents such as:

terraform {
  required_providers {
    # standard form
    aws = {
      source  = "registry.terraform.io/hashicorp/aws"
      version = "4.21.0"
    }
  }
}

Run:

	#
	# Build Local Mirror
	#
	rm -rf ./mirror1 ./app1/terraform.d
	terraform -chdir=./providermirror1 \
		providers mirror -platform=linux_amd64 ../mirror1/terraform/plugins
	#
	# copy mirror files to local default plugin directories
	#
	cp -r ./mirror1/terraform ./app1/terraform.d
	#
	# Test without -chdir  and local plugin directory
	#
	rm -rf ./app1/.terraform  ./app1/terraform.tfstate ./app1/.terraform.tfstate.lock.info  ./app1/.terraform.lock.hcl
	cd ./app1 && terraform init
	#
	# Test with -chdir and local plugin directory
	#
	rm -rf ./app1/.terraform ./app1/terraform.tfstate ./app1/.terraform.tfstate.lock.info  ./app1/.terraform.lock.hcl
	terraform -chdir=./app1 init

Additional Context

References

[crw]

Thanks for the report!

[apparentlymart]

Thanks for sharing this, @fnickels!

I think this is an interesting edge case where I can see arguments on both sides for why this should or should not respect -chdir.

The current behavior arises from the fact that Terraform considers the CLI Configuration -- that is, the settings that control how Terraform's CLI layer behaves, which includes provider installation and discovery -- to be something handled before switching to the target directory, which then allows a team with many different directories to share a single CLI configuration across all of them even if it's at a path relative to the current working directory.

However, since these implied local mirror directories are automatic, and not activated by explicitly adding settings to the CLI configuration, I can see why it could seem counter-intuitive for Terraform to consider them as part of the CLI configuration. In practice, Terraform treats the implied filesystem mirror directories as if you had written a provider_installation block in the CLI configuration, and so from Terraform's perspective these are equivalent.

Terraform is working as designed here, so I'm going to classify this as an enhancement instead so that we can use this issue to decide whether to and how to change Terraform's design, rather than how to change Terraform's implementation to match the current design intent (since it already matches).

Thanks!

[fnickels]

I suspected there were some side effects I was not on top of.

The doc already specifies global locations for mirrors: /usr/local/share/terraform/plugins, and /usr/share/terraform/plugins., so using terraform/plugins from the launch directory feels weird. If you really need to manage things more granularly there are all the options in the explicit settings.

The doc states:

If a terraform.d/plugins directory exists in the current working directory then Terraform will also include that directory,
regardless of your operating system.

Based on this I would argue this looks like a bug, because -chdir changes the CWD for everything else.

I would suggest changing either the doc to represent the code behavior or the code behavior to represent the doc. I can put up an alternate PR that only changes the current working directory for the provider mirror. It is just a bit more work than the PR I posted yesterday, which has side effects.

[fnickels]

Posted up an alternate implementation of a fix that preserves most of the side-effects of the original implementation.

[apparentlymart]

Hi @fnickels! Thanks for sharing this additional information.

I think before we start thinking about whether and how to change Terraform's design to address your use-case we should probably discuss a bit more what exactly your use-case is.

Can you say a little more about why it's useful to you to have a separate filesystem mirror directory for each of your configurations? While I'm sure the two PRs you have opened will make what you tried work, we first need to determine whether they are the best way to meet this need, and whether what you've done there is likely to be useful to other Terraform users with similar use-cases. We might find that there's a different design that would still allow you to achieve your goal.

Thanks!

[fnickels]

Per the documentation, I placed my mirror files (terraform.d/plugins) at the root of the application.

In my example, my .tf files are all in ./app1

from my CWD if I do

cd ./app1
terraform init

terraform picks up the providers I have populated in the mirror directories as expected. All is good.

But, if I choose to use:

terraform -chdir=./app1 init

My mirror files are ignored and I get the provider files from the public mirrors. Not expected, not good.

Putting mirror files at ./ as a default seems arbitrary. Especially when the documentation calls out:

If a terraform.d/plugins directory exists in the current working directory then Terraform will also include that directory,
regardless of your operating system.

If someone is going to the effort to manage mirrors based on arbitrary directories, the explicit options provided appear to be more than sufficient. But taking the documentation at face value and using the implicit behaviors I would expect both to work the same.

The real question is what is the current working directory when -chdir is in play?

I have read https://www.terraform.io/cli/commands#switching-working-directory-with-chdir and I see what you are getting at, but for the average user, I don't think this nuance is easy to decern and understand. The behavior of mirrors is not called out here in a clean fashion.

I posted a 3rd alternate implementation that I believe preserves the backward compatibility and allows the ./app1/terraform.d/plugins directory in my example when using -chdir.

At the end of the day, I don't care too much what you guys ultimately choose to do, but in my eyes, the current behavior is inaccurate when referring to the current documentation. I would simply suggest either changing the documentation to reflect the actual behavior or modifying the code to match the documentation.

Running similar commands in two different ways relying on implicit behaviors and getting two different results does not feel good. and could result in hours of troubleshooting for the average user and may lead them to believe terraform is flaky.

I love your product, I use it almost every day, just trying to help make it better.