Add option to s3 backend to force using the instance profile credentials.

[tmccombs]

OpenTofu Version

1.6.2

Use Cases

My CI system sets the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

This is great for getting credentials to the account that opentofu is managing resources for.

However, I want to use a single s3 bucket to store the state for stacks across multiple accounts, and reference the outputs with terraform_remote_state data source.

It would be most convenient if I could just use the EC2 instance credentils of the host to access the bucket. However, since AWS credential

It would be more convenient to get this working if I could configure the backend or remote state data source to use the credentials of the ec2 instance. But as far as I can tell there isn't a straighforward way to do that.

Attempted Solutions

There are a couple workarounds for this.

You can grant all of the roles that terraform runs with access to the s3 bucket (and possibly kms key) using cross account permissions on the resource policies. However, this adds a lot of complexity, and doesn't work for granting access to a dynamodb table if locking is used.

You can grant all of the roles permission to assume a role back in the original account, and give that role the permissions needed. However, this is even more complicated than the first solution.

You can have something get temporary credentials from the instance metadata endpoint, and then store those credentials in a file that you reference in the backend or terraform_remote_state data source. But again, this is very complicated.

Proposal

Add an option to the s3 backend, and associated terraform_remote_state configuration that if set to true tells terraform to ignore the environment variables for aws credentials, and instead prefer the instance metadata credentials.

References

No response

[Evi1Pumpkin]

Hey @tmccombs, thank you for submitting this issue.
I'm not sure I completely understand your use case. Can't you unset the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env variables, and then use EC2 instance profiles instead (as explained here)?

[tmccombs]

No, because I need the AWS_ACCESS_KEY_ID and AWS_SECEET_ACCESS_KEY for the aws provider, but I want to use the ec2 instance credentials for the s3 backend.

[sftim]

There are workarounds for this that don't need any change to core Tofu. For example, use profiles with credential_process defined, and write either one or two short wrapper programs to yield the appropriate credentials.

[tmccombs]

I guess that might work.

Although, it is rather more complicated in a CI/CD environment, since instead of just setting environment variables, you have to inject files. One of which needs to be in the runners home directory, which may not be writable.

[lorengordon]

You could put the aws config file anywhere you like, using the env AWS_CONFIG_FILE.