Use Terraform to inform AWS Auth

[dylanratcliffe]

We are seeing a very, very high number of user hit issues when authenticating with AWS. This is probably because our approach to this is pretty basic, it basically consists of:

• Ask the user what account they want
• Try to authenticate to everything

This can cause a number of problems, namely

• We are authenticated to way more regions than we need to be, which slows things down
• It's possible for a user to auth correctly, and populate the cache, but for the complete wrong account

The ideal way to do this would be to:

1. Parse the terraform config and work out exactly what the AWS config should be
2. Replicate this config exactly in how we authenticate

Yeah but how?

There are a number of things that we would have to solve here:

• Work out all the possible places that the provider can be configured, and whether these can be dynamic i.e. assuming a role etc.
• Implement a HCL parser (this should be easy to find in opentofu somewhere)
• Work out which files we need to be parsing
• Pass all this to the AWS source

One we have this though, we would be able to tell for sure what AWS access Terraform expected and give a really good error if we don't have that access

AWS Provider Config Options

In HCL

You can pass things directly to the config in HCL like so:

provider "aws" {
  region     = "us-west-2"
  access_key = "my-access-key"
  secret_key = "my-secret-key"
}

There is also a block for assuming a role:

provider "aws" {
  assume_role {
    role_arn     = "arn:aws:iam::123456789012:role/ROLE_NAME"
    session_name = "SESSION_NAME"
    external_id  = "EXTERNAL_ID"
  }

  assume_role_with_web_identity {
    role_arn                = "arn:aws:iam::123456789012:role/ROLE_NAME"
    session_name            = "SESSION_NAME"
    web_identity_token_file = "/Users/tf_user/secrets/web-identity-token"
  }
}

Env Vars

Credentials can be provided by using the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and optionally AWS_SESSION_TOKEN environment variables. The Region can be set using the AWS_REGION or AWS_DEFAULT_REGION environment variables.

After some analysis all of these env vars are just exactly the AWS ones, which can be retrieved using NewEnvConfig()

Full Reference

Since all of the env vars are just the default AWS ones, we can almost certainly do something like this:

cfg, err := config.LoadDefaultConfig(context.TODO(),
	config.WithRegion("us-west-2"),
	// Then just use more funcitons to map here
)

General Settings

Setting	Provider	Environment Variable	Shared Config	Go Function
Access Key ID	access_key	AWS_ACCESS_KEY_ID	aws_access_key_id	WithCredentialsProvider()
Secret Access Key	secret_key	AWS_SECRET_ACCESS_KEY	aws_secret_access_key	WithCredentialsProvider()
Session Token	token	AWS_SESSION_TOKEN	aws_session_token	WithCredentialsProvider()
Region	region	AWS_REGION or AWS_DEFAULT_REGION	region	WithRegion()
Custom CA Bundle	custom_ca_bundle	AWS_CA_BUNDLE	ca_bundle	WithCustomCABundle
EC2 IMDS Endpoint	ec2_metadata_service_endpoint	AWS_EC2_METADATA_SERVICE_ENDPOINT	N/A	WithEC2IMDSEndpoint()
EC2 IMDS Endpoint Mode	ec2_metadata_service_endpoint_mode	AWS_EC2_METADATA_SERVICE_ENDPOINT_MODE	N/A	WithEC2IMDSEndpointMode()
Disable EC2 IMDS	skip_metadata_api_check	AWS_EC2_METADATA_DISABLED	N/A	WithEC2IMDSClientEnableState()
HTTP Proxy	http_proxy	HTTP_PROXY or http_proxy	N/A	Doesn't exist, we will need to set the env var
HTTPS Proxy	https_proxy	HTTPS_PROXY or https_proxy	N/A	Doesn't exist, we will need to set the env var
Non-Proxied Hosts	no_proxy	NO_PROXY or no_proxy	N/A	Doesn't exist, we will need to set the env var
Max Retries	max_retries	AWS_MAX_ATTEMPTS	max_attempts	WithRetryMaxAttempts()
Profile	profile	AWS_PROFILE or AWS_DEFAULT_PROFILE	N/A	WithSharedConfigProfile()
Retry Mode	retry_mode	AWS_RETRY_MODE	retry_mode	WithRetryMode
Shared Config Files	shared_config_files	AWS_CONFIG_FILE	N/A	WithSharedConfigFiles()
Shared Credentials Files	shared_credentials_files	AWS_SHARED_CREDENTIALS_FILE	N/A	WithSharedCredentialsFiles()
S3 Use Regional Endpoint for us-east-1	s3_us_east_1_regional_endpoint	AWS_S3_US_EAST_1_REGIONAL_ENDPOINT	s3_us_east_1_regional_endpoint	I vote we skip this as it's an old thing provided for compatibility (reference)
Use DualStack Endpoints	use_dualstack_endpoint	AWS_USE_DUALSTACK_ENDPOINT	use_dualstack_endpoint	WithUseDualStackEndpoint()
Use FIPS Endpoints	use_fips_endpoint	AWS_USE_FIPS_ENDPOINT	use_fips_endpoint	WithUseFIPSEndpoint()

Assume Role Settings

These will need to be provided to WithAssumeRoleCredentialOptions() using an AssumeRoleOptions struct

Setting	Provider	Shared Config	Go struct field
Role ARN	role_arn	role_arn	RoleARN
Duration	duration	duration_seconds	Duration
External ID	external_id	external_id	ExternalID
Policy	policy	N/A	Policy
Policy ARNs	policy_arns	N/A	PolicyARNs
Session Name	session_name	role_session_name	RoleSessionName
Source Identity	source_identity	N/A	SourceIdentity
Tags	tags	N/A	Tags
Transitive Tag Keys	transitive_tag_keys	N/A	TransitiveTagKeys

Assume Role With Web Identity Settings

These will need to be provided to WithWebIdentityRoleCredentialOptions() using an WebIdentityRoleOptions struct

Setting	Provider	Environment Variable	Shared Config	Go struct field
Role ARN	role_arn	AWS_ROLE_ARN	role_arn	RoleARN
Web Identity Token	web_identity_token	N/A	N/A	TokenRetriever
Web Identity Token File	web_identity_token_file	AWS_WEB_IDENTITY_TOKEN_FILE	web_identity_token_file	N/A. You can set this directly in the result config though e.g. here
Duration	duration	N/A	duration_seconds	Duration
Policy	policy	N/A	policy	Policy
Policy ARNs	policy_arns	N/A	policy_arns	PolicyARNs
Session Name	session_name	AWS_ROLE_SESSION_NAME	role_session_name	RoleSessionName

File parsing

As far as I can tell we will need to parse all files in the current directory that match *.tf. Looks like the parsing will be pretty easy using this: https://github.com/hashicorp/hcl/blob/main/hclsimple/hclsimple.go

[dylanratcliffe]

We need some changes to the AWS source to allow us to provide our own config which have been made here: overmindtech/aws-source#549

[dylanratcliffe]

Okay @DavidS-ovm I've got this as far as I can. We now have a ParseAWSProviders() method that will recursively find and parse all .tf files in a given directory, and return their config. We can then pass that to ConfigFromProvider() which converts the settings within the Terraform provider to an aws.Config object that can be passed directly to the source, we're following the logic in the Terraform AWS provider as closely as we can here and I'm fairly confident that we'll be able to get 1:1 support out of the box.

All we need now is to alter the overmind terraform command to use this instead of the default mechanism. However this is where I got stuck. I can't even work out how to begin if I'm honest. Obviously we'll need to modify InitializeSources() but looking at the existing logic around how we get the user input is melting my brain so I think we're going to have to either pair on it next week or get you to do this part for me.

Work-in-progress can be found in the terraform-auto-config branch