-
-
Notifications
You must be signed in to change notification settings - Fork 966
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature Request: Use aws credentials profile #48
Comments
I don't know if it will help, but here is some documentation from AWS that may be relevant .... http://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html "You can also use the SDK to select a profile by specifying os.Setenv("AWS_PROFILE", test-account) before constructing any service clients or by manually setting the credential provider, as shown in the following example":
|
Have you tried using the |
To be fair, this is now the second time we've gotten this request. In #40 (the first request), I argued that the credentials file is unique to each developer and therefore embedding a profile name in the But I'm wondering if @jgartrel has standardized the AWS profile names across the team? If that's true, then each developer does in fact have a common set of profile names, and I can see why it'd be convenient to explicitly declare which profile you want. Note also that this becomes especially useful in multi-account setups. |
@brikis98 I have and I do use AWS_PROFILE, but it is non-optimal because I have to specify credential keys in our TF configs when they differ from the AWS_PROFILE ones Usecase:
|
The plan was to have separate prod and stage profiles that would allow us to have separate remote_state data sources with environment specific shared info in them. This would allow us to not have our prod secrets exposed to all developers. But considering the vault provider just got merged down, we may not need to do this. |
Just an FYI, this is a very strong anti-pattern. Creating a unique IAM User account per developer is a pretty straightforward task, and means you no longer have to share keys. Keep in mind your security is generally only as strong as your weakest link, so using Vault to encrypt secrets would likely break down not because of Vault, but because a developer might leak their credentials, intentionally or otherwise.
Since Terraform supports the use of an optional |
So the use case is that everyone on the same team uses the same profile name in their |
That is the case ONLY for remote_state and terragrunt locking. All other AWS terraform operations are done to the target account, using developer supplied credentials via |
Would the team accept a pull request that looked something like this ... (Forgive me) as I have never programmed in GO before: |
@josh-padnick , I Agree that it is an anti-pattern, and it was only a temporary workaround. But, the use case is still I think a valid one: To have a different AWS account in which you store locks, store remote_state and share remote_state. We have several deployments that span multiple AWS accounts and providers, and we expect users to use the central lock and remote state storage account. In this account we can ensure the s3 bucket does encryption versioning, etc... and that state outputs can be shared amongst projects. |
@jgartrel I'm a bit concerned about using this profile only for the DynamoDB lock. What if you specify one profile in the Relying on the env var keeps things simple and consistent. You could easily write a shell script for your team that wraps terragrunt with the proper profile: export AWS_PROFILE=foo
exec terragrunt "$@" Perhaps we could even update terragrunt to scan your Terraform code and find the |
So I was thinking there are 3 main AWS account scenarios for terragrunt:
The crux of this issue is to give terragrunt users some options when they have multiple AWS accounts. Requiring users to have a specific account's credentials in the |
Today we do use 'make' as a wrapper to terragrunt, and we do set AWS_PROFILE, however, that has led to a number of unintended side-effects for multiple account situations. Ideally, I do not want to store shared credentials in our TF repos and I don't want our AWS admins accidentally provisioning resources in the centralized production account mistakenly when they forgot to set a "profile" on their AWS providers in TF. FWIW, I don't think it is wise to do "magic", such as scan the TF files for the "profile" section of the provider. I was rather looking for a simple and straight forward way to declaratively express it, and even require it. |
Somewhat related, terragrunt currently also ignores the AWS_DEFAULT_REGION environment variable and hardcodes a default region of Does this make sense to include in this thread or is this worth a separate issue? |
@jgartrel Thanks for the detailed explanation! Now I understand your use case better. It makes sense and it seems like something we should support, so I'd welcome the PR you suggested. The only question is whether the |
@davidski Please open a separate issue for that. The |
Another vote for being able to provide the credentials for the DynamoDB access. I orchestrate across multiple AWS accounts and each of my terraform provider configs references specific credentials files stored outside the various repos (secured using Linux file ACLs today, with a view of moving to Vault in the future). Being able to specify the shared_credentials_file would be great for dynamodb lock access configuration. PS - we also use assume_role in our Terraform provider blocks to gain access to different accounts, which breaks when the environment variables are set (they take precedence in Terraform over credentials files). |
FWIW - I wrap the running of Terragrunt in a rake task and have that export the correct environment variables and validate the account that's in use (actual vs expected) but we have a strange setup with multiple accounts per environment (2 x app cluster accounts and a "core" sidecar account) so orchestration is fun. |
I think I run into a similar issue. We want tp create a gitlab ci pipeline. It uses a vault and stores the AWS credentials as variables. I'm not able to access them when I start terragrunt. I don't realy get it with the default profile. I was testing with export AWS_PROFILE=default but still not working. How can I access the variables with the credentials? |
We have a common AWS account to store statefiles and manage locking for our terraform installations. It would be fantastic if you could allow the use of a specific "profile" in ~/.aws/credentials.
See: https://www.terraform.io/docs/state/remote/s3.html#profile
Configuration might look like this:
The credentials file might look like this:
It would also be nice to specify the credentials file location, the default obviously being "~/.aws/credentials"
The text was updated successfully, but these errors were encountered: