Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS assume role not working #11270

Closed
hkalyana opened this Issue Jan 18, 2017 · 18 comments

Comments

Projects
None yet
@hkalyana
Copy link

hkalyana commented Jan 18, 2017

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version -0.8.0

Run terraform -v to show the version. If you are not running the latest version of Terraform, please upgrade because your issue may have already been fixed.

Affected Resource(s)

Please list the resources as a list, for example:

  • aws_provider

Copy-paste your Terraform configurations here - for large Terraform configs,

provider "aws"
{
    region = "${var.region}"
    assume_role {
    role_arn = "arn:aws:iam::<trustedaccount>:role/trustedrole"
    }
 }

Expected Behavior

Expected the resources to be created on the trusting account

Actual Behavior

Received error The role "arn:aws:iam:::role/trustedrole" cannot be assumed.

  There are a number of possible causes of this - the most common are:
    * The credentials used in order to assume the role are invalid
    * The credentials do not have appropriate permission to assume the role
    * The role ARN is not valid

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

  1. Created iam role in the trusting account with the necessary permission .
  2. Created a policy on the trusted account with sts assume role permission and pointing the arm of the trusting account in the resource
    3)Attached the policy to the user
    4)Used the same role ARN of the trusting account in the terraform assume role

Important Factoids

The procedure I followed is working for me when I try to switch the roles in AWS GUI and I am able to create resources on the trusting account .

References

@aaomoware

This comment has been minimized.

Copy link

aaomoware commented Feb 17, 2017

I am facing the same issue. Below is my configuration for the aws provider.

provider "aws" {
token = "${var.aws_token}"
region = "${var.aws_region}"
access_key = "${data.vault_generic_secret.aws_access_key.data["value"]}"
secret_key = "${data.vault_generic_secret.aws_secret_key.data["value"]}"

assume_role {
	role_arn = "arn:aws:iam::0000000000:role/MyRoleNAme"
}

}

Like @hkalyana I can assume the same role and create resources from the AWS GUI. The aws_*_keys are correct.

@aaomoware

This comment has been minimized.

Copy link

aaomoware commented Feb 20, 2017

@hkalyana

I found a way round it. The terraform documentation on this issue is unclear.

follow: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html

remove the assume_role block; it's useless, for now.
assume_role {
role_arn = "arn:aws:iam::0000000000:role/MyRoleNAme"
}

the aws provider required attributes should be populated with the corresponding outputs you get when temporary credentials are requested from sts.amazonaws.com.

there are various ways you can make the call to sts. Anisble has a cool module for this specific purpose
http://docs.ansible.com/ansible/sts_assume_role_module.html

hope it helps you resolve your issue.

@alefauch

This comment has been minimized.

Copy link

alefauch commented Mar 6, 2017

Having exactly the same problem here.

Role delegation is working fine in both AWS Web Interface and aws CLI but not with terraform.

@aaomoware could you provide more details on your workaround ? If I get your post right, looks like you have to use an external tool to get temporary credentials and then feed them to terraform.

@aaomoware

This comment has been minimized.

Copy link

aaomoware commented Mar 6, 2017

@alefauch

  • hosts: localhost
    tasks:
    • name: get sessions
      sts_assume_role:
      mfa_token: "{{mfa_token}}"
      region: "eu-west-1"
      role_arn: "arn:aws:iam::0000000000000:role/RoleName"
      mfa_serial_number: "arn:aws:iam::000000000000:mfa/username"
      role_session_name: "NameofTheCreatedSession"
      register: assume_role

    • name: write credentials to vault
      shell: vault write secret/{{item.path}} value={{item.value}}
      environment:
      VAULT_ADDR: "{{ansible_env.VAULT_ADDR}}"
      VAULT_TOKEN: "{{ansible_env.VAULT_TOKEN}}"
      with_items:

      • { path: "RUDEBOY/AWS/access_key", value: "{{assume_role.sts_creds.access_key}}" }
      • { path: "RUDEBOY/AWS/secret_key", value: "{{assume_role.sts_creds.secret_key}}" }
      • { path: "RUDEBOY/AWS/session_token", value: "{{assume_role.sts_creds.session_token}}" }

I use vault to store my credentials....hence im writing to vault. Terraform supports reading/writing from and to vault.

You can use the aws api, wrapped in a script, to accomplish the above steps; i use ansible because it's easier, works well, and it's already in use in my project to configure instances brought up by terraform

You can add the ansible run to a cron job, every 1hr. The max duration for the temporary keys is 3600(1hr)

Notes:
http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html
http://docs.ansible.com/ansible/sts_assume_role_module.html

@jfriis

This comment has been minimized.

Copy link

jfriis commented Mar 8, 2017

Has anyone actually gotten assume_role to work? I'm trying to use it and have been seeing the same error as above. Would love a working example of the feature

@alefauch

This comment has been minimized.

Copy link

alefauch commented Mar 9, 2017

@jfriis

I have a working solution with Terraform and AWS with assume role.

Note that I'm not using Terraform to get a session token or to assume the role.

First you want to have a session, especialy if you have MFA activated like me.

aws sts get-session-token --profile <YOUR_AWS_SESSION_PROFILE> --duration-seconds <YOUR_AWS_SESSION_DURATION> --serial-number <YOUR_AWS_MFA_SERIAL> --token-code <YOUR_AWS_MFA_VALUE> --output text

<YOUR_AWS_SESSION_PROFILE> is the same one that you have in ~/.aws/config usually "default"
<YOUR_AWS_SESSION_DURATION> is the session duration in seconds, I use 36000 (10 hours)
<YOUR_AWS_MFA_SERIAL> is the serial of you MFA if you use one this should look like arn:aws:iam::ACCOUNT_ID:mfa/YOUR_USERNAME
<YOUR_AWS_MFA_VALUE> is the current MFA value of your token (usually 6 digits)

Note that you can skip --serial-number and --token-code if you are not using a MFA.

This command will return to you some tokens:
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN

You have to save them and source them in order to use them. The easy way is to store them in a file like this and the "source" the file in question in order to have those variables set in your environment (this is for Linux):

export AWS_ACCESS_KEY_ID=AAAAADDDDDAAAAA
export AWS_SECRET_ACCESS_KEY=BLABLABLA
export AWS_SESSION_TOKEN=LONNNGGGG_TOKEN

Then you can call the assume role:

aws sts assume-role --role-arn <YOUR_AWS_ASSUMRED_ROLE_ARN> --role-session-name <YOUR_AWS_ASSUMRED_ROLE_SESSION_NAME> --duration <YOUR_AWS_ASSUMED_ROLE_DURATION> --output text

<YOUR_AWS_ASSUMRED_ROLE_ARN>: Will look like arn:aws:iam::ACCOUNT_ID:role/YOUR_ROLENAME
<YOUR_AWS_ASSUMRED_ROLE_SESSION_NAME>: you can put YOUR_ROLENAME as value
<YOUR_AWS_ASSUMED_ROLE_DURATION>: is the assume role credentials duration in seconds, I use 3600 (1 hour) which is the maximum allowed value.

This command will return to you the same tokens but with new values:
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN

Like the previous ones, you have to save them and source them in order to use them. The easy way is to store them in a file and the "source" the file in question in order to have those variables set in your environment.

Note that you should save those credentials in a separate file than the get-session-token credentials.

Your Terraform provider configuration should look this this:

provider "aws" {
  region     = "${var.region}"
  access_key = "${var.aws_access_key}"
  secret_key = "${var.aws_secret_key}"
  token = "${var.aws_token}"
}

You have to populate the variables using the assume-role environment variables AWS_ACCESS_KEY_ID for access_key, AWS_SECRET_ACCESS_KEY for secret_key and AWS_SESSION_TOKEN for token.

Now you should be able to lauch Terraform without errors.

The reason you have to save those assume-role and get-session-token credentials separately is that if your get-session-token is still valid, you can directly source the get-session-token variables and call assume-role with them.

Hope I was clear enough ;)

@david-kli

This comment has been minimized.

Copy link

david-kli commented Mar 16, 2017

@jfriis In case you were still looking for confirmation, I was able to successfully use the assume role functionality with new 0.9 release.

I have a single entry in the aws credentials file which contains a personal IAM key from a Jump style account (no other AWS resources are created or allowed here except IAM resources).
~/.aws/credentials

[my-iamjump-account]
aws_access_key_id = <ACCESS_KEY>
aws_secret_access_key = <SECRET_KEY>
region = us-east-1

In my terraform project, I created a backend.tf file which contained the new remote backend config and the AWS provider config. The S3 bucket is created in the destination AWS account and all resources specified in the project were properly created.

terraform {
  backend "s3" {
    bucket = "my-terraform-bucket"
    key = "my-project/terraform.tfstate"
    region = "us-east-1"
    profile = "my-iamjump-account"
    role_arn = "arn:aws:iam::<DESTINATION_AWS_ACCOUNT_NUMBER>:role/user/myadminrole"
 }
}

provider "aws" {
  profile = "my-iamjump-account"
  region = "us-east-1"
  assume_role {
    role_arn = "arn:aws:iam::<DESTINATION_AWS_ACCOUNT_NUMBER>:role/user/myadminrole"
    session_name = "terraform"
  }
}

This setup allowed me to terraform init and terraform apply without any AWS environment variables configured.

@agrmckinnon

This comment has been minimized.

Copy link

agrmckinnon commented Apr 6, 2017

@david-kli
In the example above is the role being assumed in the same AWS account as the user?

I have been able to successfully do this with 0.9 (great), but cannot do this if the user's access keys belong to Account A and the role to be assumed is in Account B I receive the following

(these are 'bona fide' accounts and roles - which allow the role to be assumed successfully via the AWS Console - switch roles functionality)

* module.aws_cloudtrail.provider.aws.${var.alias}: 
The role "arn:aws:iam::<ACCOUNT_B_ACCTNO>:role/<ROLE2ASSUME>" cannot be assumed.

  There are a number of possible causes of this - the most common are:
    * The credentials used in order to assume the role are invalid
    * The credentials do not have appropriate permission to assume the role
    * The role ARN is not valid

My use case is that I have a hub and spoke scenario where I want to use the hub account to build out infrastructure on spoke accounts using a user from the hub account who has permissions to assume a role in each spoke account.

Would appreciate if you or anyone can confirm if you can / cannot do this across account.

@david-kli

This comment has been minimized.

Copy link

david-kli commented Apr 6, 2017

@agrmckinnon
My example had the user and role in separate accounts.

I use a similar hub and spoke setup which requires a minimal amount of terraforming using the root account during initial account setup. After that, the build of the build out is accomplished with assumed roles.

The aws jump account in my setup only contains IAM users and group definitions. I create one group here per unique remote account & role combination. The group policies contain the sts:assumeRole for the destination account/role arn. Each remote account has the roles defined with a trust relationship defined back to the jump account.

Your error message makes me wonder if something is getting lost with the provider usage inside a module. You mentioned you can assume the role with the AWS console. Can you also perform similar actions with the aws cli? It might also be worthwhile to work up a test which does not use a provider inside in a module.

@agrmckinnon

This comment has been minimized.

Copy link

agrmckinnon commented Apr 6, 2017

@david-kli
Thank you for the quick feedback and support - After a thorough review and a minor tweak I have this working now

In my case we also had the role available to the user for console access and I had added the condition to the trust policy

,
    "Condition": {"Bool": {"aws:MultiFactorAuthPresent": "true"}}

This enforced that the user had to have MFA'd via the console before assuming the role. This had the detrimental effect of receiving the error, as defined in the previous post from me, when assuming the role via terraform.

After removing the condition from the trust policy I can successfully assume the role on the account on the spoke account via Terraform.

Our goal is to have our production activities undertaken by an automation tool using instance profiles and assuming roles.

For development there will be an element of running terraform with assume roles from an engineers own credentials.

In the short/medium term I plan to separate the engineers user into two users to avoid future confusion

  • One to access the console with MFA
  • One to access the api's without MFA

Thanks Again

@FransUrbo

This comment has been minimized.

Copy link

FransUrbo commented Apr 30, 2017

I'm having problem getting assume role to work on v0.9.4:

~/.aws/config

Doesn't exist - but I'm hoping to put only the region here and get rid of it from the *.tf files

~/.aws/credentials:

[root]
# Account: ACCOUNT1_ID
aws_access_key_id = ACCOUNT1_ACCESSKEY
aws_secret_access_key = ACCOUNT1_SECRETKEY

[core]
# Account: ACCOUNT2_ID
aws_access_key_id = ACCOUNT2_ACCESSKEY
aws_secret_access_key = ACCOUNT2_SECRETKEY

[test]
# Account: ACCOUNT3_ID
role_arn = arn:aws:iam::ACCOUNT3_ID:role/Cross_Account
source_profile = core

provider.tf

provider "aws" {
  region  = "eu-west-1"
  profile = "core"
  alias   = "core"
}

provider "aws" {
  region  = "eu-west-1"
  profile = "test"
}

remote.tf

data "terraform_remote_state" "base" {
  backend   = "s3"
  config {
    bucket  = "my-terraform-bucket"
    key     = "terraform-base.tfstate"
    region  = "eu-west-1"
    profile = "root"
  }
}

data "terraform_remote_state" "core" {
  backend   = "s3"
  config {
    bucket  = "my-terraform-bucket"
    key     = "terraform-core.tfstate"
    region  = "eu-west-1"
    profile = "root"
  }
}

data "terraform_remote_state" "test" {
  backend   = "s3"
  config {
    bucket  = "my-terraform-bucket"
    key     = "terraform-test.tfstate"
    region  = "eu-west-1"
    profile = "root"
  }
}

The following shell commands work just fine:

$ aws iam list-users --profile root
$ aws iam list-users --profile core
$ aws iam list-users --profile test

So the aws command sees nothing wrong with my credentials file.

However, terraform_0.9.4 refresh gives me:

data.terraform_remote_state.core: Refreshing state...
data.terraform_remote_state.test: Refreshing state...
data.terraform_remote_state.base: Refreshing state...
data.aws_caller_identity.core: Refreshing state...
Error refreshing state: 1 error(s) occurred:

* provider.aws: No valid credential sources found for AWS Provider.
  Please see https://terraform.io/docs/providers/aws/index.html for more information on
  providing credentials for the AWS Provider

For testing, I have disabled the MultiFactorAuthPresent requirement in arn:aws:iam::ACCOUNT3_ID:role/Cross_Account.

@drdamour

This comment has been minimized.

Copy link

drdamour commented May 9, 2017

i'm getting what @FransUrbo is getting in tf 9.4....is MFA supposed to work (will TF prompt my for my MFA token?)

@ashb

This comment has been minimized.

Copy link
Contributor

ashb commented May 25, 2017

@drdamour No, at current it will not :(

@FransUrbo

This comment has been minimized.

Copy link

FransUrbo commented May 25, 2017

For me, it doesn't work even if i disable MFA everywhere...

@drdamour

This comment has been minimized.

Copy link

drdamour commented May 26, 2017

is the problem people are running into possibly that you can't assume the role if you are already in the role? that's what we run into from time to time.

I can assume roles from my main session...and thus assume_role works fine...but if i use aws cli to assume the role then running terraform fails on teh assume_role call...as my new role does not have permissions to assume itself.

seems like terraform should check if the session role is already the role defined in assume_role before it tries to assume it...however i don't know if the aws api allows you to get your current role..and i don't know go so can't really test...anyone got some skills?

looks like you CAN get the current role http://docs.aws.amazon.com/cli/latest/reference/iam/get-role.html

@FransUrbo

This comment has been minimized.

Copy link

FransUrbo commented May 26, 2017

@drdamour If you look at my code at the top, you'll see that's not what's happening for me. At least, I don't think so.

I have my core account, which then assumes a role in the other account.

@ojkelly

This comment has been minimized.

Copy link

ojkelly commented Jun 13, 2017

In case anyone comes across this, I've adapted a work around from kops

Save the following to a file called terraform-mfa.

Then make it executable with $ chmod +x terraform-mfa.

Then put it somewhere in your $PATH: $ mv terraform-mfa /usr/local/bin

Then call terraform-mfa instead of terraform.

#!/usr/bin/env bash

# From https://github.com/kubernetes/AWS/issues/226#issuecomment-278879348
# and https://github.com/kubernetes/AWS/blob/master/docs/mfa.md

# set -euo pipefail

main() {
	local role_arn="${AWS_MFA_ROLE_ARN:-}"
	local serial_number="${AWS_MFA_ARN:-}"
	local token_code

	if [ -z "${role_arn}" ]; then
		echo "Set the AWS_MFA_ROLE_ARN environment variable" 1>&2
		return 1
	fi

	if [ -z "${serial_number}" ]; then
		echo "Set the AWS_MFA_ARN environment variable" 1>&2
		return 1
	fi

	echo -n "Enter MFA Code: "
	read -s token_code

	# NOTE: The keys should not be exported as AWS_ACCESS_KEY_ID
	# or AWS_SECRET_ACCESS_KEY_ID. This will not work. They
	# should be exported as other names which can be used below. This prevents
	# them from incorrectly being picked up from libraries or commands.
	temporary_credentials="$(aws \
		sts assume-role \
		--role-arn="${role_arn}" \
		--serial-number="${serial_number}" \
		--token-code="${token_code}" \
		--role-session-name="terraform-access"
	)"

	unset AWS_PROFILE

	export "AWS_ACCESS_KEY_ID=$(echo "${temporary_credentials}" | jq -re '.Credentials.AccessKeyId')"
	export "AWS_SECRET_ACCESS_KEY=$(echo "${temporary_credentials}" | jq -re '.Credentials.SecretAccessKey')"
	export "AWS_SESSION_TOKEN=$(echo "${temporary_credentials}" | jq -re '.Credentials.SessionToken')"

	exec terraform "$@"
}


main "$@"
@catsby

This comment has been minimized.

Copy link
Member

catsby commented Jun 14, 2017

Hey all – this issue has been migrated to the new AWS provider repo, here: terraform-providers/terraform-provider-aws#472

I've posted a follow up comment there with an example config, if anyone can take a look, thanks!

terraform-providers/terraform-provider-aws#472 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.