Does not support AssumeRole or MFA profiles

[phene]

AWSCurl does not support profiles that assume role (for cross-account access) or MFA.

[okigan]

@phene could you capture to this issue more details how profile with assume role is setup

[phene]

Please see the Using IAM Roles section of AWS CLI configuration guide. The role_arn and mfa_serial options are the relevant pieces of information that awscurl presently ignores.

[scraly]

We have the same needs and same issues.
We use a AWS user without policies & rights, we ask to this user to assume a role.
It's working corerctly with aws cli, but doen't work with awscurl.
Is it possible to solve this behavior?

[tdi]

+1 to this

[okigan]

assume role would require full support for boto (right?) which seems a bit large dependency to add into awscurl

as assume role just replaces env variables to the ones corresponding to the role have you considered just running aws sts assume-role ... before running awscurl?

as shown in: https://gist.github.com/brianredbeard/035ee1419bc38a0e2d854fb828d585d7

source <(AWS_PROFILE=redbeard aws sts assume-role --role-arn arn:aws:iam::123456789123:role/OrganizationAccountAccessRole --role-session-name "DevAccount" | jq -r '.Credentials | @sh "export AWS_SESSION_TOKEN=\(.SessionToken)\nexport AWS_ACCESS_KEY_ID=\(.AccessKeyId)\nexport AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey) "')

[tdi]

I do not think boto is big dependency - or let me rephrase it - problematic. Chance is when you use awscurl - you already have boto installed along with botocore etc. This way awscurl could simplify codebase and leveraging ofifcial's SDK config/env/profile readers and MFA and cross roles.

[okigan]

so trying to understand value proposition, if you have boto installed why use awscurl?

[phene]

The AWS CLI does not make it easy to create signed requests to Elasticsearch. That is the sole reason I use awscurl.

[legal90]

The issue was partially fixed by #63.
It works with env variables:

AWS_PROFILE=my-mfa-profile-with-assume-role awscurl $MY_HOST/test/v1/url

but doesn't work with CLI arg:

awscurl --profile my-mfa-profile-with-assume-role $MY_HOST/test/v1/url

Traceback (most recent call last):
  File "/Users/myuser/Library/Python/2.7/bin/awscurl", line 8, in <module>
    sys.exit(main())
  File "/Users/myuser/Library/Python/2.7/lib/python/site-packages/awscurl/awscurl.py", line 501, in main
    inner_main(sys.argv[1:])
  File "/Users/myuser/Library/Python/2.7/lib/python/site-packages/awscurl/awscurl.py", line 471, in inner_main
    args.profile)
  File "/Users/myuser/Library/Python/2.7/lib/python/site-packages/awscurl/awscurl.py", line 393, in load_aws_config
    access_key, secret_key, security_token = cred.access_key, cred.secret_key, cred.token
AttributeError: 'NoneType' object has no attribute 'access_key'

[yegeniy]

Following up on #44 (comment) and #44 (comment):

Until #102 is merged, you can do the following to invoke awscurl with an assumed role in a profile called "temp":

1. Generate commands to configure your ~/.aws/credentials file with a "temp" profile:

aws sts assume-role --role-arn arn:aws:iam::1234567890:role/TheRoleName --role-session-name "testing" | jq -r '.Credentials | "aws configure set aws_session_token \(.SessionToken) --profile temp\naws configure set aws_access_key_id \(.AccessKeyId) --profile temp\naws configure set aws_secret_access_key \(.SecretAccessKey) --profile temp\n"'

2. Execute those three commands (just copy paste them on your terminal). Verify that there is now a profile named [temp] in your ~/.aws/credentials file

3. Finally, invoke awscurl as awscurl --profile temp <url>.

---

Assuming roles in AWS and dealing with permissions gets complicated quickly. So, don't forget about using -v flag to debug (awscurl -v).

[john-aws]

Note: the AWS_PROFILE workaround above doesn't appear to work for the external credential_process case where the profile in ~/.aws/config looks like the following (and which works correctly for all AWS SDK clients e.g. awscli):

[profile my-profile]
output = json
region = us-east-1
credential_process = cli-get-credentials name@example.com --role my-iam-role

Running:

AWS_PROFILE=my-profile awscurl $URL

Results in:

ValueError: No access key is available

[erpel]

I was going to open this as a separate issue "Using aws-adfs credentials only works with 0.27" but when researching related tickets was unsure if this is useful as a separate bug as it introduces very little new information.

TLDR: This was working with 0.27.

I started using awscurl a short while ago when 0.27 was the current version which was installed. After upgrading to 0.29 today, my workflow broke.
When looking into it I realized that 0.26 also does not work.
I'm pretty confident that the change from PR #146 enabled my style of credentials.

What does the credential setup look like?

Using aws-adfs (https://github.com/venth/aws-adfs), logging in to a "federated role" in AWS account A and then using entries in ~/.aws/config to assume working roles in different accounts (B).

[profile playground]
role_arn = arn:aws:iam::<ID B>:role/admin-spoke-account-role
source_profile = default
region = eu-central-1

[default]
region = eu-central-1
output = json
adfs_config.ssl_verification = True
adfs_config.role_arn = arn:aws:iam::<ID A>:role/adfs-devops-role
adfs_config.adfs_host = adfs.company.com
adfs_config.session_duration = 28000
adfs_config.provider_id = urn:amazon:webservices
adfs_config.sspi = False
adfs_config.duo_factor = None
adfs_config.duo_device = None
adfs_config.adfs_user = username@company.com

aws-adfs puts temporary credentials into ~/.aws/credentials, the whole setup works fine with most tools that use boto3 and allow setting a profile.

This is the awscurl command I use to test: awscurl -v -X POST --region eu-central-1 --profile playground --service aps "$AMP_QUERY_ENDPOINT?query=up"

This whole situation seems quite complex, I found a number of related issues:

• Different workflow broken with 0.27: Credentials issue introduced in 0.27.0 when running on EC2 instance. #163
• Touches on a similar topic: Execute credential process to get temporary credentials. #102
• Reports the lack of support for an assume role workflow that's similar: Does not support AssumeRole or MFA profiles #44
• Reports --profile not working (closed as duplicate): --profile does nothing #89

It was quite surprising to me that setting the AWS_PROFILE variable works this differently than the --profile argument - I would not have tried this from the documentation alone. Maybe the visibility of this workaround can be improved.

[okigan]

@erpel wow - thanks for detailed report! I’ll have another look

[vazkarvishal]

Are there any plans to fix this? It is almost unusable without assumeRoles being supported.

[erpel]

It would probably be a very easy fix to introduce a new flag to "force" usage of Session(profile=profile) from botocore.session.
Integrating this in a way that automatically does the right thing for all use cases with minimal manual config is probably the hard part.
If the maintainers indicate that they'd be interested in a solution with an additional flag, I'd be happy to contribute an MR.

[okigan]

@erpel yes, PR, welcome -- I will have a look at integrating that.

[erpel]

Take a look at #172 please. It looks like there might be a possibility of more workflows working out of the box without a dedicated flag.