New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Assume role with EC2 instance profile as the source profile #1390

Closed
pgcp opened this Issue Jun 15, 2015 · 80 comments

Comments

Projects
None yet
@pgcp
Contributor

pgcp commented Jun 15, 2015

Right now you can execute commands using credentials from one of these sources: root credentials, IAM credentials, temporary credentials from an EC2 instance profile, and temporary credentials from assuming a role via IAM credentials. I would like to execute commands by using temporary credentials from assuming a role via the EC2 instance profile.

I need this ability because I'm using two AWS accounts and I'm using an EC2 instance to run AWS CLI commands against both accounts. The EC2 instance profile allows me to perform tasks for one account, but I need to assume a cross-account role to perform tasks for the other account. Unfortunately there is no way to get AWS CLI to assume the cross-account role even though the EC2 instance profile has permissions to assume that role.

I tried removing the source_profile property from my role-based profile in hopes that the source_profile would use the instance profile, but that failed. After looking at AssumeRoleProvider in awscli/customizations/assumerole.py, I see that AWS CLI can only assume a role if the source profile has actual credentials in the config file. So currently that excludes any use of an instance profile to assume a different role.

@kyleknap

This comment has been minimized.

Member

kyleknap commented Jun 16, 2015

Marking as feature request.

@pgcp

This comment has been minimized.

Contributor

pgcp commented Jun 18, 2015

I've implemented a change to assumerole.py that accomplishes the goal in the feature request. In my implementation, AssumeRoleProvider falls back to directly calling InstanceMetadataProvider if the source profile is missing. Although this meets my use-case, I'm not sure if this is the approach that others would be pleased with. For one thing, there is no fallback to environment variables or another role. And the InstanceMetadataProvider created is actually a separate instance from the one created in botocore's create_credential_resolver() function.

I was hoping to implement it in such a way that that roles could be assumed to an arbitrary depth (a role is assumed from a role which is assumed from a role... etc.) and the ending source profile could be any valid credential provider (Environment variables, IAM credentials, instance metadata, etc.). The current credential resolver structure in botocore doesn't make it easy to do this. The create_credential_resolver() function is written with the assumption that you can create the credential resolver with the one profile you've selected and you won't need another.

Should I submit my pull request for the simplified version that just directly calls the InstanceMetadataProvider, or is a more robust implementation required? The better version probably would require some changes to botocore.

@ColinHebert

This comment has been minimized.

ColinHebert commented Nov 26, 2015

@pgcp Would you mind submitting your solution? It should get the ball rolling for this issue.

@bflad

This comment has been minimized.

bflad commented Dec 2, 2015

Just ran into this myself! Albeit without visible text around 'aws_access_key_id' as seen below in the pip installed AWS CLI 1.9.9 on an Ubuntu 14.04 machine:

$ aws --profile otheraccount ec2 describe-instances

'aws_access_key_id'
$ cat ~/.aws/config
[default]
region=us-east-1

[profile otheraccount]
source_profile = default
role_arn = arn:aws:iam::############:role/my_other_accounts_role
@ColinHebert

This comment has been minimized.

ColinHebert commented Jan 6, 2016

@kyleknap, is it possible to consider adding this issue to the roadmap?

@kyleknap

This comment has been minimized.

Member

kyleknap commented Jan 6, 2016

@ColinHebert This is a feature we would like to add as we have gotten multiple requests for it and it would be useful. However, we do not have a timetable for when it will get done. When we start making progress on this feature, the issue will be updated.

@craigday

This comment has been minimized.

craigday commented Feb 23, 2016

This is just a "me too" comment. Ran into exactly this problem today. Please implement this functionality. The model (at least) seems to suggest it should work this way.

@s2maki

This comment has been minimized.

s2maki commented Mar 1, 2016

+1. Just hit this problem myself. @pgcp please do submit your changes as a pull request. Even if it ultimately gets rejected, it'll help get the correct change made. I looked at assumerole.py, and it wasn't obvious to me what that change would look like.

@josefsachsconning

This comment has been minimized.

josefsachsconning commented Mar 10, 2016

Echoing s2maki, @pgcp it would be great to see a pull request. Thanks in advance.

@reppard

This comment has been minimized.

reppard commented Mar 16, 2016

I've ran into this myself. Just my +1 here.

@inhumantsar

This comment has been minimized.

inhumantsar commented Mar 18, 2016

+1

@gene1wood

This comment has been minimized.

gene1wood commented Mar 19, 2016

While you wait for this to be implemented, feel free to use my somewhat odd workaround. It works great for me and unblocks me from waiting on this feature

https://gist.github.com/gene1wood/34b02fa3091e184e1997

# Usage : . $0 ROLE_ARN [PARENT_PROFILE_NAME]
#
# This tool will generate temporary credentials for an assumed role, save
# those ephemeral credentials in the awscli config and set the alias of 
# "aaws" to use this new ephemeral awscli profile
#
# Examples
#   . $0 arn:aws:iam::123456789012:role/ExampleRole
#   aaws ec2 describe-instances
# or
#   . $0 arn:aws:iam::234567890123:role/ExampleRole staging
#   aaws --region us-west-2 ec2 describe-instances
@pgcp

This comment has been minimized.

Contributor

pgcp commented Mar 23, 2016

The code for AssumeRoleProvider was moved to botocore since this issue was created. Because of that I've submitted a pull request to that repo instead of this one.

The code in the pull request would change the behavior of AWS CLI so that if a profile had a role_arn but not a source_profile then credentials would be retrieved from the EC2 instance profile (i.e. credentials via instance metadata service). Previously an error was returned if the source_profile was missing.

I'd much prefer having a configuration profile that explicitly declares that its credentials are from the EC2 instance profile and then reference that profile as the source_profile, but perhaps that can be a separate feature request. In the meantime, I'm just trying to fill a gap in functionality that several of us need.

@reppard

This comment has been minimized.

reppard commented May 6, 2016

We created a gem as a work around for this issue:
https://github.com/manheim/awssume

@rodlogic

This comment has been minimized.

rodlogic commented Jun 13, 2016

+1

1 similar comment
@brianantonelli

This comment has been minimized.

brianantonelli commented Jul 15, 2016

+1

@copumpkin

This comment has been minimized.

copumpkin commented Aug 4, 2016

Finding myself wanting this, after wanting it in botocore and boto and boto3 too. In general, AssumeRole and friends are hugely powerful tools, but all the APIs and tooling around them feels pretty limited.

@homer-editionlingerie

This comment has been minimized.

homer-editionlingerie commented Sep 15, 2016

Would request that too. Didn't realise it's not supported. When EC2 instance profile was encouraged, such a feature is nearly a must.

@kachenjr

This comment has been minimized.

kachenjr commented Sep 22, 2016

+1

1 similar comment
@peter-playbasis

This comment has been minimized.

peter-playbasis commented Sep 28, 2016

+1

@james-huston

This comment has been minimized.

james-huston commented Sep 29, 2016

+1 and a case of your favorite beer.

Until then @gene1wood has got some hackery that helped me get something going for my use case. Writing a local profile based on the values coming back make your temp credentials available to other commands.

#1390 (comment)

@gene1wood

This comment has been minimized.

gene1wood commented Sep 30, 2016

@james-huston thanks for the shoutout. You can also find that bash hack as well as a python version here : https://github.com/gene1wood/aws_assume_roles

@philmcmahon

This comment has been minimized.

philmcmahon commented Nov 25, 2016

+1

@dmitry-livchak

This comment has been minimized.

dmitry-livchak commented Dec 1, 2016

+1 An absolute must have! It is even advertised as a feature in IAM CLI documentation:

You can run an AWS CLI command using a role only when you are signed in as an IAM user, as an the section called “Identity Providers and Federation” (the section called “About SAML 2.0 Federation” or the section called “About Web Identity Federation”) already using a role, or when run from within an Amazon EC2 instance that is attached to a role through its instance profile.

@RAR

This comment has been minimized.

RAR commented Aug 17, 2017

+1

1 similar comment
@FraaJad

This comment has been minimized.

FraaJad commented Sep 2, 2017

+1

@aaugustin

This comment has been minimized.

aaugustin commented Sep 6, 2017

I'm using the following workaround:

sts_client = boto3.client(
    service_name='sts',
    region_name=settings.AWS_REGION_NAME,
)
response = sts_client.assume_role(
    RoleArn=settings.AWS_ASSUME_ROLE_ARN,
    RoleSessionName=settings.AWS_ASSUME_ROLE_SESSION_NAME,
)
credentials = response['Credentials']
credentials = {
    'aws_access_key_id': credentials['AccessKeyId'],
    'aws_secret_access_key': credentials['SecretAccessKey'],
    'aws_session_token': credentials['SessionToken'],
}

client = boto3.client(
    service_name='sns',
    region_name=settings.AWS_REGION_NAME,
    **credentials,
)

Unfortunately even more code is required to refresh credentials regularly — they expire after one hour.

@BastianVoigt

This comment has been minimized.

BastianVoigt commented Sep 7, 2017

+1

4 similar comments
@apurv

This comment has been minimized.

apurv commented Sep 7, 2017

+1

@MattTunny

This comment has been minimized.

MattTunny commented Sep 18, 2017

+1

@marktb1

This comment has been minimized.

marktb1 commented Sep 25, 2017

+1

@robcoward

This comment has been minimized.

robcoward commented Sep 28, 2017

+1

@techfishio

This comment has been minimized.

techfishio commented Sep 28, 2017

+1 two years in and still no fix, what gives?

@qoomon

This comment has been minimized.

qoomon commented Oct 2, 2017

+1

3 similar comments
@antongurov

This comment has been minimized.

antongurov commented Oct 6, 2017

+1

@MatthewDaize

This comment has been minimized.

MatthewDaize commented Oct 6, 2017

+1

@corlettb

This comment has been minimized.

corlettb commented Oct 16, 2017

+1

@grahamjenson

This comment has been minimized.

grahamjenson commented Oct 16, 2017

We just released a tool assume-role (https://github.com/coinbase/assume-role) that can export the credentials without using a profile. We use it to manage loads of roles across dozens of accounts. Hope this helps. PRs are also welcome!

@mkostrikin

This comment has been minimized.

mkostrikin commented Oct 24, 2017

+1

diurnalist added a commit to diurnalist/prometheus that referenced this issue Oct 24, 2017

Allow getting credentials via EC2 role
This is subtly different than the existing `role_arn` solution, which
allows Prometheus to assume an IAM role given some set of credentials
already in-scope. With EC2 roles, one specifies the role at instance
launch time (via an instance profile.) The instance then exposes
temporary credentials via its metadata. The AWS Go SDK exposes a
credential provider that polls the instance metadata endpoint[1]
already, so we can simply use that and it will take care of renewing the
credentials when they expire.

Without this, if this is being used inside EC2, it is difficult to
cleanly allow the use of STS credentials. One has to set up a proxy role
that can assume the role you really want, and launch the EC2 instance
with the proxy role. This isn't very clean, and also doesn't seem to be
supported very well[2].

[1]:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
[2]: aws/aws-cli#1390

diurnalist added a commit to diurnalist/prometheus that referenced this issue Oct 24, 2017

Allow getting credentials via EC2 role
This is subtly different than the existing `role_arn` solution, which
allows Prometheus to assume an IAM role given some set of credentials
already in-scope. With EC2 roles, one specifies the role at instance
launch time (via an instance profile.) The instance then exposes
temporary credentials via its metadata. The AWS Go SDK exposes a
credential provider that polls the [instance metadata endpoint][1]
already, so we can simply use that and it will take care of renewing the
credentials when they expire.

Without this, if this is being used inside EC2, it is difficult to
cleanly allow the use of STS credentials. One has to set up a proxy role
that can assume the role you really want, and launch the EC2 instance
with the proxy role. This isn't very clean, and also doesn't seem to be
[supported very well][2].

[1]:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
[2]: aws/aws-cli#1390

brian-brazil added a commit to prometheus/prometheus that referenced this issue Oct 25, 2017

Feature: Allow getting credentials via EC2 role (#3343)
* Allow getting credentials via EC2 role

This is subtly different than the existing `role_arn` solution, which
allows Prometheus to assume an IAM role given some set of credentials
already in-scope. With EC2 roles, one specifies the role at instance
launch time (via an instance profile.) The instance then exposes
temporary credentials via its metadata. The AWS Go SDK exposes a
credential provider that polls the [instance metadata endpoint][1]
already, so we can simply use that and it will take care of renewing the
credentials when they expire.

Without this, if this is being used inside EC2, it is difficult to
cleanly allow the use of STS credentials. One has to set up a proxy role
that can assume the role you really want, and launch the EC2 instance
with the proxy role. This isn't very clean, and also doesn't seem to be
[supported very well][2].

[1]:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
[2]: aws/aws-cli#1390

* Automatically try to detect EC2 role credentials

The `Available()` function exposed on ec2metadata returns a simple
true/false if the ec2 metadata is available. This is the best way to
know if we're actually running in EC2 (which is the only valid use-case
for this credential provider.)

This allows this to "just work" if you are using EC2 instance roles.
@MarkRose

This comment has been minimized.

MarkRose commented Nov 6, 2017

+1

@lorengordon

This comment has been minimized.

Contributor

lorengordon commented Nov 14, 2017

@aaugustin

This comment has been minimized.

aaugustin commented Nov 22, 2017

Thank you very much!

@iammr-schuck

This comment has been minimized.

iammr-schuck commented Dec 11, 2017

+1, for whatever that's worth. Trying to get a cross acct code pipeline working with codecommit>codebuild>codepipeline>deploy to Lambda but these run under roles in their separated accounts, and without this feature it just doesn't work cleanly.

@feraudet

This comment has been minimized.

feraudet commented Jan 17, 2018

+1

@mr-brody

This comment has been minimized.

mr-brody commented Jan 29, 2018

As mentioned in the referenced link[1] there a recent feature where you can create a profile and specify the credential source as ec2 instance metadata which resolves this issue[2].

[1]boto/botocore#1313
[2]https://docs.aws.amazon.com/cli/latest/topic/config-vars.html

The interesting configuration part is below:

In ~/.aws/config

[profile crossaccount]
role_arn=arn:aws:iam:...
credential_source=Ec2InstanceMetadata

@sureshgans

This comment has been minimized.

sureshgans commented Apr 12, 2018

cross account roles works if an instanceprofile is associated to an EC2 instance and trust is established between account1 and account2 for assuming role.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment