Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Assume role with EC2 instance profile as the source profile #1390

Closed
pgcp opened this issue Jun 15, 2015 · 80 comments
Closed

Feature request: Assume role with EC2 instance profile as the source profile #1390

pgcp opened this issue Jun 15, 2015 · 80 comments

Comments

@pgcp
Copy link
Contributor

@pgcp pgcp commented Jun 15, 2015

Right now you can execute commands using credentials from one of these sources: root credentials, IAM credentials, temporary credentials from an EC2 instance profile, and temporary credentials from assuming a role via IAM credentials. I would like to execute commands by using temporary credentials from assuming a role via the EC2 instance profile.

I need this ability because I'm using two AWS accounts and I'm using an EC2 instance to run AWS CLI commands against both accounts. The EC2 instance profile allows me to perform tasks for one account, but I need to assume a cross-account role to perform tasks for the other account. Unfortunately there is no way to get AWS CLI to assume the cross-account role even though the EC2 instance profile has permissions to assume that role.

I tried removing the source_profile property from my role-based profile in hopes that the source_profile would use the instance profile, but that failed. After looking at AssumeRoleProvider in awscli/customizations/assumerole.py, I see that AWS CLI can only assume a role if the source profile has actual credentials in the config file. So currently that excludes any use of an instance profile to assume a different role.

@kyleknap
Copy link
Member

@kyleknap kyleknap commented Jun 16, 2015

Marking as feature request.

@pgcp
Copy link
Contributor Author

@pgcp pgcp commented Jun 18, 2015

I've implemented a change to assumerole.py that accomplishes the goal in the feature request. In my implementation, AssumeRoleProvider falls back to directly calling InstanceMetadataProvider if the source profile is missing. Although this meets my use-case, I'm not sure if this is the approach that others would be pleased with. For one thing, there is no fallback to environment variables or another role. And the InstanceMetadataProvider created is actually a separate instance from the one created in botocore's create_credential_resolver() function.

I was hoping to implement it in such a way that that roles could be assumed to an arbitrary depth (a role is assumed from a role which is assumed from a role... etc.) and the ending source profile could be any valid credential provider (Environment variables, IAM credentials, instance metadata, etc.). The current credential resolver structure in botocore doesn't make it easy to do this. The create_credential_resolver() function is written with the assumption that you can create the credential resolver with the one profile you've selected and you won't need another.

Should I submit my pull request for the simplified version that just directly calls the InstanceMetadataProvider, or is a more robust implementation required? The better version probably would require some changes to botocore.

@ColinHebert
Copy link

@ColinHebert ColinHebert commented Nov 26, 2015

@pgcp Would you mind submitting your solution? It should get the ball rolling for this issue.

@bflad
Copy link

@bflad bflad commented Dec 2, 2015

Just ran into this myself! Albeit without visible text around 'aws_access_key_id' as seen below in the pip installed AWS CLI 1.9.9 on an Ubuntu 14.04 machine:

$ aws --profile otheraccount ec2 describe-instances

'aws_access_key_id'
$ cat ~/.aws/config
[default]
region=us-east-1

[profile otheraccount]
source_profile = default
role_arn = arn:aws:iam::############:role/my_other_accounts_role
@ColinHebert
Copy link

@ColinHebert ColinHebert commented Jan 6, 2016

@kyleknap, is it possible to consider adding this issue to the roadmap?

@kyleknap
Copy link
Member

@kyleknap kyleknap commented Jan 6, 2016

@ColinHebert This is a feature we would like to add as we have gotten multiple requests for it and it would be useful. However, we do not have a timetable for when it will get done. When we start making progress on this feature, the issue will be updated.

@craigday
Copy link

@craigday craigday commented Feb 23, 2016

This is just a "me too" comment. Ran into exactly this problem today. Please implement this functionality. The model (at least) seems to suggest it should work this way.

@s2maki
Copy link

@s2maki s2maki commented Mar 1, 2016

+1. Just hit this problem myself. @pgcp please do submit your changes as a pull request. Even if it ultimately gets rejected, it'll help get the correct change made. I looked at assumerole.py, and it wasn't obvious to me what that change would look like.

@josefsachsconning
Copy link

@josefsachsconning josefsachsconning commented Mar 10, 2016

Echoing s2maki, @pgcp it would be great to see a pull request. Thanks in advance.

@reppard
Copy link

@reppard reppard commented Mar 16, 2016

I've ran into this myself. Just my +1 here.

@inhumantsar
Copy link

@inhumantsar inhumantsar commented Mar 18, 2016

+1

@gene1wood
Copy link

@gene1wood gene1wood commented Mar 19, 2016

While you wait for this to be implemented, feel free to use my somewhat odd workaround. It works great for me and unblocks me from waiting on this feature

https://gist.github.com/gene1wood/34b02fa3091e184e1997

# Usage : . $0 ROLE_ARN [PARENT_PROFILE_NAME]
#
# This tool will generate temporary credentials for an assumed role, save
# those ephemeral credentials in the awscli config and set the alias of 
# "aaws" to use this new ephemeral awscli profile
#
# Examples
#   . $0 arn:aws:iam::123456789012:role/ExampleRole
#   aaws ec2 describe-instances
# or
#   . $0 arn:aws:iam::234567890123:role/ExampleRole staging
#   aaws --region us-west-2 ec2 describe-instances
@pgcp
Copy link
Contributor Author

@pgcp pgcp commented Mar 23, 2016

The code for AssumeRoleProvider was moved to botocore since this issue was created. Because of that I've submitted a pull request to that repo instead of this one.

The code in the pull request would change the behavior of AWS CLI so that if a profile had a role_arn but not a source_profile then credentials would be retrieved from the EC2 instance profile (i.e. credentials via instance metadata service). Previously an error was returned if the source_profile was missing.

I'd much prefer having a configuration profile that explicitly declares that its credentials are from the EC2 instance profile and then reference that profile as the source_profile, but perhaps that can be a separate feature request. In the meantime, I'm just trying to fill a gap in functionality that several of us need.

@reppard
Copy link

@reppard reppard commented May 6, 2016

We created a gem as a work around for this issue:
https://github.com/manheim/awssume

@rodlogic
Copy link

@rodlogic rodlogic commented Jun 13, 2016

+1

1 similar comment
@brianantonelli
Copy link

@brianantonelli brianantonelli commented Jul 15, 2016

+1

@copumpkin
Copy link

@copumpkin copumpkin commented Aug 4, 2016

Finding myself wanting this, after wanting it in botocore and boto and boto3 too. In general, AssumeRole and friends are hugely powerful tools, but all the APIs and tooling around them feels pretty limited.

@homer-editionlingerie
Copy link

@homer-editionlingerie homer-editionlingerie commented Sep 15, 2016

Would request that too. Didn't realise it's not supported. When EC2 instance profile was encouraged, such a feature is nearly a must.

@kachenjr
Copy link

@kachenjr kachenjr commented Sep 22, 2016

+1

1 similar comment
@peter-playbasis
Copy link

@peter-playbasis peter-playbasis commented Sep 28, 2016

+1

@james-huston
Copy link

@james-huston james-huston commented Sep 29, 2016

+1 and a case of your favorite beer.

Until then @gene1wood has got some hackery that helped me get something going for my use case. Writing a local profile based on the values coming back make your temp credentials available to other commands.

#1390 (comment)

@gene1wood
Copy link

@gene1wood gene1wood commented Sep 30, 2016

@james-huston thanks for the shoutout. You can also find that bash hack as well as a python version here : https://github.com/gene1wood/aws_assume_roles

@philmcmahon
Copy link

@philmcmahon philmcmahon commented Nov 25, 2016

+1

@dmitry-livchak
Copy link

@dmitry-livchak dmitry-livchak commented Dec 1, 2016

+1 An absolute must have! It is even advertised as a feature in IAM CLI documentation:

You can run an AWS CLI command using a role only when you are signed in as an IAM user, as an the section called “Identity Providers and Federation” (the section called “About SAML 2.0 Federation” or the section called “About Web Identity Federation”) already using a role, or when run from within an Amazon EC2 instance that is attached to a role through its instance profile.

@jpdoria
Copy link

@jpdoria jpdoria commented Aug 16, 2017

+1 Please make this happen.

@RAR
Copy link

@RAR RAR commented Aug 17, 2017

+1

1 similar comment
@fraajad
Copy link

@fraajad fraajad commented Sep 2, 2017

+1

@aaugustin
Copy link

@aaugustin aaugustin commented Sep 6, 2017

I'm using the following workaround:

sts_client = boto3.client(
    service_name='sts',
    region_name=settings.AWS_REGION_NAME,
)
response = sts_client.assume_role(
    RoleArn=settings.AWS_ASSUME_ROLE_ARN,
    RoleSessionName=settings.AWS_ASSUME_ROLE_SESSION_NAME,
)
credentials = response['Credentials']
credentials = {
    'aws_access_key_id': credentials['AccessKeyId'],
    'aws_secret_access_key': credentials['SecretAccessKey'],
    'aws_session_token': credentials['SessionToken'],
}

client = boto3.client(
    service_name='sns',
    region_name=settings.AWS_REGION_NAME,
    **credentials,
)

Unfortunately even more code is required to refresh credentials regularly — they expire after one hour.

@BastianVoigt
Copy link

@BastianVoigt BastianVoigt commented Sep 7, 2017

+1

4 similar comments
@apurv
Copy link

@apurv apurv commented Sep 7, 2017

+1

@MattTunny
Copy link

@MattTunny MattTunny commented Sep 18, 2017

+1

@marktb1
Copy link

@marktb1 marktb1 commented Sep 25, 2017

+1

@robcoward
Copy link

@robcoward robcoward commented Sep 28, 2017

+1

@TheBitFish
Copy link

@TheBitFish TheBitFish commented Sep 28, 2017

+1 two years in and still no fix, what gives?

@qoomon
Copy link

@qoomon qoomon commented Oct 2, 2017

+1

3 similar comments
@antongurov
Copy link

@antongurov antongurov commented Oct 6, 2017

+1

@MatthewDaize
Copy link

@MatthewDaize MatthewDaize commented Oct 6, 2017

+1

@corlettb
Copy link

@corlettb corlettb commented Oct 16, 2017

+1

@grahamjenson
Copy link

@grahamjenson grahamjenson commented Oct 16, 2017

We just released a tool assume-role (https://github.com/coinbase/assume-role) that can export the credentials without using a profile. We use it to manage loads of roles across dozens of accounts. Hope this helps. PRs are also welcome!

@mkostrikin
Copy link

@mkostrikin mkostrikin commented Oct 24, 2017

+1

brian-brazil added a commit to prometheus/prometheus that referenced this issue Oct 25, 2017
* Allow getting credentials via EC2 role

This is subtly different than the existing `role_arn` solution, which
allows Prometheus to assume an IAM role given some set of credentials
already in-scope. With EC2 roles, one specifies the role at instance
launch time (via an instance profile.) The instance then exposes
temporary credentials via its metadata. The AWS Go SDK exposes a
credential provider that polls the [instance metadata endpoint][1]
already, so we can simply use that and it will take care of renewing the
credentials when they expire.

Without this, if this is being used inside EC2, it is difficult to
cleanly allow the use of STS credentials. One has to set up a proxy role
that can assume the role you really want, and launch the EC2 instance
with the proxy role. This isn't very clean, and also doesn't seem to be
[supported very well][2].

[1]:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
[2]: aws/aws-cli#1390

* Automatically try to detect EC2 role credentials

The `Available()` function exposed on ec2metadata returns a simple
true/false if the ec2 metadata is available. This is the best way to
know if we're actually running in EC2 (which is the only valid use-case
for this credential provider.)

This allows this to "just work" if you are using EC2 instance roles.
@MarkRose
Copy link

@MarkRose MarkRose commented Nov 6, 2017

+1

@lorengordon
Copy link
Contributor

@lorengordon lorengordon commented Nov 14, 2017

@aaugustin
Copy link

@aaugustin aaugustin commented Nov 22, 2017

Thank you very much!

@iammr-schuck
Copy link

@iammr-schuck iammr-schuck commented Dec 11, 2017

+1, for whatever that's worth. Trying to get a cross acct code pipeline working with codecommit>codebuild>codepipeline>deploy to Lambda but these run under roles in their separated accounts, and without this feature it just doesn't work cleanly.

@feraudet
Copy link

@feraudet feraudet commented Jan 17, 2018

+1

@mr-brody
Copy link

@mr-brody mr-brody commented Jan 29, 2018

As mentioned in the referenced link[1] there a recent feature where you can create a profile and specify the credential source as ec2 instance metadata which resolves this issue[2].

[1]boto/botocore#1313
[2]https://docs.aws.amazon.com/cli/latest/topic/config-vars.html

The interesting configuration part is below:

In ~/.aws/config

[profile crossaccount]
role_arn=arn:aws:iam:...
credential_source=Ec2InstanceMetadata

@sureshgans
Copy link

@sureshgans sureshgans commented Apr 12, 2018

cross account roles works if an instanceprofile is associated to an EC2 instance and trust is established between account1 and account2 for assuming role.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet