Authentication using AWS AssumeRole and GetSessionToken with AWS STS

[cyberduck]

04d44b0 created the issue

I am using amazon AssumeRole function to assume a role that can access an S3 bucket. This means to connect to S3, it needs more than just SecretKey and AccessKey, it also need SecurityToken or SessionToken which is an extremely large string.

If you set these 3 things in your environment, you can use tools like awscli etc from command line. I would like to use cyberduck instead as it can thread nicely. But unfortunately cyberduck only supports IAM users and not roles.

It does support roles from an EC2 instance, so I think it should be very easy to support from my own OSX laptop? I was thinking of just running a local proxy for 169.254.169.254 to fake the fact I am not running on EC2, but it seemed like overkill.

I notice a few people are suggesting entry of the security token - but isn't that short-lived? Don't see how that's a stable configuration solution. When configuring AWS CLI for this, I'd have an entry for the master account, and then one entry for each assumed role, such as:

[profile master]
region = us-east-1
output = json

[profile security]
role_arn = arn:aws:iam::999999999999:role/MyAccessRole
source_profile = master
region=eu-west-1
output=json

I think you need a way to collect and use this information, mainly the role_arn and reference to the source_profile.

[cyberduck]

@dkocher commented

Referencing Support S3 authentication via IAM Role credentials #8610.

[cyberduck]

eaa4aa5 commented

I also would like this fixed. STS provides safer access privileges.

Does it take time to fix?

[cyberduck]

@dkocher commented

Replying to [comment:3 limitusus]:

I also would like this fixed. STS provides safer access privileges.

AWS Security Token Service (STS)

[cyberduck]

@dkocher commented

Can you post the transcript from the log drawer (⌘-L) for the authentication failure that we get when trying to authenticate with the AccessKeyId and SecretAccessKey only with the token missing.

[cyberduck]

eaa4aa5 commented

Sure.
I set ACCESS_KEY_ID as user name and SECRET_ACCESS_KEY as password, without setting SESSION_TOKEN, and got the below transcript log.
Please check it out.

GET / HTTP/1.1
Date: Thu, 10 Sep 2015 15:34:51 GMT
x-amz-request-payer: requester
Authorization: AWS ASIAJMHJ3CDT6TZFE5VQ:n6tV+L8WNQJPzE8U9bLJupZwEgg=
Host: s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.2.18004 (Mac OS X/10.10.5) (x86_64)
HTTP/1.1 403 Forbidden
x-amz-request-id: 18855270603EBB20
x-amz-id-2: MLFktIVbM4nJRyzpHZhEdMTBODjIdcFGbPfzJXIPbavXlyzFfYDbW75Hx7fNk5wFMC/RU3PXHig=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Thu, 10 Sep 2015 15:34:51 GMT
Server: AmazonS3

[cyberduck]

677cedb commented

I also need this!

[cyberduck]

@dkocher commented

If you are running Cyberduck within EC2, please refer to Connecting with temporary access credentials from EC2.

[cyberduck]

@dkocher commented

Can you clarify the use case and if we should use the AWS STS API to obtain the session token from https://sts.amazonaws.com.

[cyberduck]

@dkocher commented

AssumeRole

Returns a set of temporary security credentials (consisting of an access key ID, a secret access key, and a security token) that you can use to access AWS resources that you might not normally have access to. Typically, you use AssumeRole for cross-account access or federation.

[cyberduck]

@dkocher commented

Do you require multi-factor authentication (MFA) information for AssumeRole?

[cyberduck]

@dkocher commented

Milestone renamed

[cyberduck]

@dkocher commented

Ticket retargeted after milestone closed

[cyberduck]

@dkocher commented

#9992 closed as duplicate.

[cyberduck]

@dkocher commented

#10013 closed as duplicate.

[cyberduck]

@dkocher commented

We possibly just need to add support to allow the input of a SessionToken in the authentication prompt to set the X-Amz-Security-Token header.

[cyberduck]

@dkocher commented

Related ticket #10009.

[cyberduck]

@dkocher commented

Ticket retargeted after milestone closed

[cyberduck]

655a51a commented

I also need this.

What would be AWESOME: instead of collecting the access_key_id and secret_key, you instead would collect the $AWS_DEFAULT_PROFILE or "--profile" as used in aws s3 CLI commands, so if we have configured awscli to use roles via lines in ~/.aws/config, this would simply work without having to double-enter the data in two locations.

Happy if you provide an option to both store inside Cyberduck AND if not stored internally, attempt to lookup the profile inside ~/.aws/* too.

But, AWS best practice for the last few years has been to use role assumption in any multi-account scenario, and they've been pushing multi-account at the enterprise level also for a few years, so I think you need to prioritize this - it seems to have been a request for years now.

[cyberduck]

a082b69 commented

What is the status of this feature request? It has been delayed and/or removed from the schedule multiple times over the past two years. As mentioned by someone else a few months ago, please make this a priority and add support for session tokens soon.

[cyberduck]

@dkocher commented

Is reading credentials from ~/.aws/credentials of any help?

[cyberduck]

a082b69 commented

If Cyberduck supported the use of aws_session_token from the credentials file, then this would definitely be the route to take. However, using only the aws_access_key_id and aws_secret_access_key from that file does not allow authentication to the service. It needs a combination of all three values. You may want to add support for the legacy variable name - aws_security_token - which shares the same value as aws_session_token (at least in our environment). The token information is generated via the STS service when authenticating via SAML-based identity provider (whether that is Okta or ADFS or Auth0 or other provider).

Here's a truncated profile in my credentials file:

[aws-account-name]
aws_access_key_id = ASIA...
aws_secret_access_key = ++LQV7...
aws_session_token = FQoDY...
aws_security_token = FQoDY...
last_updated = 2018-03-26T18:14:35Z

More information:
https://aws.amazon.com/blogs/security/a-new-and-standardized-way-to-manage-credentials-in-the-aws-sdks/

[cyberduck]

655a51a commented

aws_session_token and aws_security_token are, I think, the wrong way to fix this. That's not how AWS recommends you configure cross-account roles in AWS CLI.

The original ticket description remains the correct approach, IMHO. The IAM access code should look up a profile in ~/.aws/config - NOT - specify secret/access keys explicitly. This profile may contain either the secret/access keys needed, or it may contain a role_arn combined with a reference to a source_profile. It is the combination of the source_profile to get the secret/access key, with the role_arn to assume that role in another account, which is needed to access the S3 bucket in the other account.

I'm pretty sure all the code necessary to make this work is open source and visible in the AWS CLI GitHub project, someone just needs to refactor it to work here. Not sure why this is taking so long...

[cyberduck]

a082b69 commented

Both scenarios could/should be supported. Cyberduck should not define best practice or method. If the SDK or boto support a specific credentials configuration (or cross-account access configuration) for authentication, then clients like Cyberduck should also strive to support this authentication method(s). We have no issues with other clients, such as the AWSCLI or Terraform, supporting aws_session_token, so the expectation is that Cyberduck would also support it.

As you said, this should be a rather simple change since this is standard support within the SDK. Perhaps this needs to be split into two separate requests, but they’re closely related.

[cyberduck]

ce2701a commented

I would also suggest that allowing for MFA is/should be closely tied to this feature. S3 security is at the forefront of many highlevel breaches in the news recently and permitting temporary credentials via sts:AssumeRole coupled with sts:GetSessionToken and allowing a client to use MFA significantly improves safety of data.

We have 100's of Cyberduck users who can no longer use the product due to this missing feature. I am very much looking forward to seeing this implemented so my user community can return to using CyberDuck

[cyberduck]

49f3158 commented

Are there any updates on when this will be available?
I've increased the priority to high as mentioned above, hundreds of Cyberduck users are no longer able to use the product due to this missing feature. This means that potential customers will also look elsewhere.
Many companies are starting to use the AssumeRole function, and so the need for this is increasing by the day.
Please add the feature in soon!

My suggestion is to pull all the credentials from the .aws/credentials file, upon every connect. The contents of the file will change, and so having the ability to pull the information every time will be useful and enable users not to have to re-enter the details (for if their credentials have a timeout/expiry of 60 mins).

[cyberduck]

@dkocher commented

Thanks everyone for the input provided!

As we are not accustomed ourselves to using session tokens, we would love to get some feedback if requirements are met when we implement this with the following constraints:

• Read AWS credential profiles file from the default location (~/.aws/credentials). We already do this – note the AWS access key being set when you add a new bookmark and set the protocol to S3.
• If the AWS Access Key set in the bookmark matches, read the session token.
• We use the default profile "default" unless a custom profile name is set in the connection profile using the key Context. Refer to Profiles.
• When Contextis set to an URL, the session token is retrieved from the EC2 instance metadata service (already supported).

[cyberduck]

@dkocher commented

References

• AWS Security Token Service
• AWS CLI Command Reference

[cyberduck]

a082b69 commented

@dkocher, I will be more than willing to test our use case when the above constraints are accounted for. For us, the big one is the "read the session token" portion. I believe what you have stated in the above criteria will address our scenario, which is passing the session token for authentication when a named profile is referenced. Thanks for your work and research on this ticket.

[cyberduck]

ce2701a commented

A typical use case we have is switching roles between accounts that require MFA for the assume role to succeed. A sample of the type of config file most users are using is adding the mfa_serial to the config default profile and then referencing this in other profiles. This setup is using a single sign-on account '00000000000' for user management for passwords/access keys and MFA. Then the users will assume role into a different account to access S3.

When accessing S3 the UI should allow the user to input the MFA token to retrieve an sts:SessionToken which will carry the MFA characteristics along to be used to get sts:AssumeRole credentials.

User Credentials -> Session Credentials with MFA -> Assume Role into accounts with S3 data.

~user/.aws/config

[default]
region=us-east-1
output=json
mfa_serial=arn:aws:iam::000000000000:mfa/user@domain.com 
[profile assumerole]
role_arn=arn:aws:iam::11111111111111:role/role1account1
source_profile=default
[profile assumerole2]
role_arn=arn:aws:iam::22222222222222:role/role2account2
source_profile=default

[cyberduck]

a082b69 commented

@mcnicr MFA support with session tokens might be better served as a different ticket/feature request that is dependent on this one for session token support. Just my opinion.

[cyberduck]

a58e687 commented

I already have a script that gets me this far in ~/.aws/credentials:

[publish_profile]
output = json
region = us-west-1
aws_access_key_id = AAAAAAAAAAAAAAAAAAAA
aws_secret_access_key = KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
aws_session_token = SSSSSSSSSS_//_////SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS=

I want to configure Cyberduck with just profile = "publish_profile" and have it use those three values. It would also be nice to catch any expiration error so you could remind the user that their session has expired and they need to run through their external SSO tool again to refresh the aws_session_token.

[cyberduck]

@dkocher commented

Related issue #10222 asking for generic support of MFA.

[cyberduck]

@dkocher commented

Replying to [8880 tigris]:

It does support roles from an EC2 instance, so I think it should be very easy to support from my own OSX laptop? I was thinking of just running a local proxy for 169.254.169.254 to fake the fact I am not running on EC2, but it seemed like overkill.

AWS Vault looks like an interesting project which supports exposing credentials running a local EC2 Instance Metadata server which should work together with the profile for Connecting with temporary access credentials (Token) from EC2.

[cyberduck]

@dkocher commented

Documentation forthcoming in Connecting using AssumeRole from AWS Security Token Service (STS).

[cyberduck]

@dkocher commented

We used the following steps to test our implementation.

1. Create an IAM user testuser
2. Create an IAM policy userpolicy with the following policy document:

      {
        "Version": "2012­10­17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::123456789012:role/testrole",
            "Condition": {
              "Bool": {"aws:MultiFactorAuthPresent": true}
            }
} ]
}

3. Attach the “userpolicy” policy to the “testuser” user.
4. Create an IAM role testrole, specifying 123456789012 as the account and electing to
   require MFA.
5. Create an IAM policy rolepolicy with the following policy document:

      {
        "Version": "2012­10­17",
        "Statement": [{
          "Effect": "Allow",
          "Action": "s3:*",
          "Resource": "*"
}] }

6. Attach the rolepolicy policy to the testrole role.

7. Generate an access key and secret for testuser

8. Configure an MFA device for testuser

9. Create a file ~/.aws/credentials with the following contents (substituting where indicated):

   [testuser]
   aws_access_key_id=<access key for testuser>
   aws_secret_access_key=<secret key for testuser>
   [testrole]
   role_arn=arn:aws:iam::123456789012:role/testrole
   source_profile=testuser
   mfa_serial=arn:aws:iam::123456789012:mfa/testuser

10. Install the S3 (Credentials from AWS Security Token Service) profile and configure a bookmark using the testrole profile by entering testrole in ‘’Profile Name’’.

[cyberduck]

@dkocher commented

In 02b8e76.

[cyberduck]

@dkocher commented

In d9516d4.

[cyberduck]

@dkocher commented

Skip saving temporary credentials in keychain in 4d47ed8.

[cyberduck]

@dkocher commented

Please update to the latest snapshot build available to test this integration.

[cyberduck]

a082b69 commented

I tested this for our method of accessing AWS via a SAML provider, where the security tokens are written directly into our credentials file via a tool we use to generate those tokens via STS (post-SAML auth). I have confirmed this method is now working to access S3 buckets. Thanks again!

[cyberduck]

a082b69 commented

That said, it seems like the latest snapshot builds have broken our ability to use the traditional S3 profile with access/secret key auth. When prompted for secret key, clicking Login just beeps at us (multiple computers, multiple users, existing and new keys, keys work in other programs and CLIs, etc.).

[cyberduck]

@dkocher commented

Replying to [comment:52 jibi-waba]:

That said, it seems like the latest snapshot builds have broken our ability to use the traditional S3 profile with access/secret key auth. When prompted for secret key, clicking Login just beeps at us (multiple computers, multiple users, existing and new keys, keys work in other programs and CLIs, etc.).
Fix in a721c4c.

[cyberduck]

a58e687 commented

Replying to [comment:40 dt1001]:

I already have a script that gets me this far in ~/.aws/credentials ...
I want to configure Cyberduck with just profile = "publish_profile" and have it use those three values ...

I just tested this with nightly build 28570 and it worked perfectly. Thank you so much! Really makes this enterprise friendly. Going to get my company to buy several licenses now.

[cyberduck]

bb7e371 commented

Hello,

Is this supposed to work on windows?
I've got the cli configured with ~/.aws/credentials and listing bucket works fine. However with the cyberduck sts profile bookmark set to the some profile, cyberduck asks for a password.

Cheers.

[cyberduck]

e51fc8f commented

This credentials file configuration (previously mentioned by dt001) works perfectly with commercial S3 regions (server: s3.amazonaws.com, region: us-west-1) but not with AWS GovCloud (server: s3-us-gov-west-1.amazonaws.com, region: us-gov-west-1). I'm using s3-us-gov-west-1.amazonaws.com as the "Server" and cyberduck gets into a loop where it says "Authenticating as publish_profile" followed by "Login failed". I am using version 6.9.3. Any ideas?

[publish_profile]
output = json
region = us-gov-west-1
aws_access_key_id = AAAAAAAAAAAAAAAAAAAA
aws_secret_access_key = KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
aws_session_token = SSSSSSSSSS_//_////SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS=

[cyberduck]

bb7e371 commented

Replying to [comment:56 cduser]:

This credentials file configuration (previously mentioned by dt001) works perfectly with commercial S3 regions (server: s3.amazonaws.com, region: us-west-1) but not with AWS GovCloud (server: s3-us-gov-west-1.amazonaws.com, region: us-gov-west-1). I'm using s3-us-gov-west-1.amazonaws.com as the "Server" and cyberduck gets into a loop where it says "Authenticating as publish_profile" followed by "Login failed". I am using version 6.9.3. Any ideas?

Did you try us-gov-west-1 as a region in your credentials file? I guess the issue is that it tries to connect to the wrong STS endpoing which is built from that string.

[cyberduck]

e51fc8f commented

Replying to [comment:57 fguerraz]:

Replying to [comment:56 cduser]:

This credentials file configuration (previously mentioned by dt001) works perfectly with commercial S3 regions (server: s3.amazonaws.com, region: us-west-1) but not with AWS GovCloud (server: s3-us-gov-west-1.amazonaws.com, region: us-gov-west-1). I'm using s3-us-gov-west-1.amazonaws.com as the "Server" and cyberduck gets into a loop where it says "Authenticating as publish_profile" followed by "Login failed". I am using version 6.9.3. Any ideas?

Did you try us-gov-west-1 as a region in your credentials file? I guess the issue is that it tries to connect to the wrong STS endpoing which is built from that string.

Hi fguerraz,

Sorry. Yes I have in my credentials file us-gov-west-1 instead of us-west-1. I tried both, us-gov-west-1 fails (using the s3-us-gov-west-1.amazonaws.com server) and us-west-1 works (using the s3.amazonaws.com server). I'm going to edit my original comment to replace us-west-1 with us-gov-west-1.

Thanks!

[cyberduck]

@dkocher commented

Replying to [comment:56 cduser]:

This credentials file configuration (previously mentioned by dt001) works perfectly with commercial S3 regions (server: s3.amazonaws.com, region: us-west-1) but not with AWS GovCloud (server: s3-us-gov-west-1.amazonaws.com, region: us-gov-west-1). I'm using s3-us-gov-west-1.amazonaws.com as the "Server" and cyberduck gets into a loop where it says "Authenticating as publish_profile" followed by "Login failed". I am using version 6.9.3. Any ideas?

[publish_profile]
output = json
region = us-gov-west-1
aws_access_key_id = AAAAAAAAAAAAAAAAAAAA
aws_secret_access_key = KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
aws_session_token = SSSSSSSSSS_//_////SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS=

Can you confirm you use the AWS GovCloud connection profile from (https://cyberduck.io/s3/). Please open a new ticket if the issue persists.

[cyberduck]

@dkocher commented

We fixed the connection profiles to include the region in 050907e.

[cyberduck]

e51fc8f commented

Replying to [comment:59 dkocher]:

Replying to [comment:56 cduser]:

This credentials file configuration (previously mentioned by dt001) works perfectly with commercial S3 regions (server: s3.amazonaws.com, region: us-west-1) but not with AWS GovCloud (server: s3-us-gov-west-1.amazonaws.com, region: us-gov-west-1). I'm using s3-us-gov-west-1.amazonaws.com as the "Server" and cyberduck gets into a loop where it says "Authenticating as publish_profile" followed by "Login failed". I am using version 6.9.3. Any ideas?

[publish_profile]
output = json
region = us-gov-west-1
aws_access_key_id = AAAAAAAAAAAAAAAAAAAA
aws_secret_access_key = KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
aws_session_token = SSSSSSSSSS_//_////SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS=

Can you confirm you use the AWS GovCloud connection profile from (https://cyberduck.io/s3/). Please open a new ticket if the issue persists.

Hi dkocher,

I tried using the AWS GovCloud connection profile (https://svn.cyberduck.io/trunk/profiles/S3%20Gov%20Cloud.cyberduckprofile). The problem is that this profile doesn't seem to have the option to use S3(Credentials from AWS Security Token Service). It seems like to use a temporary token I need to use this other profile (https://svn.cyberduck.io/trunk/profiles/S3%20(Credentials%20from%20AWS%20Security%20Token%20Service).cyberduckprofile). I tried adding the following config to the S3(Credentials from AWS Security Token Service) profile (to change the S3 URL) but didn't work (unless I'm missing something).

        <key>Default Port</key>
        <string>443</string>
        <key>Default Hostname</key>
        <string>s3-us-gov-west-1.amazonaws.com</string>

Is there a way to support both *AWS GovCloud * and S3 (Credentials from AWS Security Token Service).

Thanks!

[cyberduck]

e51fc8f commented

I opened a new ticket #10594

[dkocher]

We are about to add support for AssumeRoleWithWebIdentityResponse and third party OpenID Connect providers in #14972.