Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for credential_process in ~/.aws/config #4838

Open
kennu opened this issue Mar 18, 2018 · 17 comments
Open

Support for credential_process in ~/.aws/config #4838

kennu opened this issue Mar 18, 2018 · 17 comments

Comments

@kennu
Copy link
Contributor

@kennu kennu commented Mar 18, 2018

This is a Feature Proposal

Description

AWS CLI supports the credential_process option in ~/.aws/config profiles, which allows you to store credentials somewhere else than the plain ~/.aws/credentials file. For instance, you can create a script that fetches them from 1Password using the 1Password CLI.

Serverless Framework doesn't support this so if you use this feature, you will get a Profile does not exist error when deploying. I know supporting this may not be quite trivial, but it might be very convenient.

Additional Data

  • Serverless Framework Version you're using: 1.26.1
  • Operating System: macOS
  • Stack Trace:
  • Provider Error messages:
@meatherly
Copy link

@meatherly meatherly commented Jan 11, 2019

Just needs to be merged aws/aws-sdk-js#1923. Then we can implement it on serverless

@eahefnawy eahefnawy added the cat/dx label Jan 31, 2019
@StevenACoffman
Copy link
Contributor

@StevenACoffman StevenACoffman commented Jun 13, 2019

This has been merged in aws/aws-sdk-js#2559 and is available with aws-sdk-js v2.474.0 through v2.429.0

Would appreciate if this got picked up.

gshively11 added a commit to gshively11/serverless that referenced this issue Jul 22, 2019
- This only affects the aws provider
- Resolves [Issue serverless#4838](serverless#4838)
- Retrieving credentials from the AWS provider is now an asynchronous process
gshively11 added a commit to gshively11/serverless that referenced this issue Jul 22, 2019
- This only affects the aws provider
- Resolves [Issue serverless#4838](serverless#4838)
- Retrieving credentials from the AWS provider is now an asynchronous process
@gshively11 gshively11 mentioned this issue Jul 22, 2019
5 of 8 tasks complete
@mungojam
Copy link

@mungojam mungojam commented Apr 29, 2020

It would be lovely to see this progress again (sorry I can't help at the moment). Currently I have to keep switching my config files between using credential_process for all other SDKs and tools and using temporary credentials when calling serverless.

@thomasmichaelwallace
Copy link
Contributor

@thomasmichaelwallace thomasmichaelwallace commented Aug 28, 2020

Just another +1.

credential_process seems to be a good work around for #7567 (see https://github.com/benkehoe/aws-sso-credential-process).

At the moment I'm trying to understand why the serverless framework uses it's own approach to credentials; which seems to be the reason that credential_process is supported by the aws-sdk out-of-the-box, but doesn't work here.

@thomasmichaelwallace
Copy link
Contributor

@thomasmichaelwallace thomasmichaelwallace commented Aug 28, 2020

@flyinprogrammer - I'm actually trying to get together a PR to see about using it.

However, because sls hasn't used the default chain, there's a few additional cases I'm working through.

For comment, it's looking something like this:

    // Tweak the default provider chain to match serverless extensions:
    // https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/CredentialProviderChain.html#defaultProviders-property
    const credentialProvider = new AWS.CredentialProviderChain([
      function () { return new AWS.EnvironmentCredentials('AWS'); },
      function () { return new AWS.EnvironmentCredentials('AMAZON'); },
      // add serverless specific AWS_${STAGE} environment
      function () { return new AWS.EnvironmentCredentials(environment); },
      // prefer serverless configured profile and readline token fn for ini:
      function () { return new AWS.SharedIniFileCredentials({ profile, tokenCodeFn, filename }); },
      // function () { return new AWS.ECSCredentials(); }
      // prefer serverless configured profile for process:
      function () { return new AWS.ProcessCredentials({ profile }); },
      // function () { return new AWS.TokenFileWebIdentityCredentials(); },
      // function () { return new AWS.EC2MetadataCredentials(); }
      // support serverless config credentials
      function () { return new AWS.Credentials(providerCredentials) },
    ]);
@dougmoscrop
Copy link
Contributor

@dougmoscrop dougmoscrop commented Sep 15, 2020

Looking for feedback on a first pass at using the provider chain here: dougmoscrop@49cd19a

@drissamri
Copy link

@drissamri drissamri commented Oct 9, 2020

Is there any workaround currently for using AWS SSO until this is done?

@thomasmichaelwallace
Copy link
Contributor

@thomasmichaelwallace thomasmichaelwallace commented Oct 9, 2020

If you've only got one profile, you can use this: https://github.com/benkehoe/aws-export-credentials

@flyinprogrammer
Copy link

@flyinprogrammer flyinprogrammer commented Oct 9, 2020

@drissamri i've been using aws-vault like so:

aws-vault exec profile_name -- sls ...

which just causes sls to rely on environment variables existing.

@emilhdiaz
Copy link

@emilhdiaz emilhdiaz commented Oct 19, 2020

@thomasmichaelwallace I installed aws-sso-credential-process through pipx: pipx install aws-sso-credential-process, configured that against my desired aws profile in ~/.aws/config, and finally wrote this bash script (aws-sso) to wrap any shell commands that do not support AWS CLI v2 SSO profiles yet:

#!/bin/bash
set -euE -o pipefail

CREDS=$(aws-sso-credential-process --profile "${AWS_PROFILE}")

(
  unset AWS_PROFILE &&
  AWS_ACCESS_KEY_ID=$(echo "${CREDS}" | jq -r '.AccessKeyId') \
  AWS_SECRET_ACCESS_KEY=$(echo "${CREDS}" | jq -r '.SecretAccessKey') \
  AWS_SESSION_TOKEN=$(echo "${CREDS}" | jq -r '.SessionToken') \
  AWS_DEFAULT_REGION=$(aws configure list | grep region | awk '{print $2}') \
  "$@"
)
SUBSHELL_STATUS=$?

if [ $SUBSHELL_STATUS -eq 0 ]; then
exit 0
else
echo "aws-sso command failed."
exit 1
fi

To use:

aws-sso sls deploy
@thomasmichaelwallace
Copy link
Contributor

@thomasmichaelwallace thomasmichaelwallace commented Oct 19, 2020

☝️ it's an interesting solution - the main struggle we have is that we rely on provider.profile a lot, where as these effectively create a single profile configuration controlled outside of serverless.

I'm actually planning on picking up @dougmoscrop 's implementation and working that into a PR today.

So hopefully native serverless support for this is in the future...

@thomasmichaelwallace
Copy link
Contributor

@thomasmichaelwallace thomasmichaelwallace commented Oct 19, 2020

☝️ watch this space.

@thomasmichaelwallace
Copy link
Contributor

@thomasmichaelwallace thomasmichaelwallace commented Oct 28, 2020

So - there's quite a lot of refactoring planned for/around the files that need to be touched by this PR.

Unfortunately I don't have the time to look at these (#8441, #8444, #8445, #8446 - for anyone that does.) Once they're clear I'll come back and get the PR back up off the ground again.

I need this feature, now, however, and so I'll be maintaining a patch with the core (support for credential_process) code.

For anyone else who wants to use this patch in the interim - I'll be maintaining it (with some instructions) here: https://gist.github.com/thomasmichaelwallace/5ef97b1fbadf8df2bca21bebafd2dd7e

@medikoo
Copy link
Member

@medikoo medikoo commented Nov 10, 2020

For reference: Work on that has been started here: #8415 which was closed temporarily, until dependent issues, as listed below are addressed

  • #8441 - Test: Refactor lib/plugins/aws/invokeLocal/index.test.js
  • #8444 - Refactor: Seclude lib/utils/log.js (general logging) util
  • #8445 - Refactor: Seclude lib/utils/aws/request.js (AWS request logic) util
  • #8446 - Test: Refactor lib/plugins/aws/provider/awsProvider.test.js

Any help is addressing above will be appreciated

@thomasmichaelwallace
Copy link
Contributor

@thomasmichaelwallace thomasmichaelwallace commented Feb 5, 2021

For anyone following - I did a resync of my patch to the latest (v2.30.3) version,

While I've got everything back in sync - @medikoo - is the plan still to block on the issues mentioned in November?

@medikoo
Copy link
Member

@medikoo medikoo commented Feb 5, 2021

While I've got everything back in sync - @medikoo - is the plan still to block on the issues mentioned in November?
@medikoo

@thomasmichaelwallace yes I believe nothing changed, to welcome this enhancement safely, first we need to improve our tests (it's quite sensitive change)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.