Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for an ~/.aws/config.d/ directory #9036

Open
QuinnyPig opened this issue Mar 24, 2022 · 7 comments
Open

Add support for an ~/.aws/config.d/ directory #9036

QuinnyPig opened this issue Mar 24, 2022 · 7 comments
Labels
cross-sdk feature-request A feature should be added or improved. p1 This is a high priority issue

Comments

@QuinnyPig
Copy link

Is your feature request related to a problem? Please describe.

My ~/.aws/config file is massive. I have a proliferation of personal accounts, work accounts, and each client grants me a profile within dozens of accounts as well. I'd like to bring some order to that chaos, as well as not have a single file that lists every AWS account to which I currently have configured access.

Describe the solution you'd like
Being able to break out profiles on a per-client, or along a personal / work divide via different configuration files enables a number of useful workflows. I'd like a subdirectory within ~/.aws/ named config.d; files within that directory can then be included when ~/.aws/config is read by various ecosystem tools. They can be independently managed, and not result in the obnoxious and manual / error prone approach of either editing the file by hand to reflect new accounts / the removal of old ones, or the somehow-worse problem of building a variety of "aws config file generator" scripts bespoke to every customer who writes one of the blasted things.

Describe alternatives you've considered
I have tried and discarded:

  • Generating the config file from scripts
  • Ignoring the problem in the forlorn hope it would annoy me less with time
  • manually editing the file
  • hiring an assistant to log me into AWS accounts manually so I don't have to worry about it

Additional context
The conf.d/ structure is well understood within Linux, and is a well adopted pattern ecosystem-wide.

@QuinnyPig QuinnyPig added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Mar 24, 2022
@kellertk
Copy link
Contributor

Hi there! Thanks for the feature request - I can absolutely see the benefits here and I'll make sure that we discuss it with the larger team.

There is one other option, though it's about as much work as hiring an assistant to log into accounts for you, and that's sourcing credentials with an external process. The big scary security warnings on that page are there to remind you that it's very easy to leak credentials into logs and the like if you're not careful, but it's possible for someone to write a conf.d parser as a script and emit those credentials to the SDK via credentials_process. Process-sourced credentials are actually used quite frequently internally at AWS, because we develop with ephemeral accounts, short credential lifetimes, and other internal-specific restrictions that are embodied in a process that vends credentials for us. But absolutely not everyone is going to want to write a custom client just to manage their SDK credentials, so this feature request still makes sense.

One thing that we're trying to do (which we haven't always done in the past) is not have the SDKs behave differently when it comes to reading config files, handling credentials, and that sort of thing. Which means we wouldn't want to implement this in just CLI, but all of the other SDKs as well. Because of that I'm going to move this issue into the repo that we're using to track cross-SDK feature requests. Expect some more information from us once we've had a chance to review this across the SDKs and prioritized this.

@kellertk kellertk transferred this issue from aws/aws-cli Mar 25, 2022
@kdaily kdaily added cross-sdk and removed needs-triage This issue or PR still needs to be triaged. labels Mar 25, 2022
@ranman
Copy link

ranman commented Mar 27, 2022

Another option is to use something like git's [includeIf "dir:~/work/"] functionality: https://git-scm.com/docs/git-config#_conditional_includes

@kellertk
Copy link
Contributor

We discussed this feature internally across the other SDK teams and we're interested so far. We'll be having one of our engineers do a test implementation to look at things like ergonomics, performance, and effort required in all of the SDKs. Also, there is some complexity we'll have to work out, because we have both credentials and config files to consider, but we're looking at it :)

As always, your thumbs up reactions let us know how urgent this is for the community. Thanks!

@QuinnyPig
Copy link
Author

Recently dealt with a customer with a few thousand AWS accounts. My kingdom for this feature!

@ronaldbradford
Copy link

+1 for me. I could definitely benefit from managing individual files not sed'ingy way thru a single configt file for existence before modified or removal of an unknown number of lines

@nitrocode
Copy link

nitrocode commented Dec 29, 2022

The way I've worked around this limitation is by creating different configuration files per client and setting the AWS_CONFIG_FILE and using direnv

Example

clients/client1/.envrc
clients/client1/.aws-config
clients/client2/.envrc
clients/client2/.aws-config
# ...

Then each .envrc contains

export AWS_CONFIG_FILE=$(pwd)/.aws-config

Example (note that direnv allow is only needed the first time)

$ cd ~/git/work/client1
$ direnv allow
$ echo $AWS_CONFIG_FILE
AWS_CONFIG_FILE=/home/user/git/work/client1/.aws-config

@amason13
Copy link

amason13 commented Jan 12, 2024

my solution was to add .awsprofile file in each project directory.

inside each file you save the name of the profile you'd like to activate.

then add the following script to your .zshrc file

### script to alias CD with activate aws profile
hasAwsProfile () {
  if [ -f .awsprofile ]; then
    export AWS_PROFILE="`cat .awsprofile`"
  fi
}
awsProfileCd () {
cd "$@" && hasAwsProfile
}

alias cd="awsProfileCd"

@bhoradc bhoradc added the p1 This is a high priority issue label Aug 29, 2024
@tim-finnigan tim-finnigan transferred this issue from aws/aws-sdk Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cross-sdk feature-request A feature should be added or improved. p1 This is a high priority issue
Projects
None yet
Development

No branches or pull requests

9 participants