New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for AWS config and credential files #441

Merged
merged 14 commits into from Mar 21, 2016

Conversation

Projects
None yet
6 participants
@alexconst
Contributor

alexconst commented Jan 27, 2016

When either access_key_id or secret_access_key are not set it will attempt to read from environment variables, if those are empty then it will attempt to read from config and credentials.
It allows choosing a profile (by default it's "default") and an "info" directory (by default $HOME/.aws).
Supported information: region, aws_access_key_id, aws_secret_access_key and aws_session_token.

Closes issue #151

alexconst added some commits Jan 27, 2016

Add support for AWS config and credential files
When either access_key_id or secret_access_key are not set it will
attempt to read from environment variables, if those are empty then it
will attempt to read from config and credentials.
It allows choosing a profile (by default it's [default]) and an "info"
directory (by default $HOME/.aws).
Supported information: region, aws_access_key_id, aws_secret_access_key
and aws_session_token.
Move credentials validation and error message
AWS info (credentials and config) verification is done at validate and
the error message is read from the locales yml file.
@ThatGerber

This comment has been minimized.

Show comment
Hide comment
@ThatGerber

ThatGerber Feb 2, 2016

Thanks for the work getting this started. We've been looking for something like this as well.

Two things:

  1. If you're trying to mimic the way the AWS CLI works, config and credential filepaths should fall back to declared values in AWS_SHARED_CREDENTIALS_FILE and AWS_CONFIG_FILE environmental variables before defaulting to ~/.aws/credentials or ~/.aws/config.
  2. You can clean up the config/credential file parser by using an INI/PythonConfig parser such as https://rubygems.org/gems/configparser or https://rubygems.org/gems/iniparse

ThatGerber commented Feb 2, 2016

Thanks for the work getting this started. We've been looking for something like this as well.

Two things:

  1. If you're trying to mimic the way the AWS CLI works, config and credential filepaths should fall back to declared values in AWS_SHARED_CREDENTIALS_FILE and AWS_CONFIG_FILE environmental variables before defaulting to ~/.aws/credentials or ~/.aws/config.
  2. You can clean up the config/credential file parser by using an INI/PythonConfig parser such as https://rubygems.org/gems/configparser or https://rubygems.org/gems/iniparse

alexconst added some commits Feb 4, 2016

Fix tests
Fix AWS environment variable names in tests.
Fix AWS variables coupling: id and secret must both be present.
Updated code to work together with finalize defaults.
Set tests to run with defined order, making debugging easier.
Add test for AWS shared credentials
Test case: with EC2 credential environment variables set
Add test for AWS shared credentials
Test case: without EC2 credential environment variables but with
AWS_CONFIG_FILE and AWS_SHARED_CREDENTIALS_FILE set
Add support for AWS shared credentials location
Add support for AWS shared credentials location environment variables
AWS_CONFIG_FILE and AWS_SHARED_CREDENTIALS_FILE
@alexconst

This comment has been minimized.

Show comment
Hide comment
@alexconst

alexconst Feb 5, 2016

Contributor

@ThatGerber I've implemented 1) and 2).
Let me know if it works for you.

Contributor

alexconst commented Feb 5, 2016

@ThatGerber I've implemented 1) and 2).
Let me know if it works for you.

@alexconst

This comment has been minimized.

Show comment
Hide comment
@alexconst

alexconst Feb 5, 2016

Contributor

Updated description:
This PR reads AWS config and credentials.
Behaviour aims to mimic what is described in AWS documentation:
http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
http://docs.aws.amazon.com/cli/latest/topic/config-vars.html
Which is the following (stopping at the first successful case):

  1. read config and credentials from environment variables
  2. read config and credentials from files at location defined by environment variables
  3. read config and credentials from files at default location

The mandatory fields for a successful "get credentials" are the id and the secret keys.
Region is not required since Config#finalize falls back to sensible defaults.
The behaviour is all-or-nothing (ie: no mixing between vars and files).

It also allows choosing a profile (by default it's [default]) and an "info"
directory (by default $HOME/.aws), which can be specified in the Vagrantfile.
Supported information: region, aws_access_key_id, aws_secret_access_key, and aws_session_token.

Contributor

alexconst commented Feb 5, 2016

Updated description:
This PR reads AWS config and credentials.
Behaviour aims to mimic what is described in AWS documentation:
http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
http://docs.aws.amazon.com/cli/latest/topic/config-vars.html
Which is the following (stopping at the first successful case):

  1. read config and credentials from environment variables
  2. read config and credentials from files at location defined by environment variables
  3. read config and credentials from files at default location

The mandatory fields for a successful "get credentials" are the id and the secret keys.
Region is not required since Config#finalize falls back to sensible defaults.
The behaviour is all-or-nothing (ie: no mixing between vars and files).

It also allows choosing a profile (by default it's [default]) and an "info"
directory (by default $HOME/.aws), which can be specified in the Vagrantfile.
Supported information: region, aws_access_key_id, aws_secret_access_key, and aws_session_token.

@alexconst

This comment has been minimized.

Show comment
Hide comment
@alexconst

alexconst Feb 22, 2016

Contributor

@rtyler Any chance for this and the other PRs to get some love?

Contributor

alexconst commented Feb 22, 2016

@rtyler Any chance for this and the other PRs to get some love?

@alexconst

This comment has been minimized.

Show comment
Hide comment
@alexconst

alexconst Mar 1, 2016

Contributor

@mitchellh is anyone actively maintaining this project?

Contributor

alexconst commented Mar 1, 2016

@mitchellh is anyone actively maintaining this project?

@madsem

This comment has been minimized.

Show comment
Hide comment
@madsem

madsem Mar 6, 2016

@mitchellh don't understand why packer is reading ~/.aws/credentials but vagrant-aws is not. Please look into merging

madsem commented Mar 6, 2016

@mitchellh don't understand why packer is reading ~/.aws/credentials but vagrant-aws is not. Please look into merging

Show outdated Hide outdated Gemfile Outdated
@alexconst

This comment has been minimized.

Show comment
Hide comment
@alexconst

alexconst Mar 8, 2016

Contributor

@rtyler I believe I fixed the gemspec issue.

Contributor

alexconst commented Mar 8, 2016

@rtyler I believe I fixed the gemspec issue.

@alexconst

This comment has been minimized.

Show comment
Hide comment
@alexconst

alexconst Mar 20, 2016

Contributor

@rtyler any updates on this?

Contributor

alexconst commented Mar 20, 2016

@rtyler any updates on this?

rtyler added a commit that referenced this pull request Mar 21, 2016

Merge pull request #441 from alexconst/feat/aws_files
Add support for AWS config and credential files

@rtyler rtyler merged commit db7bf88 into mitchellh:master Mar 21, 2016

@lantins

This comment has been minimized.

Show comment
Hide comment
@lantins

lantins Apr 9, 2016

If possible, I'd love to see a new gem released that includes this change.

lantins commented Apr 9, 2016

If possible, I'd love to see a new gem released that includes this change.

@kenorb

This comment has been minimized.

Show comment
Hide comment
@kenorb

kenorb Apr 15, 2016

I have this error after applying this PR:

$ vagrant up --provider=aws
~/.vagrant.d/gems/gems/vagrant-aws-0.7.0/lib/vagrant-aws/config.rb:537:in `read_aws_files': undefined method `[]' for nil:NilClass (NoMethodError)
    from ~/.vagrant.d/gems/gems/vagrant-aws-0.7.0/lib/vagrant-aws/config.rb:513:in `get_aws_info'
    from ~/.vagrant.d/gems/gems/vagrant-aws-0.7.0/lib/vagrant-aws/config.rb:326:in `finalize!'

My config file is like:

[default]
region = us-east-1
output = text

[testing]
region = us-east-1
output = text

What's wrong?

kenorb commented Apr 15, 2016

I have this error after applying this PR:

$ vagrant up --provider=aws
~/.vagrant.d/gems/gems/vagrant-aws-0.7.0/lib/vagrant-aws/config.rb:537:in `read_aws_files': undefined method `[]' for nil:NilClass (NoMethodError)
    from ~/.vagrant.d/gems/gems/vagrant-aws-0.7.0/lib/vagrant-aws/config.rb:513:in `get_aws_info'
    from ~/.vagrant.d/gems/gems/vagrant-aws-0.7.0/lib/vagrant-aws/config.rb:326:in `finalize!'

My config file is like:

[default]
region = us-east-1
output = text

[testing]
region = us-east-1
output = text

What's wrong?

@ThatGerber

This comment has been minimized.

Show comment
Hide comment
@ThatGerber

ThatGerber Apr 16, 2016

@kenorb You are configuring your profiles incorrectly, that's why it's not working.

http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-multiple-profiles

The method is that config files should have [profile NAME]. Credentials files are set up as [NAME].

ThatGerber commented Apr 16, 2016

@kenorb You are configuring your profiles incorrectly, that's why it's not working.

http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-multiple-profiles

The method is that config files should have [profile NAME]. Credentials files are set up as [NAME].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment