Goals

• Enforce a good password policy
• Enforce two factors authentication (MFA), for all users, and even for cli/api usage
• Encourage using a readonly role for day to day unprivileged operations
• Leverage Terraform pgp_key method to safely store and transmit users' credentials
• Optional: use aws-vault to keep AWS keys safe on user's laptop

Structure

We don't grant rights to the users by default (except the minimum rights needed to manage their own password, mfa, account). Users must assume roles to do anything else.

The "assume role" privileges are protected by policies on both sides (the assumed role, the assuming group/user):

• The assumed role's "assume policy" mandate a mfa authenticated user (filtering allowed group here would be more natural, but AWS don't allow using group in principals specifications, so we can't)
• The various user's groups are given the privileges to assume specific roles (depending on the group)

Even privileged/admin group is given the right to assume the readonly role. Admins don't have to run everything with super privileges: they can use least privileges for normal, day to day, read operations.

AWS limits to keep in mind:

• A given user can be member of 10 groups max.
• A group (or user) can have up to 10 policies attached.

Decrypt users secrets

For each created user account, Terraform will also generate an initial password and AWS key pair, encrypted with the user's public PGP key. Secrets don't ends up in the tfstate file, can safely be sent to the user, and can't be read by the person that create the account.

The users' PGP public keys can be provided either as "keybase:userid", or inline (base64 encoded).

To decrypt their secrets the user should:

# or "echo ... | keybase pgp decrypt"
echo "wcBMA..." | base64 -d | gpg --decrypt -

AWS configuration

In AWS console (web interface) :

Navigation bar > user@account > Switch Role > ...

For all tools using AWS SDK (awscli, terraform, ansible, etc):

# ~/.aws/config
[default]
region=eu-west-1

# no need for this keypair if we use aws-vault (recommended)!
aws_access_key_id= XXX
aws_secret_access_key= YYY

[profile readonly]
role_arn=arn:aws:iam::123456789012:role/readonly-role
source_profile=default
mfa_serial=arn:aws:iam::123456789012:mfa/myusername

[profile admin]
role_arn=arn:aws:iam::123456789012:role/admin-role
source_profile=default
mfa_serial=arn:aws:iam::123456789012:mfa/myusername

Then we can use the profiles, awscli will automatically ask for the MFA pin at first access (and will generate a ephemeral key pair and session token in ~/.aws/cache) :

aws s3 --profile readonly ls s3://

export AWS_PROFILE=readonly
aws s3 ls s3://

env AWS_PROFILE=readonly terraform plan
env AWS_PROFILE=admin    terraform apply

If we don't use aws-vault (which will take care of generating sesions token and ephemeral keys for us), we have to write AWS keys in ~/.aws/config to get the the MFA prompt when calling an alternate profile:

# without aws keys in ~/.aws/config ,
# this won't work and display a dumb 'aws_access_key_id':
$ env AWS_ACCESS_KEY_ID='XXX' AWS_SECRET_ACCESS_KEY='YYY' \
    aws --profile readonly s3 ls s3://
'aws_access_key_id'

# with the creds in ~/.aws/config, this work fine, and will generate
# ephemeral key pair + token in ~/.aws/cache/, with a 1h lifetime:
$ aws --profile readonly s3 ls s3://
Enter MFA code:

# ensure readonly restrictions are enforced:
$ aws --profile readonly s3 mb s3://bentest-42-42
make_bucket failed: s3://bentest-42-42 An error occurred (AccessDenied) ...

# use a privileged role
$ aws --profile admin s3 mb s3://bentest-42-42
Enter MFA code:

To avoid writing keys in ~/.aws/config or using aws-vault, we can manually assume a role, but that's tedious. We have to generate a session token (using a MFA pin code), then use this generated a token and a temporary AWS key pair, and use that to call sts assume role, to generate yet another key pair + token.

Then we can use this final ephemeral key pair and token to do actual stuff:

# get the MFA device's serial number
$ env AWS_ACCESS_KEY_ID='XXX' AWS_SECRET_ACCESS_KEY='XXX' \
    aws iam list-virtual-mfa-devices

# generate a session token ; token-code is generated by the MFA device
$ env AWS_ACCESS_KEY_ID='XXX' AWS_SECRET_ACCESS_KEY='XXX' aws sts get-session-token
    --serial-number arn:aws:iam::123456789012:mfa/myusername --token-code 979612

# using the ephemeral keys and token generated by the previous command,
# get new (!) ephemeral keys and tokens for the assumed role:
$ export AWS_ACCESS_KEY_ID='YYY' AWS_SECRET_ACCESS_KEY='YYY' AWS_SECURITY_TOKEN='YYY' \
    aws sts assume-role --role-arn arn:aws:iam::123456789012:role/readonly-role \
    --role-session-name dev

# then we can use the generated keys and token for effective work
$ export AWS_ACCESS_KEY_ID='ZZZ' AWS_SECRET_ACCESS_KEY='ZZZ' AWS_SECURITY_TOKEN='ZZZ'
$ aws s3 ls s3://

aws-vault saves us all this tedious work, and more importantly does not require us to write our keys in ~/.aws/config or in shell's environment (which may end up in shell's history, or collected by a keylogger).

Usage with aws-vault

Using aws-vault is not requiered, but that's a good way to improve user's AWS keys safety.

aws-vault will store the access/secret key encrypted in the MacOs Keychain, Linux' Kwallet, gnome-keyring, or an encrypted file.

It will use that key to generate (and rotate) ephemeral credentials and inject those ephemeral credentials as environment variables to shells and applications.

This way,

• Our AWS key pair remains encrypted at rest
• MFA is enforced, even for cli tools and API calls
• Applications only see ephemeral keys, scoped to the assumed role (leaks are less devastating)
• aws-vault pass those ephemeral keys to apps using environment (nothing is written to disk)
• Switching beween roles becomes easy and natural

Being a (staticaly linked) Go application, installing aws-vault is easy:

sudo curl -Lo /usr/local/bin/aws-vault \
  https://github.com/99designs/aws-vault/releases/download/v4.2.1/aws-vault-linux-amd64
sudo chmod +x /usr/local/bin/aws-vault

vim ~/.aws/config # remove AWS keys (if any)

We only need to register the base profile (here, "default"), not the assumed role's profile. aws-vault will discover, and leverage the other roles in ~/.aws/config without any pre-declaration.

$ aws-vault add default
Enter Access Key ID: AKIAI...
Enter Secret Access Key: ...
Added credentials to profile "default" in vault

# on linux, if you get errors like
# "aws-vault: error: The name org.kde.kwalletd was not provided by any .service files"
# ... then you must specify the backend name:

$ export AWS_VAULT_BACKEND=secret-service

Now we can consume the AWS API using aws-vault generated creds:

$ aws-vault exec default  -- aws iam get-login-profile
$ aws-vault exec readonly -- aws s3 ls s3://
Enter token for arn:aws:iam::123456789012:mfa/myusername: XXXX

$ aws-vault exec admin    -- aws s3 mb s3://test-b-4242

# if we get tired of prefixing "aws-vault exec" before every command,
# we can let aws-vault spawn an authenticated shell for us,

$ aws-vault exec default 

AWS won't ask the MFA pin for every command we type, there's a 4h (by default) session ttl:

# see the ongoing sessions
$ aws-vault --debug list

# expires the current session
$ aws-vault --debug remove --sessions-only default

# launch a new session with a different ttl
$ aws-vault --debug exec --session-ttl=15m --assume-role-ttl=1h --

Further reading

• IAM Best Practices
• Assuming a Role
• STS temporary security credentials
• Tutorial: Enable Your Users to Configure Their Own Credentials and MFA Settings
• Secure (and usable) multi-AWS account IAM setup
• Potential Gaps in Suggested Amazon Web Services’ Security Policies for MFA