Install awscli

Install the Universal Command Line Environment for AWS

These are instructions to walk through the setup of an Amazon IAM account on a local workstation so commands can be issued to the AWS API from a terminal.

Previous mac-ops Users

This is the easiest method because your ~/.aws directory was restored during ./bootstrap.sh execution. In this case the process is much simpler:

Open a Terminal
Run aws configure list to confirm nothing is configured.
Edit ~/.bashrc and configure AWS_PROFILE.
Source-in the file: source ~/.bashrc
Run aws configure list again and you will see your profile configured and listed.

In all other cases:

You have NOT been using or are new to the mac-ops method, or
You just want to get up and running with the AWS cli quickly, then

keep reading. This process will get you set up securely for not only macOS but Linux as well.

Prerequisites

Python is the only package required to install the awscli.

Install Python

Any of these methods assume that you should - first - have a fully updated OS.

The install steps also assume that you should be able to upgrade python without considerations. Maybe you have written a program that depends on python3.3? Upgrade carefully.

Past that, whether python2 or python3 doesn't really matter; the only difference is your requirements. The awscli will install from either version. If it helps, I have both versions installed side-by-side with Vim compiled for python3 support; the python2 install only exists for Ansible. Then, all new development is done in python3 and Ansible will still run happily. But these are my distinctions.

Linux:

Both python and python3 are installed by default on Fedora and Debian; the versions of python (what you can install and update to) depend on the distro. A quick note about python versions and various distros:

The python language versions and pip package installations are the least-standardized and messiest management of any important package available for install.

That said, there are a few rules about installing additional python that should normally be install via pip/pip3:

Only use packages - provided by the distro - to install additional python packages.
When the distro does NOT offer a python package you need, then use pip/pip3 to install it.

We still have to get our work done, right?

Fedora Workstation:

Upgrading Python is easy:

sudo dnf upgrade -y python*
...

python3 --version
Python 3.5.2

Debian - and derivatives, like Ubuntu:

sudo apt-get upgrade python python3

To display the system-supported version info:

apt-cache policy python python3
python:
  Installed: 2.7.9-1
  Candidate: 2.7.9-1
  Version table:
 *** 2.7.9-1 0
        500 http://httpredir.debian.org/debian/ jessie/main amd64 Packages
        100 /var/lib/dpkg/status
python3:
  Installed: 3.4.2-2
  Candidate: 3.4.2-2
  Version table:
 *** 3.4.2-2 0
        500 http://httpredir.debian.org/debian/ jessie/main amd64 Packages
        100 /var/lib/dpkg/status

NOTE: getting the latest version of python for any platform requires a bit of research on your part. The Fedora Python support page and the Debian Python support page.

macOS: (Homebrewed Python/Python3)

I HIGHLY recommend Homebrew for any macOS program installations. If you're using mac-ops this is already done for you.

brew install python python3

Despite what the on-screen instructions tell you - this is all that needs to be done.

Install the awscli package

If you've made it this far, and per the directions above, the Linux the install will require sudo. The macOS install will not. Your version info should look similar - but not the same. If you're using mac-ops this is already done for you.

Linux:

Fedora Workstation

Install the distro-supported package:

sudo dnf install -y awscli
...

aws --version
aws-cli/1.11.40 Python/3.5.2 Linux/4.8.6-300.fc25.x86_64 botocore/1.5.3

Debian - and derivatives, like Ubuntu

sudo apt-get install -y awscli
...

aws --version
aws-cli/1.4.2 Python/3.4.2 Linux/3.16.0-4-amd64

macOS:

There are no distro-supported packages on macOS, only App Store updates - which are shamefully behind. Using Homebrew, all Python packages are available via pip - the way the Python gods intended. If you're using mac-ops this is already done for you.

pip3 install --upgrade pip setuptools

pip3 install --upgrade awscli

aws --version
aws-cli/1.11.51 Python/3.6.0 Darwin/16.4.0 botocore/1.5.14

The awscli Configuration Info

From here-down, the steps should be the same regardless of OS.

First, check the current configuration info; it's very likely blank. Type aws configure list and review the output:

aws configure list
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key                <not set>             None    None
secret_key                <not set>             None    None
    region                <not set>             None    None

Minimally, you should have this information:

aws_access_key_id
aws_secret_access_key
region
output <type>

If you're an AWS veteran then you already have this information; skip to configure the awscli. If not, you need to get it; here's how:

Create an AWS Account and Create a New IAM User

Open a browser and navigate to the AWS Signup and create an account.
Follow the instructions and swipe the card. There is no other way. Everything will be free for the first 12 months anyway. By then you'll be making enough money to afford this easily.
The credentials used to create the AWS account were assigned to a root level user. Do not use these credentials. Rather, create a less-privileged user.
Navigate to IAM: Services (menu; top-left) > Security, Identity > IAM.
Select Groups from the left nav.
Create a new group; Group Name: 'sysadmins' - or something intuitively obvious.
Attach Policy: AdministratorAccess. It's broad but you're just starting out. This can be narrowed later.
Review: if everything looks good: Create Group.
Select Users from the left nav.
Assign a 'User Name' to this account. To be clear, this account is for you. It's just less privileged than the root account. This will become your daily login user name.
Add Users to Group: Provide a User Name and grant 'Programmatic access' and 'AWS Management Console access'. Since this account is for you, specify 'Custom password'. Uncheck 'Require password reset'.
Click the button 'Next: Permissions'. Check the box for the 'sysadmins' group.
Click the button 'Next: Review'. If everything looks good: Create User.
Leave this browser window up for a moment.
Open the AWS Console in a new tab.
To the right of your name, note the region. For example, mine is Oregon.
Check the region index to see which region is yours. For example, my default region is in Oregon and AWS understands that as 'us-west-2'.

Configure The awscli

Back in the Terminal enter this command: aws configure --profile default

In the case of multiple profiles, default should be the first and/or primary account used.

The setup will ask you some questions, provide the answers from steps above: (example info)

AWS Access Key ID [None]: AKWHATNBBI9s8cSyoYo    # Get this from your open browser window
AWS Secret Access Key [None]: OXQP3fZjKJcfQzX/xtljcgmXDstK4HaVrCgP4ioD  # Also in the browser
Default region name [None]: us-west-2    # from Step 17 above
Default output format [None]: JSON

NOTE: if you don't know which output format to use then default to JSON.

Export Variables to the Environment

Life can be hard or easy. If you want an easy life, set some variables in ~/.bashrc so that programs like Packer and Terraform can use them. Here's how:

Edit your ~/.bashrc file and add this content. If you're using mac-ops this is already in ~/.bashrc but it's unconfigured.

###############################################################################
###                                 Amazon                                  ###
###############################################################################
complete -C "$(type -P aws_completer)" aws
#export AWS_PROFILE='default'
export AWS_CONFIG_FILE="$HOME/.aws/config"

Uncomment this line and save the file.

This is what's been done:

aws cli command completion is enabled. You can tab-complete commands now.
A value for AWS_PROFILE has been provided to your user environment. It's more of a secrete but it's only available to your environment. Nobody else can see this value, only you.
The AWS cli has been directed to the standard location of it's configuration file.

To activate these changes, open a terminal and type:

source ~/.bashrc

List the user configuration now:

aws configure list --profile default
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                  default             manual  --profile
access_key     ****************yoYo shared-credentials-file    
secret_key     ****************4ioD shared-credentials-file    
    region                us-west-2      config-file    /Users/yourUserName/.aws/config

This looks much better. NOTE: the value of 'config-file' is /Users/yourUserName/.aws/config. We'll come back to that. However IF THIS DID NOT WORK FOR YOU simply Quit/exit out of the terminal and open a new session and re-run the last step.

After this step you can close the AWS-related browser tabs.

The AWS Directory

When the profile was listed above it revealed a parameter config-file with a value of /Users/yourUserName/.aws/config. Let's dig into that a bit; the tree program shows:

tree ~/.aws/
/Users/yourUserName/.aws/
|-- config
|-- credentials
`-- shell
    |-- 1.9.9-completions.json.docs
    |-- awsshellrc
    `-- cache
        `-- completions-1.9.9.json

If these files are cat'd-out they show all the details; these are the important files:

cat ~/.aws/config 
[profile default]
region = yourAWSRegion
output = JSON

cat ~/.aws/credentials 
[default]
aws_access_key_id = AKWHATNBBI9s8cSyoYo
aws_secret_access_key = OXQP3fZjKJcfQzX/xtljcgmXDstK4HaVrCgP4ioD

Of course your values will be different than these.

Observations:

The value for AWS_PROFILE (in ~/.bashrc) matches the AWS profile name in both the config and credentials files.
AWS has been told explicitly where the AWS_CONFIG_FILE is: "$HOME/.aws/config".
It's not that AWS wouldn't otherwise be able to find this file but, for the purposes of starting out, anything that does NOT fit this description is wrong. It's simply good to understand these relationships and and be able to control the aws cli behavior.

AWS Security

The security of these files, as well as your AWS account itself, is of the greatest importance.

If these files were stolen the thief would have the ability to influence your AWS environment. Since this is not the root account (the account used when signing up) it wouldn't be the end of the world. You could just destroy this account then build and configure a new one; then the thief would be out.

However, the best offense is still a good defense. Some things to consider:

Never allow anyone the use of your laptop; if they do it should be under the guest account; kids especially.
I strongly urge you to setup (MFA) Multi-Factor Authentication for your AWS account.

MFA security is, of course, not important while experimenting in AWS; you are very low-value target after all. But, as long as you are learning, you should be versed in the subject. Besides, any good employer will enforce MFA. Plus it's super simple to setup and use:

Download the Google Authenticator available for both iPhone and Android.
Follow the instructions on the MFA page.
Configure your account default info into the Authenticator.
Sit on your hands. Review every small detail. When you are confident MFA is set up correctly...
Log out of AWS and login again.

You will see the common login page that asks for your user name and password. After you click the button, you will be directed to a page that says:

"Amazon Web Services Sign In With Authentication Device"

Check the Google Authenticator app, it will display a 6-digit number. Enter that number in the field under your email address. Then click the button to sign in.

This method is as good as any RSA SecurID with the added benefit of being totally free.

At this point it's a good idea to revisit the AWS Console Home page and resolve any issues in the 'Security Status' section. You're done when 5/5 steps have a green check next to them.

OpenSSH Keys

OpenSSH keys are the most common way of securing communications between 2 points, an origin (your laptop for example) and a destination, like AWS. As stated in the Amazon EC2 Key Pairs page:

Amazon EC2 uses public–key cryptography to encrypt and decrypt login information. Public–key cryptography uses a public key to encrypt a piece of data, such as a password, then the recipient uses the private key to decrypt the data. The public and private keys are known as a key pair.

This is an important concept. Probably the most important for your online security. I encourage you to ingest as much information on the subject of the SSH Encryption and Connection Process as you can; here is a friendly intro.

In the following steps you will create keys and associate them with your AWS account. Afterwards, all communications between you and AWS will be private. The added benefit of this is that you will not have to manually enter a password every time you issue a command to AWS. This process will trade-in an old, failed mechanism of manual logins for a new, ultra-secure method of the automatic ssh key exchange.

The net result? You'll swap your 10-20 character password for a process that involves the secure, mathematical handling of number-crunching unique strings over 300 characters long automatically. Few things are more secure than this, if any.

To begin, open the terminal again and check to see if you have an OpenSSH key-pair:

ls -al ~/.ssh/
total 16
-rw------- 1 $USER staff  809 Feb 17 12:03 authorized_keys
-rw------- 1 $USER staff 1679 Feb 17 12:04 id_rsa
-rw-r--r-- 1 $USER staff  400 Feb 17 12:04 id_rsa.pub
-rw-r--r-- 1 $USER staff  803 Feb 17 12:52 known_hosts

If you see some files called id_rsa AND id_rsa.pub then you already have keys. Skip the step of creating them.

If you don't see id_rsa AND id_rsa.pub then you have no keys. If you don't have a key-pair yet, create one. NOTE: In the following example, substitute the email address associated to the IAM account (default) for "emailAddress@domain.tld":

Open a terminal window and type:

ssh-keygen -t rsa -b 4096 -C "emailAddress@domain.tld"

When asked for a 'passphrase' - DO NOT ENTER A PASSPHRASE - just hit the Enter key for both requests. The whole process should look similar to this.

ssh-keygen -t rsa -b 4096 -C "emailAddress@domain.tld"
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/yourUserName/.ssh/id_rsa.
Your public key has been saved in /home/yourUserName/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:fFigrWSVc/gZEoznVtDZ7QvCiN8RDJowWgM3ABfBf3M yourUserName@localhost.localdomain
The key's randomart image is:
+---[RSA 4096]----+
|.o*=B  o*O o .   |
| ..+ =.*B.O . .  |
|  ..  *+.O.+ .   |
|    .o+oEo* . .  |
|     ..=S..o . . |
|        ...   .  |
|                 |
|                 |
|                 |
+----[SHA256]-----+

Check the keys again:

$ ls -al ~/.ssh/
/Users/userName/.ssh/id_rsa  
/Users/userName/.ssh/id_rsa.pub

Check the contents of the public key: (scroll right)

cat ~/.ssh/id_rsa.pub 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPeicikCRElogyXtTEbboqngSfv+sU5OmypMZUrOXNZTwaamo9yzxG0IKdejsZzwYRTQh+uSneZ8BxMlW5XKg9aVnEqPDUO8/XzkN/E74FZNgrJosZ1Ce/EqdGOoKiltMYhJGI7b3+wP1bRvnR7jIqSr6UDI3Ockwxk8GTlUx2l9+jXL7niJqVjLlHDRvrPPEhv8N4761u8eGOd2lzAtnB/maIG9o+copEXudh/HG1k2qD4TmoEfJ9RDxpzNaInmi8M3nh0MXLNvwxLc94kIPXnZAZdmUHWmd1zrjxvPv5dNvRNTyyeZKLE34GomAh8nQAzpwiCRbMO5QRg62pTtn/zJlCOvljROEdVtfxCJ1RcDvTCTu8fqXH/uEi5ZLEQahwgn4UAldz7RbDrTFw35zqf8fLaojpt47mVuBhdFIJ98MNsCxZVwVnPq6UxLL426VS3d72VBt0EAs1ZtcZcZaDPoK8/uWQPT1jxyIe6aBcO6d1mS1oLpZAlWYbmoVwP+GASPhMijDsTnaNtqLTkUD/h4zMlE1oNvk+lCeZaJT5MquI/BIK2EmmFenOuNxtnvrrrAzS2uhd+U6lb6s1Y9RXuJx4FGd/ANzkA7Hs9lKQOA/lXy0ZY6wa9tym0rngJOnjyF5zXa5NyCeWjABV7XiwLCSlNUB+ljK3A7H/LEI2yQ== emailAddress@domain.tld

The key is very long. Longer than any password you can remember. At the end the email address simply acts as a unique identifier. You could maybe have one key-pair for work and another for personal stuff. This serves as an easy way to tell the keys apart. When creating a second key-pair add the "-f "$HOME/.ssh/work_keys" option and give the key-pair any name that does not already exist on your system; like:

ssh-keygen -t rsa -b 4096 -f "$HOME/.ssh/work_keys" or ssh-keygen -t rsa -b 4096 -f "$HOME/.ssh/username_servername" or whatever.

Import the Keys into Amazon EC2

Following the above example.

aws ec2 import-key-pair --public-key-material fileb://~/.ssh/id_rsa.pub --key-name "$USER.pub"

The value id_rsa.pub for the parameter --key-name can really be anything like yourAWSUserName but, to keep things simple do it exactly as shown. Experiment with this as comfort level increases.

When you are feeling more comfortable with ssh keys and you have more than one pair. You may want to control their behavior with a configuration file. Digital Ocean describes (an intro to the subject) How To Configure Custom Connection Options for your SSH Client well-enough.

Test It Out

It's not over till the testing is complete. Open a new terminal and type: aws iam get-user. This is the AWS equivalent to the program id.

aws iam get-user
{
    "User": {
        "Path": "/",
        "UserName": "yourAWSUserName",
        "UserId": "AKWHATNBBI9s8cSyoYo",
        "Arn": "arn:aws:iam::123456789101:user/yourAWSUserName",
        "CreateDate": "2017-02-19T21:17:26Z",
        "PasswordLastUsed": "2017-02-19T21:58:02Z"
    }
}

Now use sts to validate a calling identity:

aws sts get-caller-identity
{
    "UserId": "AKWHATNBBI9s8cSyoYo",
    "Account": "648053780176",
    "Arn": "arn:aws:iam::123456789101:user/yourAWSUserName"
}

You should see all the familiar settings you used to create your account. The "Arn" (Amazon Resource Name) is some of your most important account info. You needn't memorize it but you should be able to identify inconsistencies at a glance. Review your account number, and user name. Just make a mental note of it.

Also, note the beginning of the Arn structure: "arn:aws:iam::123456789101:user/yourAWSUserName". It generally fits this format: arn:partition:service:region:account-id:resourcetype/resource. The Arn structures should be understood; check the page Amazon Resource Names (ARNs) and AWS Service Namespaces at some point.

Let's do another one: aws ec2 describe-key-pairs

aws ec2 describe-key-pairs
{
    "KeyPairs": [
        {
            "KeyName": "id_rsa.pub",
            "KeyFingerprint": "c4:73:53:3o:90:f7:6e:c9:b5:1e:0e:f4:4e:dx:6c:71"
        }
    ]
}

This is the public ssh key associated with yourAWSUserName.

The Wrap

If you made it this far, successfully, then you're installed, configured and secured. Or, in other words, ready to start building on Amazon AWS!

Good luck,

TT

Provide feedback

Saved searches

Use saved searches to filter your results more quickly