Notes on Amazon AWS
If you’re trying to connect to something and get no response, check the firewall. Identify the machine’s security group, go to EC2 settings for that security group, and ensure they allow the activity you’re trying.
AWS Client Setup
How to install the AWS client on a local computer. The client is already installed on EC2 when using the Amazon Linux.
- Install the AWS command-line tools.
- On OSX, this is
sudo pip install awscli --ignore-installed six
- Instructions for other OS’s
- On OSX, this is
- Configure the command-line tools
- First, you need an access key ID and secret generated by the IAM service.
- enter your ID and secret
- set the default region to
- leave the output format blank
Working With S3
Listing bucket contents
aws s3 ls s3://dryad-upload/
Copying to a bucket
aws s3 cp myfile.txt s3://dryad-upload
Copying contents of a bucket
aws s3 sync s3://mybucket s3://mybucket2
Working With EC2
Spinning up an EC2 machine
- Use the Amazon Linux unless you have a good reason to do otherwise.
- It’s not necessary to add extra storage beyond the default, you can always add extra disks later.
- Tag the instance with a name that matches its purpose.
- Create a new keypair.
Logging into an EC2 machine
- move the keypair to somewhere you can access
- get the public DNS name of the machine from the EC2 console;
ssh -i myKeypair.pem firstname.lastname@example.org
Generating a keypair
aws ec2 create-key-pair --key-name MyKeyPair
Server permissions and access control with IAM
In general, user permissions are set up in IAM, and OpsWorks automatically applies them to the appropriate servers.
Users can login to servers with a command like:
ssh -i ~/.ssh/[your-keyfile] email@example.com
- Users belong to groups
- Groups get policies
- Policies contain access control rules
- When accessing the AWS website, IAM users must login via the special “IAM console link” on the administrator’s IAM dashboard. Dryad’s login link: https://datadryad.signin.aws.amazon.com/console
- Roles are a way for a single user to have multiple sets of permissions, sort of like sudo. They can be useful for allowing external applications to access a set of resources.
- To register a new user
- Log in to the AWS website as root (from the main login screen, follow the link near the bottom)
- Go to the IAM section. Create the new user there (make sure it’s in the correct region) and give them console permissions and ssh permissions and add them to the Developers access group.
- Then go to the OpsWorks console. From there, you should be able to edit users and import new IAM users from the region. The new user should then show up and be editable. You can turn on self-management of the user there, allowing them to set their own ssh public key.
- You need to then go to the OpsWorks stack and “run command” with the command “configure” to goose the various servers into recognizing the new user, but after that, the user should be able to ssh to any of our AWS servers.
Monitoring is performed by Amazon CloudWatch. URL-based alarms can be created and managed through Route 53, but these appear in CloudWatch, and CloudWatch is required to manage anything that cannot be represented as a URL.
Most alarms are tied to the DryadAdmin notifications. When a notification is sent by one of these alarms, it goes to Slack, the admin email, and developers' phones.