Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
105 lines (83 sloc) 3.82 KB

Notes on Amazon AWS

AWS troubleshooting

If you’re trying to connect to something and get no response, check the firewall. Identify the machine’s security group, go to EC2 settings for that security group, and ensure they allow the activity you’re trying.

AWS Client Setup

How to install the AWS client on a local computer. The client is already installed on EC2 when using the Amazon Linux.

  1. Install the AWS command-line tools.
  2. Configure the command-line tools
    • First, you need an access key ID and secret generated by the IAM service.
    • aws configure
    • enter your ID and secret
    • set the default region to us-east-1
    • leave the output format blank

Working With S3

Listing bucket contents aws s3 ls s3://dryad-upload/

Copying to a bucket aws s3 cp myfile.txt s3://dryad-upload

Copying contents of a bucket aws s3 sync s3://mybucket s3://mybucket2

Working With EC2

Spinning up an EC2 machine

  • Use the Amazon Linux unless you have a good reason to do otherwise.
  • It’s not necessary to add extra storage beyond the default, you can always add extra disks later.
  • Tag the instance with a name that matches its purpose.
  • Create a new keypair.

Logging into an EC2 machine

  • move the keypair to somewhere you can access
  • get the public DNS name of the machine from the EC2 console; e.g. ec2-52-90-82-228.compute-1.amazonaws.com
  • ssh -i myKeypair.pem ec2-user@ec2-52-90-82-228.compute-1.amazonaws.com

Generating a keypair

  • aws ec2 create-key-pair --key-name MyKeyPair

Server permissions and access control with IAM

In general, user permissions are set up in IAM, and OpsWorks automatically applies them to the appropriate servers.

Users can login to servers with a command like: ssh -i ~/.ssh/[your-keyfile] someuser@machinename.datadryad.org

About IAM

  • Users belong to groups
  • Groups get policies
  • Policies contain access control rules
  • When accessing the AWS website, IAM users must login via the special “IAM console link” on the administrator’s IAM dashboard. Dryad’s login link: https://datadryad.signin.aws.amazon.com/console
  • Roles are a way for a single user to have multiple sets of permissions, sort of like sudo. They can be useful for allowing external applications to access a set of resources.
  • To register a new user
    • Log in to the AWS website as root (from the main login screen, follow the link near the bottom)
    • Go to the IAM section. Create the new user there (make sure it’s in the correct region) and give them console permissions and ssh permissions and add them to the Developers access group.
    • Then go to the OpsWorks console. From there, you should be able to edit users and import new IAM users from the region. The new user should then show up and be editable. You can turn on self-management of the user there, allowing them to set their own ssh public key.
    • You need to then go to the OpsWorks stack and “run command” with the command “configure” to goose the various servers into recognizing the new user, but after that, the user should be able to ssh to any of our AWS servers.

Monitoring

Monitoring is performed by Amazon CloudWatch. URL-based alarms can be created and managed through Route 53, but these appear in CloudWatch, and CloudWatch is required to manage anything that cannot be represented as a URL.

Most alarms are tied to the DryadAdmin notifications. When a notification is sent by one of these alarms, it goes to Slack, the admin email, and developers' phones.

You can’t perform that action at this time.