Getting Started

Pre-requisites

The content of this tutorial assumes you are familiar with the following:

• Basic Linux system administration commands
• Using SSH to access remote servers, including creating/adding keys
• Setting environment variables

About AWS Account Access Permissions

You must have access to an AWS account to follow this tutorial.

Using a temporary account

Using a standard account

Access should be via an IAM user login; using the root account credentials is not recommended. If you are working in a temporary or sandbox account, the "AdministratorAccess" IAM role will allow you to perform all of the required changes; this role will need to be applied to your user account, or can be attached to an EC2 instance. If you wish to use the AWS CLI with authentication as an IAM user, you will need to provide your API keys when configuring the CLI. If instead you wish to use a role attached to an EC2 instance, you will automatically have the required permissions once logged in to that instance, without the need for API keys.

If you are working within an organisationally-managed AWS account and do not have permissions to modify IAM users/roles, contact your account admin. They will be able to assist by either provisioning a role to your user account with appropriate (though likely more limited) permissions, or providing access to an EC2 instance which has the required role. Provisioning a static HPC cluster requires the ability to create EC2 instances and networking components, while deploying a dynamically scaling cluster additionally requires the creation of custom IAM policies and roles, which in turn allow instances to be created and terminated automatically.

All steps documented here can be completed without exceeding default AWS account quotas. Wherever possible, we suggest using t2.micro or t3.micro instances for testing; while these instances are not appropriate for real HPC workloads, they are low-cost and for new accounts can be used for a limited number of instance-hours per month for under the free tier.

Using an EC2 instance as a bastion/entry-point

Using a bastion host as the entrypoint for any SSH connections into an AWS account is a simple way to reduce security risks. The bastion host instance should be configured to allow inbound connections only on port 22, and if possible only from a fixed IP address or IP range.

The process to create an EC2 instance via the AWS console is described here and the various methods of connecting to the instance are documented here.

For users wishing to interact with AWS using the CLI from their local device, API keys will need to be created for their IAM user account. These keys should then be set either within the AWS CLI config file or the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables of the user. For more details, see the documentation here.

If you choose to use an instance profile rather than user-owned API keys, you will need to create a profile based on the desired IAM role - the process for completing these steps is described here.

Key points to be aware of:

• The default username on an EC2 instance depends on the AMI - for marketplace images, the most common choices are likely to be ec2-user (for Amazon Linux and Amazon Linux 2) or centos for base CentOS images. Using Alinux or Alinux2 images as the OS for a simple SSH access point is fine, however CentOS should be used for any instances where the OpenHPC software stack is to be installed.
• ...

Using the Cloud9 IDE as a bastion/entry-point

[Description of what C9 is, what its advantages are for this use-case]

[Description of creating a C9 instance in the console]