aws-scaleio

A project to stand up a ScaleIO cluster in AWS, for testing against EMC {code} projects. This project is intended to be a component in a continuous-integration pipeline, so we can stop testing against the ScaleIO cluster in Kenny's garage automate testing in a more distributable fashion.

This template uses a custom AMI image that is (currently) only available in the US-West-1 (aka N.California) region. You've been warned!

##Usage

Launch this template currently in the US-West-1 (aka N.California) region only using AWS Cloudformation.

This template uses three 't2.medium' instances, which, in the N.California region, cost $0.068 per hour to run, so the AWS EC2 compute usage for this cluster should run you about $4.89/day to keep running. The template provisions six EBS volumes in total - three for the operating systems, and three 100-gigabyte volumes for storage. Pricing on EBS volumes is a little harder to figure out, but it should also be negligible.

The password for the ScaleIO admin is 'F00barbaz'. Other places a password might be used, same thing.

###AWS Web GUI Using the AWS web gui, in the services selection window:

• click 'Cloudformation' under 'Management Tools'
• click 'Create Stack', then 'Choose a Template'
• click 'Upload file to S3', and upload the .json file from this repo
• give the stack a name (like ScaleIOTesting or something)
• select a keypair that exists in the N.California region!
• alter the ssh source CIDR if you like, or don't
• click next, add tags if you want, or don't, then click next
• review your settings and click 'Create' to create the stack

The stack will take approximately two minutes to build, and then the nodes should be available for ssh login.

###AWS Commandline To launch this stack using AWS commandline tools, use a commandline similar to the following:

[user@host] ~/ $ aws cloudformation create-stack --stack-name ScaleIOTesting \
--template-body file:////home/ME/dev/aws-scaleio/ScaleIO_Testing_Cluster.json \
--parameters ParameterKey=KeyName,ParameterValue=MYKEY

The stack will take approximately two minutes to build, and then should be available for login.

##Once It's Running This template uses Red Hat Enterprise Linux (RHEL) as the base operating system, so to log in with ssh use the username "ec2-user".

The script launches three AWS instances of type 't1.medium', with the internal IP addresses 10.0.0.11, 10.0.0.12 and 10.0.0.13. The ScaleIO installer was deployed on 10.0.0.11 and then used to install ScaleIO v2.0.0-5014 onto the three nodes.

Each of the nodes has a 120Gb SSD drive attached - this drive appears as /dev/xdvc.

When the machines come up, there appears to be a bit of a race condition between 10.0.0.11 and 10.0.0.12 as to whom will become the MDM master, whatever that is. SSHing to on or more of the machines until you find the master, then issuing the following command:

scli --login --username admin --password F00barbaz

should log you into the ScaleIO MDM, then you should be able to manipulate the storage volumes.

##Using ScaleIO for n00bs

ScaleIO is awfully cool, but can be a little confusing if you haven't used it before. Here's an excellent whitepaper on how it generally works, and here's a couple of tips that I found useful to get started, having used a bunch of other big-storage thingeys before.

###Vocabulary MDM - Metadata Manager. Manages and configures the cluster. In this setup it can be either 10.0.0.11 or 10.0.0.12, depending on who won the race on startup. ¯\(ツ)/¯

SDS - ScaleIO Data Server. Where data is stored. In this cluster, all three servers are configured as SDS's.

SDC - ScaleIO Data Client. Where volumes can be mounted - typically any server you'd want a volume to be mounted on, and in this cluster all three servers are configured as SDC's.

###Commands

Log into the MDM as the admin user (note that the MDM IP might be 10.0.0.12): scli --login --username admin --password F00barbaz --mdm_ip 10.0.0.11

Create a new 20-gig volume named "testvol":: scli --add_volume --volume_name testvol --size_gb 20 --storage_pool_name default --protection_domain_name default --mdm_ip 10.0.0.11

Map that volume to the server at 10.0.0.12, so it can be mounted: scli --map_volume_to_sdc --sdc_ip 10.0.0.12 --mdm_ip 10.0.0.11 --volume_name testvol

Mounting that volume on 10.0.0.12 (executed as root on 10.0.0.12): mkfs.ext4 /dev/scinia && mount /dev/scinia /mnt/whatever

Unmapping the volume (should be unmounted first to avoid errors): scli --unmap_volume_from_sdc --sdc_ip 10.0.0.12 --mdm_ip 10.0.0.11 --volume_name testvol

Get a list of all volumes in the cluster: scli --query_all_volumes

##Using REX-Ray

• Install the latest version of REX-Ray from the REX-Ray Github repository

curl -sSL https://dl.bintray.com/emccode/rexray/install | sh -s stable

• Create /etc/rexray/config.yml and add the following entries to that file:

rexray:
  storageDrivers:
  - scaleio
scaleio:
  endpoint: https://10.0.0.11/api
  insecure: true
  userName: admin
  password: F00barbaz
  systemId: 39f2e3fe27fbc1dc
  protectionDomainName: default
  storagePoolName: default

• test with rexray volume list (success should return nothing at all)
• read the docs at (http://rexray.readthedocs.io/en/stable/) for more information.