aws

ngs-docs · Oct 11, 2016 · ce14f94 · ce14f94
1 parent ea02555
commit ce14f94
Show file tree

Hide file tree

Showing 100 changed files with 733 additions and 0 deletions.
diff --git a/aws/boot.rst b/aws/boot.rst
@@ -0,0 +1,97 @@
+**************************************
+Start an Amazon Web Services computer:
+**************************************
+
+This page shows you how to create a new "AWS instance", or a running
+computer.
+
+----
+
+Start at the Amazon Web Services console (http://aws.amazon.com/ and
+sign in to the console).
+
+0. Select "EC2 - virtual servers in the cloud"
+==============================================
+
+.. thumbnail:: images/boot-0.png
+   :width: 20%
+
+1. Switch to zone US West (N California)
+========================================
+
+.. thumbnail:: images/boot-1.png
+   :width: 20%
+
+2. Click on "Launch instance."
+==============================
+
+3. Select "Community AMIs."
+===========================
+
+.. thumbnail:: images/boot-2.png
+   :width: 20%
+
+4. Search for ami-05384865 (ubuntu-wily-15.10-amd64-server)
+===========================================================
+
+Use ami-05384865.
+
+.. thumbnail:: images/boot-3.png
+   :width: 20%
+
+5. Click on "Select."
+=====================
+
+6. Choose m4.large.
+===================
+
+.. thumbnail:: images/boot-4.png
+   :width: 20%
+
+7. Click "Review and Launch."
+=============================
+
+8. Click "Launch."
+==================
+
+.. thumbnail:: images/boot-5.png
+   :width: 20%
+
+9. Select "Create a new key pair."
+==================================
+
+Note: you only need to do this the first time you create an instance.
+If you know where your amazon-key.pem file is, you can select 'Use an
+existing key pair' here.  But you can always create a new key pair if
+you want, too.
+
+If you have an existing key pair, go to step 12, "Launch instance."
+
+.. thumbnail:: images/boot-6.png
+   :width: 20%
+
+10. Enter name 'amazon-key'.
+============================
+
+11. Click "Download key pair."
+==============================
+
+12. Click "Launch instance."
+============================
+
+13. Select View instances (lower right)
+=======================================
+
+.. thumbnail:: images/boot-8.png
+   :width: 20%
+
+14. Bask in the glory of your running instance
+==============================================
+
+Note that for your instance name you can use either "Public IP" or
+"Public DNS". Here, the machine only has a public IP.
+
+.. thumbnail:: images/boot-9.png
+   :width: 20%
+
+You can now :doc:`login-shell` or :doc:`configure-firewall`.
diff --git a/aws/configure-firewall.rst b/aws/configure-firewall.rst
@@ -0,0 +1,60 @@
+********************************
+Configure your instance firewall
+********************************
+
+Normally, Amazon computers only allow shell logins via ssh (port 22
+access).  If we want to run a Web service or something else, we need
+to give the outside world access to other network locations on the
+computer.
+
+Below, we will open ports 8000-9000, which will let us run things like
+RStudio Server.  If you want to run other things, like a Web server,
+you'll need to find the port(s) associated with those services and
+open those instead of 8000-9000.  (Tip: Web servers run on port 80.)
+
+1. Select 'Security Groups'
+---------------------------
+
+Find "Security Groups" in the lower pane of your instance's
+information page, and click on "launch-wizard-N".
+
+.. thumbnail:: images/network-0.png
+   :width: 20%
+
+2. Select 'Inbound'
+-------------------
+
+.. thumbnail:: images/network-1.png
+   :width: 20%
+
+3. Select 'Edit'
+----------------
+
+.. thumbnail:: images/network-2.png
+   :width: 20%
+
+4. Select 'Add Rule'
+--------------------
+
+.. thumbnail:: images/network-3.png
+   :width: 20%
+
+5. Enter rule information
+-------------------------
+
+Add a new rule: Custom TCP, 8000-9000, Source Anywhere.
+
+6. Select 'Save'.
+-----------------
+
+7. Return to the Instances page.
+--------------------------------
+
+.. thumbnail:: images/network-4.png
+   :width: 20%
+
+----
+
+You're done!
+
+Go back to the index: :doc:`index`
diff --git a/aws/creating-ami.rst b/aws/creating-ami.rst
@@ -0,0 +1,23 @@
+**************************************
+Creating your own Amazon Machine Image
+**************************************
+
+1. Actions, Create image
+------------------------
+
+.. thumbnail:: images/create-ami-1.png
+   :width: 20%
+
+2. Fill out name and description
+--------------------------------
+
+.. thumbnail:: images/create-ami-2.png
+   :width: 20%
+
+3. Wait for it to become available
+----------------------------------
+
+.. thumbnail:: images/create-ami-3.png
+   :width: 20%
+
+Go back to the index: :doc:`index`
diff --git a/aws/images/.DS_Store b/aws/images/.DS_Store
diff --git a/aws/images/add-volume-1.png b/aws/images/add-volume-1.png
diff --git a/aws/images/add-volume-1b.png b/aws/images/add-volume-1b.png
diff --git a/aws/images/add-volume-1c.png b/aws/images/add-volume-1c.png
diff --git a/aws/images/add-volume-1d.png b/aws/images/add-volume-1d.png
diff --git a/aws/images/add-volume-2.png b/aws/images/add-volume-2.png
diff --git a/aws/images/add-volume-3.png b/aws/images/add-volume-3.png
diff --git a/aws/images/add-volume-4.png b/aws/images/add-volume-4.png
diff --git a/aws/images/add-volume-5.png b/aws/images/add-volume-5.png
diff --git a/aws/images/add-volume-6.png b/aws/images/add-volume-6.png
diff --git a/aws/images/bak/1.tiff b/aws/images/bak/1.tiff
diff --git a/aws/images/bak/10.tiff b/aws/images/bak/10.tiff
diff --git a/aws/images/bak/11.tiff b/aws/images/bak/11.tiff
diff --git a/aws/images/bak/2.tiff b/aws/images/bak/2.tiff
diff --git a/aws/images/bak/3.tiff b/aws/images/bak/3.tiff
diff --git a/aws/images/bak/4.tiff b/aws/images/bak/4.tiff
diff --git a/aws/images/bak/5.tiff b/aws/images/bak/5.tiff
diff --git a/aws/images/bak/6.tiff b/aws/images/bak/6.tiff
diff --git a/aws/images/bak/7.tiff b/aws/images/bak/7.tiff
diff --git a/aws/images/bak/8.tiff b/aws/images/bak/8.tiff
diff --git a/aws/images/bak/9.tiff b/aws/images/bak/9.tiff
diff --git a/aws/images/bak/create-ami-1.tiff b/aws/images/bak/create-ami-1.tiff
diff --git a/aws/images/bak/create-ami-2.tiff b/aws/images/bak/create-ami-2.tiff
diff --git a/aws/images/bak/create-ami-3.tiff b/aws/images/bak/create-ami-3.tiff
diff --git a/aws/images/bak/network-1.tiff b/aws/images/bak/network-1.tiff
diff --git a/aws/images/bak/network-2.tiff b/aws/images/bak/network-2.tiff
diff --git a/aws/images/bak/network-3.tiff b/aws/images/bak/network-3.tiff
diff --git a/aws/images/bak/network-4.tiff b/aws/images/bak/network-4.tiff
diff --git a/aws/images/boot-0.png b/aws/images/boot-0.png
diff --git a/aws/images/boot-1.png b/aws/images/boot-1.png
diff --git a/aws/images/boot-10.png b/aws/images/boot-10.png
diff --git a/aws/images/boot-11.png b/aws/images/boot-11.png
diff --git a/aws/images/boot-2.png b/aws/images/boot-2.png
diff --git a/aws/images/boot-3.png b/aws/images/boot-3.png
diff --git a/aws/images/boot-4.png b/aws/images/boot-4.png
diff --git a/aws/images/boot-5.png b/aws/images/boot-5.png
diff --git a/aws/images/boot-6.png b/aws/images/boot-6.png
diff --git a/aws/images/boot-7.png b/aws/images/boot-7.png
diff --git a/aws/images/boot-8.png b/aws/images/boot-8.png
diff --git a/aws/images/boot-9.png b/aws/images/boot-9.png
diff --git a/aws/images/create-ami-1.png b/aws/images/create-ami-1.png
diff --git a/aws/images/create-ami-2.png b/aws/images/create-ami-2.png
diff --git a/aws/images/create-ami-3.png b/aws/images/create-ami-3.png
diff --git a/aws/images/create-snapshot-1.png b/aws/images/create-snapshot-1.png
diff --git a/aws/images/create-snapshot-2.png b/aws/images/create-snapshot-2.png
diff --git a/aws/images/create-snapshot-3.png b/aws/images/create-snapshot-3.png
diff --git a/aws/images/create-snapshot-4.png b/aws/images/create-snapshot-4.png
diff --git a/aws/images/network-0.png b/aws/images/network-0.png
diff --git a/aws/images/network-1.png b/aws/images/network-1.png
diff --git a/aws/images/network-2.png b/aws/images/network-2.png
diff --git a/aws/images/network-3.png b/aws/images/network-3.png
diff --git a/aws/images/network-4.png b/aws/images/network-4.png
diff --git a/aws/images/orig/add-volume-1.png b/aws/images/orig/add-volume-1.png
diff --git a/aws/images/orig/add-volume-2.png b/aws/images/orig/add-volume-2.png
diff --git a/aws/images/orig/add-volume-3.png b/aws/images/orig/add-volume-3.png
diff --git a/aws/images/orig/add-volume-4.png b/aws/images/orig/add-volume-4.png
diff --git a/aws/images/orig/add-volume-5.png b/aws/images/orig/add-volume-5.png
diff --git a/aws/images/orig/add-volume-6.png b/aws/images/orig/add-volume-6.png
diff --git a/aws/images/orig/boot-1.png b/aws/images/orig/boot-1.png
diff --git a/aws/images/orig/boot-10.png b/aws/images/orig/boot-10.png
diff --git a/aws/images/orig/boot-11.png b/aws/images/orig/boot-11.png
diff --git a/aws/images/orig/boot-2.png b/aws/images/orig/boot-2.png
diff --git a/aws/images/orig/boot-3.png b/aws/images/orig/boot-3.png
diff --git a/aws/images/orig/boot-4.png b/aws/images/orig/boot-4.png
diff --git a/aws/images/orig/boot-5.png b/aws/images/orig/boot-5.png
diff --git a/aws/images/orig/boot-6.png b/aws/images/orig/boot-6.png
diff --git a/aws/images/orig/boot-7.png b/aws/images/orig/boot-7.png
diff --git a/aws/images/orig/boot-8.png b/aws/images/orig/boot-8.png
diff --git a/aws/images/orig/boot-9.png b/aws/images/orig/boot-9.png
diff --git a/aws/images/orig/create-ami-1.png b/aws/images/orig/create-ami-1.png
diff --git a/aws/images/orig/create-ami-2.png b/aws/images/orig/create-ami-2.png
diff --git a/aws/images/orig/create-ami-3.png b/aws/images/orig/create-ami-3.png
diff --git a/aws/images/orig/create-snapshot-1.png b/aws/images/orig/create-snapshot-1.png
diff --git a/aws/images/orig/create-snapshot-2.png b/aws/images/orig/create-snapshot-2.png
diff --git a/aws/images/orig/create-snapshot-3.png b/aws/images/orig/create-snapshot-3.png
diff --git a/aws/images/orig/create-snapshot-4.png b/aws/images/orig/create-snapshot-4.png
diff --git a/aws/images/orig/network-1.png b/aws/images/orig/network-1.png
diff --git a/aws/images/orig/network-2.png b/aws/images/orig/network-2.png
diff --git a/aws/images/orig/network-3.png b/aws/images/orig/network-3.png
diff --git a/aws/images/orig/network-4.png b/aws/images/orig/network-4.png
diff --git a/aws/images/orig/terminate-1.png b/aws/images/orig/terminate-1.png
diff --git a/aws/images/orig/terminate-2.png b/aws/images/orig/terminate-2.png
diff --git a/aws/images/orig/terminate-3.png b/aws/images/orig/terminate-3.png
diff --git a/aws/images/terminate-1.png b/aws/images/terminate-1.png
diff --git a/aws/images/terminate-2.png b/aws/images/terminate-2.png
diff --git a/aws/images/terminate-3.png b/aws/images/terminate-3.png
diff --git a/aws/index.rst b/aws/index.rst
@@ -0,0 +1,38 @@
+.. labibi documentation master file, created by
+   sphinx-quickstart on Sun Nov  4 10:10:29 2012.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Starting up an Amazon Web Services machine
+==========================================
+
+Start here: :doc:`boot`
+-----------------------
+
+Full table of contents:
+-----------------------
+
+.. toctree::
+   :maxdepth: 2
+
+   boot
+   login-shell
+   login-shell-unix
+   login-shell-win
+   configure-firewall
+   install-rstudio-server
+   creating-ami
+   volumes-and-snapshots
+   terminate-instance
+   things-to-mention
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+
+
+.. @@ add AWS costs
+   
diff --git a/aws/install-rstudio-server.rst b/aws/install-rstudio-server.rst
@@ -0,0 +1,112 @@
+***********************************
+Running RStudio Server in the cloud
+***********************************
+
+In this section, we will run RStudio Server on a remote Amazon machine.
+This will require starting up an instance, configuring its network firewall,
+and installing and running some software.
+
+.. @@remember to terminate
+.. @@can we reboot and have it sart up again?
+.. @@diagram laying out zone etc.
+
+Reference documentation for running RStudio Server on Ubuntu:
+
+   https://www.rstudio.com/products/rstudio/download-server/
+
+-----
+
+1. Start up an Amazon instance
+------------------------------
+
+Start an ami-05384865 on an m4.xlarge machine, as per the instructions here:
+
+:doc:`boot`.
+
+2. Configure your network firewall
+----------------------------------
+
+Normally, Amazon computers only allow shell logins via ssh.
+Since we want to run a Web service, we need to give the outside world
+access to other network locations on the computer.
+
+Follow these instructions:
+
+:doc:`configure-firewall`
+
+(You can do this while the computer is booting.)
+
+3. Log in via the shell
+-----------------------
+
+Follow these instructions to log in via the shell:
+
+:doc:`login-shell`.
+
+4. Set a password for the 'ubuntu' account
+------------------------------------------
+
+Amazon Web Services computers normally require a key (the .pem file)
+instead of a login password, but RStudio Server will need us to log in
+with a password.  So we need to configure a password for the account
+we're going to use (which is 'ubuntu')
+
+Create a password like so::
+  
+     sudo passwd ubuntu
+
+and set it to something you'll remember.
+
+5. Install R and the gdebi tool
+-------------------------------
+
+.. @@ reference debian install instructions https://help.ubuntu.com/community/AptGet/Howto and https://www.debian.org/doc/manuals/debian-faq/ch-pkgtools.en.html
+
+Update the software catalog and install a few things::
+
+     sudo apt-get update && sudo apt-get -y install gdebi-core r-base
+
+This will take a few minutes.
+
+6. Download & install RStudio Server
+------------------------------------
+
+::
+   
+     wget https://download2.rstudio.org/rstudio-server-0.99.891-amd64.deb
+     sudo gdebi -n rstudio-server-0.99.891-amd64.deb
+
+Upon success, you should see::
+
+   Mar 07 15:20:18 ip-172-31-6-68 systemd[1]: Starting RStudio Server...
+   Mar 07 15:20:18 ip-172-31-6-68 systemd[1]: Started RStudio Server.
+
+7. Open your RStudio Server instance
+------------------------------------
+
+Finally, go to 'http://' + your hostname + ':8787' in a browser,
+eg. ::
+
+   http://ec2-XX-YY-33-165.us-west-1.compute.amazonaws.com:8787/
+
+and log into RStudio with username 'ubuntu' and the password
+you set it to above.
+
+Voila!
+
+----
+
+You can now just go ahead and use this, or you can "stop" it, or you
+can freeze into an AMI for later use.
+
+Note that on reboot, RStudio Server will start up again and all your files
+will be there.
+
+Go back to the index: :doc:`index`.
+
+.. @@CTB demonstrate graphing, etc.
+.. revisiting what we did...
+
+.. @@ rebooting; converting to AMI
+   
+.. @@ meditations on file transfer
diff --git a/aws/installing-software.rst b/aws/installing-software.rst
@@ -0,0 +1,13 @@
+'blastp' doesn't exist.
+
+sudo apt-get update
+sudo apt-cache search ncbi blast
+sudo apt-get -y install ncbi-blast+
+
+now 'blastp' exists.
+
+=> getting data
+
+@@amis locked to region
+@@note AMIs => region lock
+@@set permission on amis
diff --git a/aws/login-shell-unix.rst b/aws/login-shell-unix.rst
@@ -0,0 +1,45 @@
+**************************************************
+Log into your instance from a Mac or Linux machine
+**************************************************
+
+You'll need to do two things: first, set the permissions on
+``amazon-key.pem``::
+
+   chmod og-rwx ~/Downloads/amazon-key.pem
+
+Then, ssh into your new machine using your key::
+
+   ssh -i ~/Downloads/amazon-key.pem -l ubuntu MACHINE_NAE
+
+where you should replace MACHINE_NAME with the public IP or hostname
+of your EC2 instance, which is located at the top of the host
+information box (see screenshot below).  It should be something like
+``54.183.148.114`` or ``ec2-XXX-YYY.amazonaws.com``.
+
+Here are some screenshots!
+
+Change permissions and execute ssh
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. thumbnail:: images/boot-10.png
+   :width: 20%
+
+Successful login
+~~~~~~~~~~~~~~~~
+
+.. thumbnail:: images/boot-11.png
+   :width: 20%
+
+Host information box - MACHINE_NAME location
+============================================
+
+.. thumbnail:: images/boot-9.png
+   :width: 20%
+
+----
+
+Logging in is the starting point for most of the follow-on tutorials.
+For example, you can now install and run software on your EC2 instance.
+
+Go back to the top page to continue: :doc:`index`
+