Based on the fastai course.
The point of this repository is to abstract away
aws CLI complications. By tying some state on the AWS cloud to your laptop, cloud machines can be a one-liner away. All action centers around directories in the following managed location in your home directory:
If you launch a machine with name
name, then this directory will be created on your computer:
That directory will have a couple of scripts that give the simplified interface to AWS you wish you had, it's like magic!
~/aws-instances/name/start will start up the cloud machine ~/aws-instances/name/ip will print the public IP of the running instance ~/aws-instances/name/ssh will ssh into the running instance ~/aws-instances/name/stop will stop the currently-running instance ~/aws-instances/name/state will describe the current state of the instance ~/aws-instances/name/delete will stop the instance and delete all metadata about the instance from both your computer and AWS
You're only billed on EC2 when the machine is in running state, but even if you stop the machine you'll still be charged for the cost of storing the stopped machine image on EBS.
Launching a Machine
Requires that the default
aws cli user has been set up (i.e.,
~/.aws/config should have your API keys). This is a thin wrapper with defaults around:
setup-instance.sh ami instancetype name open-extra
This will create and start a cloud machine on AWS. Security info:
- Ports 6006 (Tensorboard), 8888-8898 (Jupyter), and 22 (ssh) are open if
true. If it's
false, only ssh is open.
- ssh is only accessible by a public key stored in your
- No Jupyter process is started on the server by default. If you do run Jupyter, I suggest following these guidelines for security.
Cleanup. If you send a
ctrl-C during the script, or there is an issue during resource allocation (invalid instance type or instance limit hit), then as long as you don't freak out and send another quit/terminate signal the script will attempt to cleanly de-allocate all reserved AWS resources. After completion the local folder
~/aws-instances/name for server with name
name is deleted. If stuff really goes wrong then you might need to manually log onto the AWS web interface to clean things up, but that shouldn't happen through normal use.
Installing Deep Learning Software
If you have a live instance running (generated by a script such as the above), then you can install my deep learning setup on it (provided you want TensorFlow, Keras, my .bashrc, etc.). See
server/server-install-deeplearn.sh for details, and to start the install on your laptop use:
Note that this is not well-polished and has lots of details specific to me right now, use at your own risk. This is intended to install on top of Ubuntu 16.04 and add tools for interactive development. This probably has lots of features not necessary for just running existing code.