Makes a bunch of EC2 spot priced instances and starts dask running on them.
Switch branches/tags
Nothing to show
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
daskec2lite ensured sgid is passed Jun 18, 2018
LICENSE First Commit Apr 24, 2018
MANIFEST.in First Commit Apr 24, 2018
README.md Update README.md Apr 25, 2018
README.rst First Commit Apr 24, 2018
daskec2lite-runner First Commit Apr 24, 2018
setup.py First Commit Apr 24, 2018

README.md

dask-ec2lite

Makes a bunch of EC2 spot priced instances and starts dask running on them.

Install

Sorry, I forgot to include boto3 and paramiko in setup.py, so you've got to install them yourself;

pip install boto3
pip install paramiko
pip install git+https://github.com/lionfish0/daskec2lite.git

Usage

daskec2lite --help

usage: daskec2lite [-h] [--pathtokeyfile [PATHTOKEYFILE]]
                   [--keyname [KEYNAME]] [--username [USERNAME]]
                   [--numinstances [NUM_INSTANCES]]
                   [--instancetype [INSTANCE_TYPE]] [--imageid [IMAGEID]]
                   [--spotprice [SPOTPRICE]] [--region [REGION_NAME]]
                   [--wpi [WORKERS_PER_INSTANCE]] [--sgid [SGID]] [--destroy]

Create an EC2 spot-price cluster, populate with a dask scheduler and workers.
Example: daskec2lite --pathtokeyfile '/home/mike/.ssh/research.pem' --keyname
'research' --username 'mike' --imageid ami-19a58760 --sgid sg-9146afe9

optional arguments:
  -h, --help            show this help message and exit
  --pathtokeyfile [PATHTOKEYFILE]
                        path to keyfile [required]
  --keyname [KEYNAME]   key name to use to access instances [required]
  --username [USERNAME]
                        user to log into remote instances as [required]
  --numinstances [NUM_INSTANCES]
                        number of instances to start
  --instancetype [INSTANCE_TYPE]
                        type of instance to request
  --imageid [IMAGEID]   AWS image to use [required]
  --spotprice [SPOTPRICE]
                        Spot price limit ($/hour/instance)
  --region [REGION_NAME]
                        Region to use
  --wpi [WORKERS_PER_INSTANCE]
                        Workers per instance
  --sgid [SGID]         Security Group ID [required]
  --destroy             Destroy the cluster