Skip to content
Start a cluster in EC2 for dask.distributed
Python Jupyter Notebook SaltStack Other
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
ci Save cluster region on creation (#74) Feb 21, 2017
dask_ec2 Check for a private_ip_address of boto3 instances if the public_ip_ad… Sep 25, 2017
notebooks add gridsearch notebook May 11, 2017
.coveragerc Rename dec2 to dask-ec2 (#21) Jun 20, 2016
.gitattributes Rename dec2 to dask-ec2 (#21) Jun 20, 2016
.gitignore Add travisci Feb 12, 2016
.style.yapf YAPF Feb 15, 2016
.travis.yml Rename dec2 to dask-ec2 (#21) Jun 20, 2016 Update README with link to the daskec2lite repo (#114) Jun 26, 2018 Get ready for v0.1.0 Feb 17, 2016
environment.yml Fix flake8 warnings (#73) Mar 25, 2017
requirements.txt Fix flake8 warnings (#73) Mar 25, 2017
setup.cfg Fix flake8 warnings (#73) Mar 25, 2017

Dask EC2 Build Status Coverage Status

Easily launch a cluster on Amazon EC2 configured with dask.distributed, Jupyter Notebooks, and Anaconda.


This project is not actively maintained. Instead, to deploy Dask on EC2 we recommend the use of Kubernetes. See for up-to-date information.

An experimental alternative daskec2lite is written with the aim of replicating some of the features of dask-ec2.


You also install dask-ec2 using pip:

$ pip install dask-ec2

You can also install dask-ec2 and its dependencies from the conda-forge repository using conda:

$ conda install dask-ec2 -c conda-forge


Note: dask-ec2 uses boto3 to interact with Amazon EC2. You can configure your AWS credentials using Environment Variables or Configuration Files.

The dask-ec2 up command can be used to create and provision a cluster on Amazon EC2:

$ dask-ec2 up --help
Usage: dask-ec2 up [OPTIONS]

  --keyname TEXT                Keyname on EC2 console  [required]
  --keypair PATH                Path to the keypair that matches the keyname
  --name TEXT                   Tag name on EC2
  --tags TEXT                   Additional EC2 tags.  Comma separated K:V
                                pairs: K1:V1,K2:V2
  --region-name TEXT            AWS region  [default: us-east-1]
  --vpc-id TEXT                 EC2 VPC ID
  --subnet-id TEXT              EC2 Subnet ID on the VPC
  --iaminstance-name TEXT       IAM Instance Name
  --ami TEXT                    EC2 AMI  [default: ami-d05e75b8]
  --username TEXT               User to SSH to the AMI  [default: ubuntu]
  --type TEXT                   EC2 Instance Type  [default: m3.2xlarge]
  --count INTEGER               Number of nodes  [default: 4]
  --security-group TEXT         Security Group Name  [default: dask-ec2-default]
  --security-group-id TEXT      Security Group ID (overwrites Security Group
  --volume-type TEXT            Root volume type  [default: gp2]
  --volume-size INTEGER         Root volume size (GB)  [default: 500]
  --file PATH                   File to save the metadata  [default:
  --provision / --no-provision  Provision salt on the nodes  [default: True]
  --anaconda / --no-anaconda    Bootstrap anaconda  [default: True]
  --dask / --no-dask            Install Dask.Distributed in the cluster
                                [default: True]
  --notebook / --no-notebook    Start a Jupyter Notebook in the head node
                                [default: True]
  --nprocs INTEGER              Number of processes per worker  [default: 1]
  --source / --no-source        Install Dask/Distributed from git master
                                [default: False]
  -h, --help                    Show this message and exit.

The minimal required arguments for the dask-ec2 up command are:

$ dask-ec2 up --keyname my_aws_key --keypair ~/.ssh/my_aws_key.pem

This will create a cluster.yaml in the directory that it was executed, and this file is required to use the other commands in the CLI.

Once a cluster is running, the dask-ec2 command can be used to create or destroy a cluster, ssh into nodes, or other functions:

$ dask-ec2
Usage: dask-ec2 [OPTIONS] COMMAND [ARGS]...

  --version   Show the version and exit.
  -h, --help  Show this message and exit.

  anaconda          Provision anaconda
  dask-distributed  dask.distributed option
  destroy           Destroy cluster
  notebook          Provision the Jupyter notebook
  provision         Provision salt instances
  ssh               SSH to one of the node. 0-index
  up                Launch instances
You can’t perform that action at this time.