Skip to content

Configuration File

Jump to bottom
Oleg Polosin edited this page Sep 12, 2018 · 14 revisions

By default, Spotty is looking for spotty.yaml file in the root directory of the project. Here is a basic example of such file:

project:
  name: MyProjectName
  remoteDir: /workspace/project
instance:
  region: us-east-2
  instanceType: p2.xlarge
  volumes:
    - snapshotName: MySnapshotName
      directory: /workspace
      size: 10
  docker:
    image: tensorflow/tensorflow:latest-gpu-py3

Available Parameters

project section:

  • name - the name of your project. It will be used to create S3 bucket and CloudFormation stack to run an instance.

  • remoteDir - directory where your project will be stored on the instance. It's usually a directory on the attached volume (see "instance" section).

  • syncFilters (optional) - filters to skip some directories or files during synchronization. By default, all project files will be synced with the instance. Example:

    syncFilters:
  - exclude:
      - .idea/*
      - .git/*
      - data/*
  - include:
      - data/test/*
  - exclude:
      - data/test/config

    It will skip ".idea/", ".git/" and "data/" directories except "data/test/" directory. All files from "data/test/" directory will be synced with the instance except "data/test/config" file.

    You can read more about filters here: Use of Exclude and Include Filter.

instance section:

  • region - region where your are going to run the instance (you can use command spotty spot-prices to find the cheapest region),
  • instanceType - type of the instance to run. You can find more information about types of GPU instances here: Recommended GPU Instances.
  • amiName (optional) - name of the AMI with NVIDIA Docker (default value is "SpottyAMI"). Use spotty create-ami command to create it. This AMI will be used to run your application inside the Docker container.
  • maxPrice (optional) - the maximum price per hour that you are willing to pay for a Spot Instance. By default, it's On-Demand price for chosen instance type. Read more here: Spot Instances.
  • rootVolumeSize (optional) - size of the root volume in GB. The root volume will be destroyed once the instance is terminated. Use attached volumes to store the data you need to keep (see "volumes" parameter below).
  • volumes (optional) - the list of volumes to attach to the instance:
    • snapshotName (optional) - name of the snapshot to restore. If a snapshot with this name doesn't exists, it will be created from the volume once the instance is terminated.
    • directory - directory where the volume will be mounted,
    • size (optional) - size of the volume in GB. Size of the volume cannot be less then the size of existing snapshot, but can always be increased.
    • deletionPolicy (optional) - possible values include: "update_snapshot" (value by default), "create_snapshot" and "delete". If this parameter is set to "update_snapshot", new snapshot with the same name will be created and the original snapshot will be deleted. For "create_snapshot" value, new snapshot will be created and the original snapshot will be renamed. For "delete" value, the volume will be deleted without creating a snapshot.
  • docker - Docker configuration:
    • image (optional) - the name of the Docker image that contains environment for your project. For example, you could use TensorFlow image for GPU (tensorflow/tensorflow:latest-gpu-py3). It already contains NumPy, SciPy, scikit-learn, pandas, Jupyter Notebook and TensorFlow itself. If you need to use your own image, you can specify the path to your Dockerfile in the file parameter (see below), or push your image to the Docker Hub and use its name.
    • file (optional) - relative path to your custom Dockerfile. For example, you could take TensorFlow image as a base one and add AWS CLI there to be able to download your datasets from S3: 
      FROM tensorflow/tensorflow:latest-gpu-py3

RUN pip install --upgrade \
  pip \
  awscli
    • workingDir (optional) - working directory for your custom scripts (see "scripts" section below),
    • dataRoot (optional) - directory where Docker will store all downloaded and built images. You could cache images on your attached volume to avoid downloading them from internet or building your custom image from scratch every time when you start an instance.
    • commands (optional) - commands which should be performed once your container is started. For example, you could download your datasets from S3 bucket to the project directory (see "project" section): 
      commands: |
  aws s3 sync s3://my-bucket/datasets/my-dataset /workspace/project/data
  • ports (optional) - list of ports to open. For example: 
    ports: [6006, 8888]
    It will open ports 6006 for Jupyter Notebook and 8008 for TensorBoard.

scripts section (optional):

  • This section contains customs scripts which can be run using spotty run <SCRIPT_NAME> command. The following example defines scripts train, jupyter and tensorflow:

    project:
  ...
instance:
  ...
scripts:
  train: |
    PYTHONPATH=/workspace/project
    python /workspace/project/model/train.py --num-layers 3
  jupyter: |
    /run_jupyter.sh --allow-root
  tensorboard: |
    tensorboard --logdir /workspace/outputs

Clone this wiki locally