Skip to content

Paperspace Setup

Jump to bottom
Niklas Schmitz edited this page Nov 30, 2018 · 14 revisions

Initial Setup

We use Paperspace as a provider of GPUs in the cloud.

Step 0: Create Machine

Create a new machine in the web interface (Core - Compute - Machines - New Machine) choose:

  • OS: Linux Templates -> Ubuntu 18.04 // EDIT: some issues.. better choose Public Templates -> ML-in-a-box ubuntu 18
  • Machine: Hourly -> P4000
  • Storage: 50GB should be enough for basic tests, can be extended later
  • you may turn off auto snapshot to save money
  • turn on Public IP

The machine might take between minutes and a few hours for provisioning. Once it's ready, you'll receive an e-mail with the IP and the machines initial password.

Step 1: Connect to your Machine

Connect to your machine via SSH to control it through a terminal

ssh paperspace@<your paperspace Public IP>           # e.g. ssh paperspace@184.123.456.78

when prompted, enter the machines password.

Step 2: Clone Mask R-CNN repo

get the Mask R-CNN (forked from matterport implementation) on the remote machine

git clone https://github.com/BrainSegmentation/Mask_RCNN.git

Step 3: Manage Dependencies

We provide a docker image based on deepo, which installs python 3, the most used deep learning libraries, jupyter, CUDA, opencv, imgaug and others.

Make sure you have docker and nvidia-docker installed. If it's the fresh Ubuntu installation, just Get Docker CE for Ubuntu and then get NVIDIA Docker:

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
  sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-docker2

change into the Mask_RCNN directory and build the image (its name will be brainsegmentation):

sudo docker build -t brainsegmentation .

Usage

Step 0: Connect

again, ssh into your remote machine as before (make sure you already started it in the web interface)

Step 1: Docker

run the docker image

sudo nvidia-docker run -it -v ~/Documents:/Documents -p 8888:8888 brainsegmentation bash

you will now have

  • a shell within the running container
  • GPU access
  • access to the Document direcory
  • an open port on 8888 for jupyter

Step 2: Jupyter

Now we connect from your local machine/notebook to the 8888 port on the remote to establish a connection with jupyter. Open a new terminal locally and type

ssh -N -L localhost:8888:localhost:8888 paperspace@184.XXX.XXX.XX (your paperspace Public IP)

now to start the jupyter server, type on the remote terminal (within the Mask_RCNN dir)

jupyter notebook --ip 0.0.0.0 --no-browser --NotebookApp.allow_remote_access=True --allow-root

copy the displayed IP address and access it from your local browser

Step 3: Get going

play around ! but remember to turn off the paperspace machine in the web interface once you're finished!

Useful links: https://github.com/reshamas/fastai_deeplearn_part1/blob/master/tools/paperspace.md https://docs.docker.com/

Clone this wiki locally