Skip to content

Configure Gigantum

Jump to bottom
Randal Burns edited this page Sep 6, 2018 · 6 revisions

Install Gigantum on your Laptop

Build an AWS GPU accelerated Linux machine

For CUDA 9.0 Ubuntu 16.04 (TensorFlow) on a p2.xlarge instance with a K80 GPU.

  • Launch an AWS instance using Ubuntu 16.04 LTS
    • You will want to increase the default storage. I used 64GB.
  • Install CUDA, docker, nvidia-docker
sudo apt udpate
sudo apt install build-essential
sudo apt-get upgrade -y linux-aws
sudo apt install awscli
sudo apt-get install -y gcc make linux-headers-$(uname -r)

# reboot

# get cuda -- this is 9.0
wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda_9.0.176_384.81_linux-run
chmod +x ./cuda_9.0.176_384.81_linux-run
sudo ./cuda_9.0.176_384.81_linux-run

# check to make sure that the driver works
nvidia-smi

# get docker
curl -fsSL get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker ubuntu

# get docker-nvidia
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install nvidia-docker2

# logout and login to reset groups
# make sure that nvidia-docker works
sudo pkill -SIGHUP dockerd
docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
  • Install python 3.6.5 and gigantum
sudo add-apt-repository ppa:jonathonf/python-3.6
sudo apt-get update
sudo apt-get install python3.6 python3.6-dev python3.6-venv
python3.6 -m venv ~/venvs/gigantum
source ~/venvs/gigantum/bin/activate
pip install -U pip
pip install gigantum
  • Update the gigantum package to use GPU enabled docker images
wget https://raw.githubusercontent.com/randalburns/atidic-fall18/master/gpu-edge-update.sh
chmod +x gpu-edge-update.sh
./gpu-edge-update.sh
# start the edge version
gigantum start -e
  • Create an ssh tunnel to access the Gigantum app from a local browser. Run this from laptop
ssh -i ~/.ssh/XXX.pem -L 10000:ec2-52-15-155-150.us-east-2.compute.amazonaws.com:10000 ubuntu@ec2-52-15-155-150.us-east-2.compute.amazonaws.com
  • Open browser to localhost:10000 login and verify GPU is enabled.
  • Create project
  • Select GPU Ubuntu 16.04 base
  • Wait for build to complete
  • Launch jupyter (Start->Run->Open Jupyter)
  • In Jupyter, in a terminal, run the command nvidia-smi

For CUDA 9.2 on Ubuntu 18.04 (on a p2.xlarge instance with a K80 GPU).

Most the same, except

  • Different CUDA driver.
wget https://developer.nvidia.com/compute/cuda/9.2/Prod2/local_installers/cuda_9.2.148_396.37_linux
chmod +x ./cuda_9.2.148_396.37_linux
sudo ./cuda_9.2.148_396.37_linux

  • python3 in system sudo apt install python3-dev python3-venv

  • Open browser to localhost:10000 login and verify GPU is enabled.

    • Create project
    • Select GPU Ubuntu 18.04 base
    • Wait for build to complete
    • Launch jupyter (Start->Run->Open Jupyter)
    • In Jupyter, in a terminal, run the command nvidia-smi
Clone this wiki locally