GPU_Utilization_Test_in_pytorch.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "GPU_Utilization_Test_in_pytorch",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python2",
      "display_name": "Python 2"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "metadata": {
        "id": "Vc6kFsLu7oTs",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#Overview\n",
        "Pytorch is not only a tool for deep learning. It is a good tool for general-purpose matrix and tensor computations (like numpy) using parallel cores of GPU's. This enables fast linear algebra computations.\n",
        "\n",
        "In this tutorial, we'll see how Pytorch can be utilized for GPU-enabled matrix computations. This ipython notebook is designed to be compatible with Google Collaboratory (https://colab.research.google.com/) which gives you completely free access to GPU's. "
      ]
    },
    {
      "metadata": {
        "id": "LQgy3k5H9dqS",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#Preparation\n",
        "##Uploading or Using Notebook\n",
        "You need to signup and apply for access before you can start using Google Colab.\n",
        "Once you have access, you can either upload this notebook using File → Upload Notebook or simply enter the codes in the cells.\n",
        "##Activating GPU\n",
        "To enable GPU backend for your notebook, go to Edit → Notebook Settings and set Hardware accelerator to GPU.\n"
      ]
    },
    {
      "metadata": {
        "id": "m2_ZIRlA9yrq",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Installing Pytorch\n",
        "We are going to use pytorch for tensor operations in GPU. Install pytorch using the following command. Doing it once is sufficient for a session."
      ]
    },
    {
      "metadata": {
        "id": "IV-EO8xCVpUu",
        "colab_type": "code",
        "outputId": "21d29176-234b-4a21-fa6f-674302f57bb7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 139
        }
      },
      "cell_type": "code",
      "source": [
        "# http://pytorch.org/\n",
        "!pip install torch\n"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting torch\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/06/a7/6a173738dd6be014ebf9ba6f0b441d91b113b1506a98e10da4ff60994b54/torch-0.4.1-cp27-cp27mu-manylinux1_x86_64.whl (519.5MB)\n",
            "\u001b[K    100% |████████████████████████████████| 519.5MB 27kB/s \n",
            "tcmalloc: large alloc 1073750016 bytes == 0x559488f5c000 @  0x7f82a2e492a4 0x55943059ab68 0x55943068692d 0x5594305ae01a 0x5594305b2d72 0x5594305ab8ca 0x5594305b324e 0x5594305ab8ca 0x5594305b324e 0x5594305ab8ca 0x5594305b324e 0x5594305ab8ca 0x5594305b37d3 0x5594305ab8ca 0x5594305b324e 0x5594305ab8ca 0x5594305b324e 0x5594305b2d72 0x5594305b2d72 0x5594305ab8ca 0x5594305b37d3 0x5594305b2d72 0x5594305ab8ca 0x5594305b37d3 0x5594305ab8ca 0x5594305b37d3 0x5594305ab8ca 0x5594305b324e 0x5594305ab8ca 0x5594305ab1e9 0x5594305dbbdf\n",
            "\u001b[?25hInstalling collected packages: torch\n",
            "Successfully installed torch-0.4.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "5dezNaEF0mMt",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Variable Initialization\n",
        "We are going to initializa a big matrix in CPU and another equally sized matrix in GPU"
      ]
    },
    {
      "metadata": {
        "id": "eIXcGP3yV6lH",
        "colab_type": "code",
        "outputId": "1b33152e-7aba-41e6-8348-47bb2c487ddf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import time\n",
        "import numpy as np\n",
        "from torch.autograd import Variable\n",
        "\n",
        "x_cpu = np.random.rand(10000,10000)\n",
        "x_gpu = Variable(torch.from_numpy(x_cpu)).cuda(0)\n",
        "print 'GPU matrix size:',x_gpu.shape\n",
        "print 'CPU matrix size:',x_cpu.shape\n"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GPU matrix size: torch.Size([10000, 10000])\n",
            "CPU matrix size: (10000, 10000)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "9TztI4lW3zZv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# CPU vs. GPU Comparison for Matrix Multiplication"
      ]
    },
    {
      "metadata": {
        "id": "ir7rD5IIxPrE",
        "colab_type": "code",
        "outputId": "0189a842-0fa8-4876-b3c3-c5e7b336b4e0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "# Compute in CPU\n",
        "oldtime = time.time()\n",
        "z_cpu = x_cpu.dot(x_cpu.T)\n",
        "cputime = time.time()-oldtime\n",
        "print 'Matrix-Matrix product time in CPU:',cputime,'seconds'\n",
        "\n",
        "# Compute in GPU\n",
        "oldtime = time.time()\n",
        "z_cpu = torch.matmul(x_gpu,torch.t(x_gpu))\n",
        "gputime = time.time()-oldtime\n",
        "print 'Matrix-Matrix product time in GPU:',gputime,'seconds'\n",
        "print 'Speed Gain in GPU:',cputime/gputime*100,'%'"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Matrix-Matrix product time in CPU: 38.0653579235 seconds\n",
            "Matrix-Matrix product time in GPU: 0.00435185432434 seconds\n",
            "Speed Gain in GPU: 874692.834055 %\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "cQz-jPqS86Ta",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#CPU vs. GPU Comparison for Random Row-Column Multiplication"
      ]
    },
    {
      "metadata": {
        "id": "8_3QFk1_4abY",
        "colab_type": "code",
        "outputId": "c02de05d-e8b7-48b0-d533-c1072342cdee",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "from itertools import izip\n",
        "\n",
        "m,n = x_cpu.shape\n",
        "idx_a = np.random.choice(np.arange(m),50000)\n",
        "idx_b = np.random.choice(np.arange(m),50000)\n",
        "\n",
        "# Compute in CPU\n",
        "oldtime = time.time()\n",
        "for i,j in izip(idx_a,idx_b):\n",
        "  row = x_cpu[i,:][None,:]\n",
        "  col = x_cpu[:,j][:,None]\n",
        "  z_cpu = row.dot(col)\n",
        "print \"Random row-column multiplication in CPU:\",time.time()-oldtime,'seconds'\n",
        "\n",
        "# Compute in GPU\n",
        "oldtime = time.time()\n",
        "for i,j in izip(idx_a,idx_b):\n",
        "  row = x_gpu[i,:].unsqueeze(0)\n",
        "  col = x_gpu[:,j].unsqueeze(1)\n",
        "  z_gpu = torch.matmul(row,col)\n",
        "print \"Random row-column multiplication (unsqueeze) in GPU:\",time.time()-oldtime,'seconds'\n",
        "\n",
        "# Compute in GPU\n",
        "oldtime = time.time()\n",
        "for i,j in izip(idx_a,idx_b):\n",
        "  row = x_gpu[i,:].view(1,-1)\n",
        "  col = x_gpu[:,j].view(-1,1)\n",
        "  z_gpu = torch.matmul(row,col)\n",
        "print \"Random row-column multiplication (view) in GPU:\",time.time()-oldtime,'seconds'\n",
        "print \"View is a bit slower\"\n",
        "\n",
        "# Compute in GPU\n",
        "oldtime = time.time()\n",
        "for i,j in izip(idx_a,idx_b):\n",
        "  row = x_gpu[i,:].unsqueeze(0)\n",
        "  col = x_gpu[:,j].unsqueeze(1)\n",
        "  z_gpu = torch.mm(row,col)\n",
        "print \"Random row-column multiplication (mm) in GPU:\",time.time()-oldtime,'seconds'\n",
        "  \n"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Random row-column multiplication in CPU: 10.1162171364 seconds\n",
            "Random row-column multiplication (unsqueeze) in GPU: 7.38484311104 seconds\n",
            "Random row-column multiplication (view) in GPU: 7.8996989727 seconds\n",
            "View is a bit slower\n",
            "Random row-column multiplication (mm) in GPU: 7.1484181881 seconds\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}