lab-04_2_load_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 4-2: Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Author: Seungjae Lee (이승재)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "    We use elemental PyTorch to implement linear regression here. However, in most actual applications, abstractions such as <code>nn.Module</code> or <code>nn.Linear</code> are used.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Slicing 1D Array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "nums = [0, 1, 2, 3, 4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4]\n"
     ]
    }
   ],
   "source": [
    "print(nums)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "index 2에서 4 전까지 가져와라. (앞 포함, 뒤 비포함)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2, 3]\n"
     ]
    }
   ],
   "source": [
    "print(nums[2:4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "index 2부터 다 가져와라."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2, 3, 4]\n"
     ]
    }
   ],
   "source": [
    "print(nums[2:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "index 2 전까지 가져와라. (역시 뒤는 비포함)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1]\n"
     ]
    }
   ],
   "source": [
    "print(nums[:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "전부 가져와라"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4]\n"
     ]
    }
   ],
   "source": [
    "print(nums[:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "마지막 index 전까지 가져와라. (뒤는 비포함!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3]\n"
     ]
    }
   ],
   "source": [
    "print(nums[:-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "assign 도 가능!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "nums[2:4] = [8, 9]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 8, 9, 4]\n"
     ]
    }
   ],
   "source": [
    "print(nums)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Slicing 2D Array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "b = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1  2  3  4]\n",
      " [ 5  6  7  8]\n",
      " [ 9 10 11 12]]\n"
     ]
    }
   ],
   "source": [
    "print(b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2,  6, 10])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 9, 10, 11, 12])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 9, 10, 11, 12])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b[-1, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 9, 10, 11, 12])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b[-1, ...]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 2, 3, 4],\n",
       "       [5, 6, 7, 8]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b[0:2, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading Data from `.csv` file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "xy = np.loadtxt('data-01-test-score.csv', delimiter=',', dtype=np.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_data = xy[:, 0:-1]\n",
    "y_data = xy[:, [-1]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(25, 3)\n",
      "25\n",
      "[[ 73.  80.  75.]\n",
      " [ 93.  88.  93.]\n",
      " [ 89.  91.  90.]\n",
      " [ 96.  98. 100.]\n",
      " [ 73.  66.  70.]]\n"
     ]
    }
   ],
   "source": [
    "print(x_data.shape) # x_data shape\n",
    "print(len(x_data))  # x_data 길이\n",
    "print(x_data[:5])   # 첫 다섯 개"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(25, 1)\n",
      "25\n",
      "[[152.]\n",
      " [185.]\n",
      " [180.]\n",
      " [196.]\n",
      " [142.]]\n"
     ]
    }
   ],
   "source": [
    "print(y_data.shape) # y_data shape\n",
    "print(len(y_data))  # y_data 길이\n",
    "print(y_data[:5])   # 첫 다섯 개"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x7f528804e590>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# For reproducibility\n",
    "torch.manual_seed(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Low-level Implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch    0/20 Cost: 26811.960938\n",
      "Epoch    1/20 Cost: 9920.529297\n",
      "Epoch    2/20 Cost: 3675.299072\n",
      "Epoch    3/20 Cost: 1366.261108\n",
      "Epoch    4/20 Cost: 512.542236\n",
      "Epoch    5/20 Cost: 196.896484\n",
      "Epoch    6/20 Cost: 80.190910\n",
      "Epoch    7/20 Cost: 37.038647\n",
      "Epoch    8/20 Cost: 21.081354\n",
      "Epoch    9/20 Cost: 15.178741\n",
      "Epoch   10/20 Cost: 12.993667\n",
      "Epoch   11/20 Cost: 12.183028\n",
      "Epoch   12/20 Cost: 11.880545\n",
      "Epoch   13/20 Cost: 11.765955\n",
      "Epoch   14/20 Cost: 11.720857\n",
      "Epoch   15/20 Cost: 11.701424\n",
      "Epoch   16/20 Cost: 11.691505\n",
      "Epoch   17/20 Cost: 11.685121\n",
      "Epoch   18/20 Cost: 11.680006\n",
      "Epoch   19/20 Cost: 11.675381\n",
      "Epoch   20/20 Cost: 11.670943\n"
     ]
    }
   ],
   "source": [
    "# 데이터\n",
    "x_train = torch.FloatTensor(x_data)\n",
    "y_train = torch.FloatTensor(y_data)\n",
    "# 모델 초기화\n",
    "W = torch.zeros((3, 1), requires_grad=True)\n",
    "b = torch.zeros(1, requires_grad=True)\n",
    "# optimizer 설정\n",
    "optimizer = optim.SGD([W, b], lr=1e-5)\n",
    "\n",
    "nb_epochs = 20\n",
    "for epoch in range(nb_epochs + 1):\n",
    "    \n",
    "    # H(x) 계산\n",
    "    hypothesis = x_train.matmul(W) + b # or .mm or @\n",
    "\n",
    "    # cost 계산\n",
    "    cost = torch.mean((hypothesis - y_train) ** 2)\n",
    "\n",
    "    # cost로 H(x) 개선\n",
    "    optimizer.zero_grad()\n",
    "    cost.backward()\n",
    "    optimizer.step()\n",
    "\n",
    "    # 100번마다 로그 출력\n",
    "    print('Epoch {:4d}/{} Cost: {:.6f}'.format(\n",
    "        epoch, nb_epochs, cost.item()\n",
    "    ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## High-level Implementation with `nn.Module`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MultivariateLinearRegressionModel(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.linear = nn.Linear(3, 1)\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.linear(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch    0/20 Cost: 28388.000000\n",
      "Epoch    1/20 Cost: 10504.286133\n",
      "Epoch    2/20 Cost: 3892.179688\n",
      "Epoch    3/20 Cost: 1447.497314\n",
      "Epoch    4/20 Cost: 543.628357\n",
      "Epoch    5/20 Cost: 209.440048\n",
      "Epoch    6/20 Cost: 85.878799\n",
      "Epoch    7/20 Cost: 40.191814\n",
      "Epoch    8/20 Cost: 23.297268\n",
      "Epoch    9/20 Cost: 17.048059\n",
      "Epoch   10/20 Cost: 14.734761\n",
      "Epoch   11/20 Cost: 13.876624\n",
      "Epoch   12/20 Cost: 13.556563\n",
      "Epoch   13/20 Cost: 13.435411\n",
      "Epoch   14/20 Cost: 13.387821\n",
      "Epoch   15/20 Cost: 13.367404\n",
      "Epoch   16/20 Cost: 13.357059\n",
      "Epoch   17/20 Cost: 13.350419\n",
      "Epoch   18/20 Cost: 13.345175\n",
      "Epoch   19/20 Cost: 13.340427\n",
      "Epoch   20/20 Cost: 13.335882\n"
     ]
    }
   ],
   "source": [
    "# 데이터\n",
    "x_train = torch.FloatTensor(x_data)\n",
    "y_train = torch.FloatTensor(y_data)\n",
    "# 모델 초기화\n",
    "model = MultivariateLinearRegressionModel()\n",
    "# optimizer 설정\n",
    "optimizer = optim.SGD(model.parameters(), lr=1e-5)\n",
    "\n",
    "nb_epochs = 20\n",
    "for epoch in range(nb_epochs+1):\n",
    "    \n",
    "    # H(x) 계산\n",
    "    prediction = model(x_train)\n",
    "    \n",
    "    # cost 계산\n",
    "    cost = F.mse_loss(prediction, y_train)\n",
    "    \n",
    "    # cost로 H(x) 개선\n",
    "    optimizer.zero_grad()\n",
    "    cost.backward()\n",
    "    optimizer.step()\n",
    "    \n",
    "    # 20번마다 로그 출력\n",
    "    print('Epoch {:4d}/{} Cost: {:.6f}'.format(\n",
    "        epoch, nb_epochs, cost.item()\n",
    "    ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset and DataLoader"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "    pandas 기초지식이 필요할 것 같다\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "너무 데이터가 크면 `x_data`, `y_data` 를 전부 다 가져오지 말고, 필요한 배치만 가져올 수 밖에 없다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[PyTorch Data Loading and Processing tutorial](https://pytorch.org/tutorials/beginner/data_loading_tutorial.html#iterating-through-the-dataset)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}