week8_CNN_1_강미경.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AlexNet\n",
    "\n",
    "- [paper](http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf)\n",
    "\n",
    "- [imagenet data(2012)](http://image-net.org/challenges/LSVRC/2012/index#task)\n",
    "\n",
    "- [code](https://pytorch.org/docs/0.4.0/_modules/torchvision/models/alexnet.html)\n",
    "\n",
    "- Model architecture\n",
    "![model_architecture](https://cv-tricks.com/wp-content/uploads/2017/03/xalexnet_small-1.png.pagespeed.ic.u_mv-jhXMI.webp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Naive Version\n",
    "CONV_1 - POOL_1 - CONV_2 - POOL_2 - CONV_3 - CONV_4 - CONV_5 - POOL_3 - FC1 - FC2 - FC3 (->SOFTMAX)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### detailed\n",
    "CONV_1(ReLU) - POOL_1 - CONV_2(ReLU) - POOL_2 - CONV_3(ReLU) - CONV_4(ReLU) - CONV_5(ReLU) - POOL_3 -(Flatten) FC1(ReLU) - FC2(ReLU) - FC3(->SOFTMAX)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Q.1 fill in the '????' & 'TODO'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 1 is a Convolution Layer_1\n",
    "\n",
    "- **Input Image size**     224 x 224 x 3 -> 227 x 227 x 3\n",
    "\n",
    "- **Number of filters**   96\n",
    "\n",
    "- **Filter size** 11 x 11 x 3\n",
    "\n",
    "- **Stride** 4\n",
    "\n",
    "- **Layer 1 Output**  55 x 55 x 96 (because of stride 4)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "55.0"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(227-11)/4 + 1\n",
    "#OH_conv = (IH + 2P - FH) / S + 1 = 55"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "34944\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'last 96 for Bias'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## TODO ##\n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "\n",
    "Conv_1 = (11*11*3) * 96 + 96 \n",
    "print(Conv_1)\n",
    "'''last 96 for Bias'''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 2 is a Max Pooling_1 Followed by Convolution_1\n",
    "\n",
    "- **Input**  55 x 55 x 96\n",
    "\n",
    "- **Max pooling**  \n",
    "\n",
    "- **Pooling size**(overlapping) 3 x 3\n",
    "\n",
    "- **Stride** 2\n",
    "\n",
    "- **Layer 2 Output** 27 x 27 x 96\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "27.0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#OH_pool\n",
    "(55 - 3) / 2 + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "\n",
    "Max_pool_1 = 0\n",
    "# Pooling, stride, padding은 hyper parameter라서 사용자가 설정함."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 3 is a a Convolution Layer_2\n",
    "\n",
    "- **Input**  27 x 27 x 96\n",
    "\n",
    "- **Number of filters**  256\n",
    "\n",
    "- **Filter size**  5 x 5 x 96 \n",
    "\n",
    "- **Stride** 1\n",
    "\n",
    "- **padding** 2\n",
    "\n",
    "- **Layer 3 Output** 27 * 27 * 256\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "27.0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#OH_conv\n",
    "(27 + 4 - 5) / 1 + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "614656"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "\n",
    "Conv_2 = (5 * 5 * 96) * 256 + 256\n",
    "Conv_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 4 is a Max Pooling_2 Followed by Convolution_2\n",
    "\n",
    "- **Input**  27 * 27 * 256\n",
    "\n",
    "- **Max pooling**  \n",
    "\n",
    "- **Pooling size**(overlapping) 3 x 3\n",
    "\n",
    "- **Stride** 2\n",
    "\n",
    "- **Layer 4 Output** 13 * 13 * 256"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "\n",
    "Max_pool_2 = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 5 is a a Convolution Layer_3\n",
    "\n",
    "- **Input**  13 * 13 * 256\n",
    "\n",
    "- **Number of filters**  384\n",
    "\n",
    "- **Filter size**  3 x 3 x 256\n",
    "\n",
    "- **Stride** 1\n",
    "\n",
    "- **padding** 1\n",
    "\n",
    "- **Layer 5 Output** 13 * 13 * 384\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13.0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(13 + 2 - 3) / 1 + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "885120"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "Conv_3 = (3 * 3 * 256) * 384 + 384\n",
    "Conv_3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 6 is  a Convolution Layer_4\n",
    "\n",
    "- **Input**  13 * 13 * 384\n",
    "\n",
    "- **Number of filters**  384\n",
    "\n",
    "- **Filter size**  3 x 3 x 384\n",
    "\n",
    "- **Stride** 1\n",
    "\n",
    "- **padding** 1\n",
    "\n",
    "- **Layer 6 Output** 13 * 13 * 384"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13.0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(13 + 2 - 3) / 1 + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1327488"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "Conv_4 = (3 * 3 * 384) * 384 + 384\n",
    "Conv_4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 7 is a Convolution Layer_5\n",
    "\n",
    "- **Input**  13 * 13 * 384\n",
    "\n",
    "- **Number of filters**  256\n",
    "\n",
    "- **Filter size**  3 x 3 x 384\n",
    "\n",
    "- **Stride** 1\n",
    "\n",
    "- **padding** 1\n",
    "\n",
    "- **Layer 7 Output** 13 * 13 * 256"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13.0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(13 + 2 - 3) / 1 + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "884992"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "Conv_5 = (3 * 3 * 384) * 256 + 256\n",
    "Conv_5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 8 is a Max Pooling_3 Followed by Convolution_5\n",
    "\n",
    "- **Input**  13 x 13 x 256\n",
    "\n",
    "- **Max pooling**  \n",
    "\n",
    "- **Pooling size**(overlapping) 3 x 3\n",
    "\n",
    "- **Stride** 2\n",
    "\n",
    "- **Layer 8 Output** 6 * 6 * 256"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.0"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(13 - 3) / 2 + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "\n",
    "Max_pool_3 = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 9 is a Fully_Connected layer_1\n",
    "\n",
    "- **input** 6 x 6 x 256\n",
    "\n",
    "- **flatten** 9216\n",
    "\n",
    "- **output size** (N,flatten) x (flatten,4096)\n",
    "\n",
    "- **N** Number of input data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9216"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "6 * 6 * 256"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "37752832"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "FC1 = (6 * 6 * 256) * 4096 + 4096\n",
    "FC1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 10 is a Fully_Connected layer_2\n",
    "\n",
    "- **input** (N,4096)\n",
    "\n",
    "- **output size** (N,4096) x (4096,4096)\n",
    "\n",
    "- **N** Number of input data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16781312"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "\n",
    "FC2 = (4096 * 4096) + 4096\n",
    "FC2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer 11 is a Fully_Connected layer_3\n",
    "\n",
    "- **input** (N,4096)\n",
    "\n",
    "- **output size** (N,4096) x (4096,Num_classes)   \n",
    "( 1000개의 fully connected layer )\n",
    "\n",
    "- **N** Number of input data\n",
    "\n",
    "- **Num_classes** Number of labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4097000"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## TODO \n",
    "\n",
    "######################################################\n",
    "#  Calculate the number of parameters in this layer  #\n",
    "######################################################\n",
    "\n",
    "FC3 = (4096 * 1000) + 1000\n",
    "FC3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "62378344"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Total_prams = Conv_1 + Conv_2 + Conv_3 + Conv_4 + Conv_5 + FC1 + FC2 + FC3\n",
    "Total_prams"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}