tutorial_tf1_day2.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"tutorial_tf1_day2.ipynb","provenance":[{"file_id":"1-JPG2XB4iTc64fH8KHweKOIIeGITrp4m","timestamp":1596221319637}],"collapsed_sections":[],"authorship_tag":"ABX9TyOoJ05/6dTaiJqap6+7y0f4"},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"XirG9UCfsBDf","colab_type":"text"},"source":["이 노트북은 김성훈 교수님의 모두를 위한 딥러닝 강좌 스크립트들을 참고, 재구성하여 만들어졌습니다.\n","\n","\n","(https://github.com/hunkim/DeepLearningZeroToAll)"]},{"cell_type":"markdown","metadata":{"id":"Ite50jjkO7g1","colab_type":"text"},"source":["Colab의 기본 텐서플로우 버전은 2.X이므로 1.X 버전으로 다운그레이드 해줍니다."]},{"cell_type":"code","metadata":{"id":"HJsKclFQO70-","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":1000},"executionInfo":{"status":"ok","timestamp":1596266431094,"user_tz":-540,"elapsed":116933,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}},"outputId":"2efdc11d-6945-4ca1-96ca-13a6708ff489"},"source":["!pip uninstall tensorflow\n","!pip install tensorflow==1.15"],"execution_count":1,"outputs":[{"output_type":"stream","text":["Uninstalling tensorflow-2.2.0:\n","  Would remove:\n","    /usr/local/bin/estimator_ckpt_converter\n","    /usr/local/bin/saved_model_cli\n","    /usr/local/bin/tensorboard\n","    /usr/local/bin/tf_upgrade_v2\n","    /usr/local/bin/tflite_convert\n","    /usr/local/bin/toco\n","    /usr/local/bin/toco_from_protos\n","    /usr/local/lib/python3.6/dist-packages/tensorflow-2.2.0.dist-info/*\n","    /usr/local/lib/python3.6/dist-packages/tensorflow/*\n","Proceed (y/n)? y\n","y\n","  Successfully uninstalled tensorflow-2.2.0\n","Collecting tensorflow==1.15\n","\u001b[?25l  Downloading https://files.pythonhosted.org/packages/3f/98/5a99af92fb911d7a88a0005ad55005f35b4c1ba8d75fba02df726cd936e6/tensorflow-1.15.0-cp36-cp36m-manylinux2010_x86_64.whl (412.3MB)\n","\u001b[K     |████████████████████████████████| 412.3MB 26kB/s \n","\u001b[?25hCollecting gast==0.2.2\n","  Downloading https://files.pythonhosted.org/packages/4e/35/11749bf99b2d4e3cceb4d55ca22590b0d7c2c62b9de38ac4a4a7f4687421/gast-0.2.2.tar.gz\n","Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (1.0.8)\n","Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (0.2.0)\n","Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (0.9.0)\n","Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (1.15.0)\n","Collecting tensorboard<1.16.0,>=1.15.0\n","\u001b[?25l  Downloading https://files.pythonhosted.org/packages/1e/e9/d3d747a97f7188f48aa5eda486907f3b345cd409f0a0850468ba867db246/tensorboard-1.15.0-py3-none-any.whl (3.8MB)\n","\u001b[K     |████████████████████████████████| 3.8MB 36.2MB/s \n","\u001b[?25hRequirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (1.12.1)\n","Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (0.34.2)\n","Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (1.1.0)\n","Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (3.3.0)\n","Requirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (1.18.5)\n","Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (3.12.2)\n","Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (1.30.0)\n","Collecting tensorflow-estimator==1.15.1\n","\u001b[?25l  Downloading https://files.pythonhosted.org/packages/de/62/2ee9cd74c9fa2fa450877847ba560b260f5d0fb70ee0595203082dafcc9d/tensorflow_estimator-1.15.1-py2.py3-none-any.whl (503kB)\n","\u001b[K     |████████████████████████████████| 512kB 41.8MB/s \n","\u001b[?25hRequirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (1.1.2)\n","Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15) (0.8.1)\n","Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow==1.15) (2.10.0)\n","Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15) (49.2.0)\n","Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15) (1.0.1)\n","Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15) (3.2.2)\n","Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15) (1.7.0)\n","Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15) (3.1.0)\n","Building wheels for collected packages: gast\n","  Building wheel for gast (setup.py) ... \u001b[?25l\u001b[?25hdone\n","  Created wheel for gast: filename=gast-0.2.2-cp36-none-any.whl size=7540 sha256=04e747301a59f46916fee516916d3a190b81c8bc9eb53a9c78a4ef5a16dfb77c\n","  Stored in directory: /root/.cache/pip/wheels/5c/2e/7e/a1d4d4fcebe6c381f378ce7743a3ced3699feb89bcfbdadadd\n","Successfully built gast\n","\u001b[31mERROR: tensorflow-probability 0.10.0 has requirement gast>=0.3.2, but you'll have gast 0.2.2 which is incompatible.\u001b[0m\n","Installing collected packages: gast, tensorboard, tensorflow-estimator, tensorflow\n","  Found existing installation: gast 0.3.3\n","    Uninstalling gast-0.3.3:\n","      Successfully uninstalled gast-0.3.3\n","  Found existing installation: tensorboard 2.2.2\n","    Uninstalling tensorboard-2.2.2:\n","      Successfully uninstalled tensorboard-2.2.2\n","  Found existing installation: tensorflow-estimator 2.2.0\n","    Uninstalling tensorflow-estimator-2.2.0:\n","      Successfully uninstalled tensorflow-estimator-2.2.0\n","Successfully installed gast-0.2.2 tensorboard-1.15.0 tensorflow-1.15.0 tensorflow-estimator-1.15.1\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"fSaHRmp7R9Rz","colab_type":"text"},"source":["필요한 모듈들을 import합니다."]},{"cell_type":"code","metadata":{"id":"REP59ZCGR6IJ","colab_type":"code","colab":{},"executionInfo":{"status":"ok","timestamp":1596266433819,"user_tz":-540,"elapsed":119639,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}}},"source":["import tensorflow as tf\n","# 텐서플로우 패키지\n","import random\n","# 재현성을 위해 난수 Seed를 고정하기 위해 불러오는 패키지입니다.\n","from tensorflow.examples.tutorials.mnist import input_data\n","# 영상인식에서 대표적인 Dataset인 MNIST를 불러오고 사용하기 위한 패키지입니다."],"execution_count":2,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"oDWZ33PrTnyK","colab_type":"text"},"source":["하이퍼 패러미터와 난수 Seed를 설정합니다."]},{"cell_type":"code","metadata":{"id":"SuU-__3LTmcV","colab_type":"code","colab":{},"executionInfo":{"status":"ok","timestamp":1596266433828,"user_tz":-540,"elapsed":119642,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}}},"source":["tf.set_random_seed(777)\n","# 재현성을 위해 난수를 고정합니다.\n","\n","# Hyperparameters\n","# 하이퍼 패러미터란 학습으로 인해 자동으로 최적화되는 값이 아닌,\n","# 사람이 Huristic하게 최적의 값을 찾아나갈 수 밖에 없는 Parameter들입니다.\n","learning_rate = 0.001\n","# 한번에 얼만큼 업데이트 할 것인가에 대한 Hyperparameter입니다.\n","training_epochs = 15\n","# 에폭은 전체 데이터셋의 크기 만큼의 한 Loop를 일컫는 단위입니다. 총 몇 에폭 돌지 설정합니다.\n","batch_size = 100\n","# 한번에 Feed하는 이미지의 갯수를 결정합니다.\n","# 초심자 단계에서 당장 신경쓸 필요는 없지만 배치사이즈는 학습속도 뿐만 아니라 최종적인 성능에도 상당한 영향을 미친다고 알려져 있습니다."],"execution_count":3,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"5KPTp1g5U305","colab_type":"text"},"source":["MNIST 데이터셋은 텐서플로우에서 자체적으로 제공하므로 텐서플로우 API를 이용해 데이터를 불러옵니다."]},{"cell_type":"code","metadata":{"id":"CBi6XRz9U0Q6","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":559},"executionInfo":{"status":"ok","timestamp":1596266434973,"user_tz":-540,"elapsed":120774,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}},"outputId":"7f46e808-4ec8-4fc9-b798-38c553ce643f"},"source":["mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)\n","# 텐서플로우에서 지원하는 API를 사용합니다.\n","#\"MNIST_data/\"는 불러올 데이터셋의 디렉토리\n","# one_hot은 One-Hot 인코딩 형식으로 불러올지 여부를 결정합니다."],"execution_count":4,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:From <ipython-input-4-87b6b4fc864d>:1: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n","WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Please write your own downloading logic.\n","WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/learn/python/learn/datasets/base.py:252: _internal_retry.<locals>.wrap.<locals>.wrapped_fn (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Please use urllib or similar directly.\n","Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.\n","WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Please use tf.data to implement this functionality.\n","Extracting MNIST_data/train-images-idx3-ubyte.gz\n","Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.\n","WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Please use tf.data to implement this functionality.\n","Extracting MNIST_data/train-labels-idx1-ubyte.gz\n","WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Please use tf.one_hot on tensors.\n","Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.\n","Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n","Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.\n","Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n","WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/learn/python/learn/datasets/mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"hg7YnPS9V17W","colab_type":"text"},"source":["모델을 구성합니다. 딥러닝 소스코드의 심장에 해당하는 부분입니다.\n","\n","\n","텐서플로우 1.X는 정적그래프로 모델의 패러미터와 연산 관계를 직접 정의합니다."]},{"cell_type":"code","metadata":{"id":"Up-LDpYFJ8AF","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":91},"executionInfo":{"status":"ok","timestamp":1596266435339,"user_tz":-540,"elapsed":121126,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}},"outputId":"21b949d5-95da-415f-a808-fcf7977f07bf"},"source":["X = tf.placeholder(tf.float32, [None, 784])\n","Y = tf.placeholder(tf.float32, [None, 10])\n","# Placeholder란 데이터가 들어갈 미리 만들어 놓은 틀이라고 생각하면 됩니다.\n","# 정적 그래프 방식을 사용하는 텐서플로우 1.X 버전에서는 이처럼 미리 입력의 형태를 지정해 놓아야 합니다.\n","\n","keep_prob = tf.placeholder(tf.float32)\n","# Dropout을 할 때 얼만큼 데이터를 보존할 지 결정하는 Parameter입니다.\n","# (1 - keep_prob)만큼 학습시에 데이터를 탈락시킵니다.\n","# 이후 Runtime중에 학습시엔 0.7, 평가시엔 1이 입력으로 들어가도록 할 것 입니다.\n","\n","# 아래 부터 실제로 데이터가 흘러다닐 신경망을 정의하게 됩니다.\n","W1 = tf.get_variable(\"W1\", shape=[784, 512], initializer=tf.random_normal_initializer(stddev=0.05))\n","# Weight를 \"W1\"라는 이름으로, [784, 512]의 모양으로, 표준편차 0.05의 정규분포에서 초기화합니다.\n","b1 = tf.get_variable(\"b1\", shape=[512], initializer=tf.zeros_initializer())\n","# Bias를 \"b1\"라는 이름으로, [512]의 모양으로, 0으로 초기화합니다.\n","L1 = tf.nn.relu(tf.matmul(X, W1) + b1)\n","# 내적연산과 활성화함수를 정의함으로서 실질적으로 입력과 Weight, bias간의 연산이 일어나도록 하는 부분입니다.\n","L1 = tf.nn.dropout(L1, keep_prob=keep_prob)\n","# Regularization을 위해 Dropout을 정의합니다.\n","\n","# 위와 같은 방식으로 나머지 부분의 신경망을 정의합니다.\n","W2 = tf.get_variable(\"W2\", shape=[512, 512],\n","                     initializer=tf.random_normal_initializer(stddev=0.05))\n","b2 = tf.get_variable(\"b2\", shape=[512],\n","                     initializer=tf.zeros_initializer())\n","L2 = tf.nn.relu(tf.matmul(L1, W2) + b2)\n","L2 = tf.nn.dropout(L2, keep_prob=keep_prob)\n","\n","W3 = tf.get_variable(\"W3\", shape=[512, 512],\n","                     initializer=tf.random_normal_initializer(stddev=0.05))\n","b3 = tf.get_variable(\"b3\", shape=[512],\n","                     initializer=tf.zeros_initializer())\n","L3 = tf.nn.relu(tf.matmul(L2, W3) + b3)\n","L3 = tf.nn.dropout(L3, keep_prob=keep_prob)\n","\n","W4 = tf.get_variable(\"W4\", shape=[512, 512],\n","                     initializer=tf.random_normal_initializer(stddev=0.05))\n","b4 = tf.get_variable(\"b4\", shape=[512],\n","                     initializer=tf.zeros_initializer())\n","L4 = tf.nn.relu(tf.matmul(L3, W4) + b4)\n","L4 = tf.nn.dropout(L4, keep_prob=keep_prob)\n","\n","W5 = tf.get_variable(\"W5\", shape=[512, 10],\n","                     initializer=tf.random_normal_initializer(stddev=0.05))\n","b5 = tf.get_variable(\"b5\", shape=[10],\n","                     initializer=tf.zeros_initializer())\n","hypothesis = tf.matmul(L4, W5) + b5\n","# hypothesis는 말 그대로 가설입니다. 내 모델을 통해 예측한 예측값(가설)이라고 생각하면 됩니다.\n","# Softmax를 거치기 전 Logit도 가설이라고 할 수 있고 Softmax를 거친 확률 값도 가설이라고 할 수 있습니다.\n","\n","cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=hypothesis, labels=Y))\n","# == cost = K.mean(K.sum(-1 * batch_ys * (hypothesis - K.log(K.tile(K.expand_dims(K.sum(K.exp(hypothesis), axis=1), axis=1), [1, 10]))), axis=1))\n","# Loss Function을 계산합니다. \n","# Softmax 함수는 tf.nn.softmax_cross_entropy_with_logits안에 내부적으로 포함되어 있으므로 생략해도 됩니다.\n","\n","\n","train_op = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)\n","# 최적화엔 Adam Optimizer를 사용합니다.\n","# learning_rate=learning_rate로 Hyperparameter인 Learning Rate를 입력해줍니다."],"execution_count":5,"outputs":[{"output_type":"stream","text":["WARNING:tensorflow:From <ipython-input-5-d9a0831db085>:18: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"frvQEAXZlnS0","colab_type":"text"},"source":["텐서플로우는 단일 GPU이고 텐서플로우 GPU버전이 설치되어 있을 경우, 별도의 코드 추가 없이도 자동으로 GPU 가속이 적용됩니다."]},{"cell_type":"code","metadata":{"id":"KNt0ofQxlmAf","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":503},"executionInfo":{"status":"ok","timestamp":1596266436986,"user_tz":-540,"elapsed":122757,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}},"outputId":"aaa1eb30-0fbe-41ec-e386-4a5694be82e7"},"source":["from tensorflow.python.client import device_lib\n","device_lib.list_local_devices()\n","# GPU가 사용가능한지만 확인해줍니다."],"execution_count":6,"outputs":[{"output_type":"execute_result","data":{"text/plain":["[name: \"/device:CPU:0\"\n"," device_type: \"CPU\"\n"," memory_limit: 268435456\n"," locality {\n"," }\n"," incarnation: 8977053806066681878, name: \"/device:XLA_CPU:0\"\n"," device_type: \"XLA_CPU\"\n"," memory_limit: 17179869184\n"," locality {\n"," }\n"," incarnation: 11732163942161247875\n"," physical_device_desc: \"device: XLA_CPU device\", name: \"/device:XLA_GPU:0\"\n"," device_type: \"XLA_GPU\"\n"," memory_limit: 17179869184\n"," locality {\n"," }\n"," incarnation: 15906247354884212287\n"," physical_device_desc: \"device: XLA_GPU device\", name: \"/device:GPU:0\"\n"," device_type: \"GPU\"\n"," memory_limit: 11330115994\n"," locality {\n","   bus_id: 1\n","   links {\n","   }\n"," }\n"," incarnation: 11282871586231109822\n"," physical_device_desc: \"device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7\"]"]},"metadata":{"tags":[]},"execution_count":6}]},{"cell_type":"markdown","metadata":{"id":"siy-GqIAWUa9","colab_type":"text"},"source":["텐서플로우는 정적으로 정의된 그래프에 Runtime중 실제 데이터를 흘려보내기 위해 Session이라는 객체를 사용합니다."]},{"cell_type":"code","metadata":{"id":"y6rMpVH_WjY_","colab_type":"code","colab":{},"executionInfo":{"status":"ok","timestamp":1596266437497,"user_tz":-540,"elapsed":123257,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}}},"source":["sess = tf.Session()\n","# 세션을 만듭니다.\n","sess.run(tf.global_variables_initializer())\n","# 모든 학습가능한 Parameter들을 초기화합니다.\n","# 앞서 tf.get_variable에 \n","# initializer=tf.random_normal_initializer()로 입력해놓은 부분은\n","# 실제로 Parameter가 초기화되는 부분이 아닌, Parameter를 어떤 방식으로 초기화 할지 정의하는 부분입니다."],"execution_count":7,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"uQQ7BesUf50X","colab_type":"text"},"source":["트레이닝 루프를 만들어 줍니다. 실질적으로 학습이 진행되는 부분입니다."]},{"cell_type":"code","metadata":{"id":"8udHl__vf8al","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":305},"executionInfo":{"status":"ok","timestamp":1596266479953,"user_tz":-540,"elapsed":165706,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}},"outputId":"c7546f30-88b3-48b6-fe21-b801fdab8de0"},"source":["for epoch in range(training_epochs):\n","    avg_cost = 0\n","    total_batch = int(mnist.train.num_examples / batch_size)\n","    # '전체 데이터셋 길이'를 '한번에 Feed하는 배치사이즈'로 나누면\n","    # 한 에폭을 돌기위해 반복해야 할 루프횟수(total_batch)가 됩니다.\n","    \n","    for i in range(total_batch):\n","      batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n","      # 학습용 데이터셋에서 Input과 Label을 불러옵니다.\n","      # next_batch 함수는 데이터를 batch_size만큼 차례대로 잘라서 불러와 줍니다.\n","      \n","      c, _ = sess.run([cost, train_op], feed_dict={X: batch_xs, Y: batch_ys, keep_prob: 0.7})\n","      # 드디어 Session을 이용해 실제로 데이터를 흘려보내주는 부분입니다.\n","      # 앞서 만들어둔 Placeholder라는 틀 X, Y, keep_prob에\n","      # 실제 데이터인 batch_xs, batch_ys, 0.7이 Feed되는 모습에 주목하십시오.\n","      #\n","      # 'batch_xs, batch_ys, 0.7'과 'X, Y, keep_prob'이 '실제 데이터'와 '정적 그래프'의 관계이듯이,\n","      # 'c, _' 쌍과 'cost, train_op' 쌍은 '실제 데이터'와 '정적 그래프'의 관계입니다.\n","      # Session을 통해 정적 그래프로부터 실제 데이터를 얻는데,\n","      # 이때 train_op는 단순히 실행만 필요하고 반환은 필요없는 Operation일 뿐이므로\n","      # 앞으로 쓰지 않을 것이라는 뜻으로 '_'로 반환받습니다.\n","\n","      avg_cost += c / total_batch\n","      # '실제 정답값과의 차이'가 얼마나 줄어들었는지 실시간으로 확인하기 위해 따로 기록해둡니다.\n","\n","    print(\"[Epoch: %7d] cost = %5.5f\"%(epoch + 1, avg_cost.item()))\n","    # '실제 정답값과의 차이'가 얼마나 줄어들었는지 실시간으로 확인합니다.\n","\n","print('Learning Finished!')"],"execution_count":8,"outputs":[{"output_type":"stream","text":["[Epoch:       1] cost = 0.33251\n","[Epoch:       2] cost = 0.15106\n","[Epoch:       3] cost = 0.11696\n","[Epoch:       4] cost = 0.09561\n","[Epoch:       5] cost = 0.08740\n","[Epoch:       6] cost = 0.07577\n","[Epoch:       7] cost = 0.07124\n","[Epoch:       8] cost = 0.06564\n","[Epoch:       9] cost = 0.05733\n","[Epoch:      10] cost = 0.05303\n","[Epoch:      11] cost = 0.05147\n","[Epoch:      12] cost = 0.04537\n","[Epoch:      13] cost = 0.04636\n","[Epoch:      14] cost = 0.05052\n","[Epoch:      15] cost = 0.04147\n","Learning Finished!\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"CUmCrSldiElv","colab_type":"text"},"source":["학습이 완료된 모델을 이용해 정확도(Accuracy)를 구해봅니다."]},{"cell_type":"code","metadata":{"id":"88VZ1V78MTkx","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":35},"executionInfo":{"status":"ok","timestamp":1596266479956,"user_tz":-540,"elapsed":165700,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}},"outputId":"42ffa3a7-9a2e-4f66-ecf0-eb27203b80f2"},"source":["# 역시 정확도를 계산하는 정적 그래프를 먼저 정의합니다.\n","correct_prediction = tf.equal(tf.argmax(hypothesis, 1), tf.argmax(Y, 1))\n","# 먼저 맞게 예측되었는지 나타내는 bool 그래프를 정의합니다.\n","# tf.argmax(hypothesis, 1) : 확률이 가장 높은 값의 Index를 취합니다.\n","# tf.argmax(Y, 1)의 경우 One-Hot 인코딩이므로 정답인(확률이 1인) 값의 Index를 취할 것 입니다. \n","# tf.equal로 정답값과 예측값을 비교하여 bool 형태로 반환합니다.\n","\n","accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n","# bool 데이터 타입은 reduce_mean 연산에 사용할 수 없으므로 float 데이터 타입으로 형변환 한 뒤,\n","# reduce_mean 연산을 해줍니다. 이렇게 하면 맞게 예측한 것들의 비율을 구하는 정적 그래프를 정의할 수 있습니다.\n","\n","print('Accuracy:', sess.run(accuracy, feed_dict={\n","      X: mnist.test.images, Y: mnist.test.labels, keep_prob: 1}))\n","# 정적 그래프에 실제 평가용 데이터를 Feed해줍니다.\n","#  mnist.test.images, mnist.test.labels는 평가용 Input과 Label이고\n","# 평가시엔 Dropout을 적용하지 않으므로 keep_prob에 1을 Feed 해줍니다."],"execution_count":9,"outputs":[{"output_type":"stream","text":["Accuracy: 0.9826\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"LS7N5uFMkFgO","colab_type":"text"},"source":["모델이 잘 작동하는지 예시 하나를 직접 눈으로 확인합니다"]},{"cell_type":"code","metadata":{"id":"_lWb0HPBj_aD","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":71},"executionInfo":{"status":"ok","timestamp":1596266479957,"user_tz":-540,"elapsed":165692,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}},"outputId":"77710f98-4a70-4e2a-ce82-aaf1a0a8d164"},"source":["random_index = random.randint(0, mnist.test.num_examples - 1)\n","# 랜덤한 값 하나를 고릅니다.\n","\n","\n","print(\"Index: \", random_index)\n","print(\"Label: \", sess.run(tf.argmax(mnist.test.labels[random_index:random_index + 1], 1))[0])\n","print(\"Prediction: \", sess.run(\n","    tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[random_index:random_index + 1], keep_prob: 1})[0])\n","# 예측값과 정답을 비교해봅니다."],"execution_count":10,"outputs":[{"output_type":"stream","text":["Index:  6956\n","Label:  9\n","Prediction:  9\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"kN_BA-D1RzaF","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":283},"executionInfo":{"status":"ok","timestamp":1596266495107,"user_tz":-540,"elapsed":813,"user":{"displayName":"정현진","photoUrl":"","userId":"02319906487058652558"}},"outputId":"996ce80d-4cf9-406f-e42f-d24617a293aa"},"source":["import matplotlib.pyplot as plt\n","import numpy as np\n","\n","plt.imshow(mnist.test.images[random_index:random_index + 1].reshape(28, 28))\n","# 예측에 사용했던 숫자의 이미지를 시각화 해봅니다."],"execution_count":12,"outputs":[{"output_type":"execute_result","data":{"text/plain":["<matplotlib.image.AxesImage at 0x7ff24ad0eb00>"]},"metadata":{"tags":[]},"execution_count":12},{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAOxklEQVR4nO3df7BU9XnH8c/DbwVJuVgIFeIPIK0kNiTeQY22Q3Q06nSKVkVpxyENE8yMtHGadMLYdsJ0mtSJjYnTWmeulUJSq01jVGaKrQRNiaOlXpDySxN+BBQGuCqmgkbgcp/+cQ/2gvd897rn7J6F5/2aubO759mz55nVD2d3v+ecr7m7AJz6BlXdAIDmIOxAEIQdCIKwA0EQdiCIIc3c2DAb7iM0spmbBEJ5V2/rsB+y/mqFwm5mV0u6V9JgSf/g7nelnj9CI3WRXVFkkwASVvvK3FrdH+PNbLCk+yRdI2mapDlmNq3e1wPQWEW+s8+QtNXdt7v7YUmPSJpVTlsAylYk7GdJerXP413ZsuOY2Xwz6zSzziM6VGBzAIpo+K/x7t7h7u3u3j5Uwxu9OQA5ioR9t6RJfR5PzJYBaEFFwv6CpKlmdq6ZDZN0i6Rl5bQFoGx1D725e7eZLZD0H+odelvs7ptK6wxAqQqNs7v7cknLS+oFQANxuCwQBGEHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBEHYgiEJTNpvZDkkHJB2V1O3u7WU0BaB8hcKe+Yy7v17C6wBoID7GA0EUDbtLesrM1pjZ/P6eYGbzzazTzDqP6FDBzQGoV9GP8Ze5+24zGydphZm97O6r+j7B3TskdUjSaGvzgtsDUKdCe3Z3353ddkl6TNKMMpoCUL66w25mI83sjGP3JV0laWNZjQEoV5GP8eMlPWZmx17nn93930vpCkDp6g67u2+X9IkSewHQQAy9AUEQdiAIwg4EQdiBIAg7EEQZJ8KgYoPPn5pbe+Xrwwq9tln6oMenLuxI1v9o53W5tevHvZhc9xtLbk7WP3LvumS95513kvVo2LMDQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBDm3ryLx4y2Nr/Irmja9k4WQz48Plnf8fnJyfo3P784t3bVaW/X1dMxg2TJeo8a9/9PrW1f8PcLkvVJX3+uzHZOCqt9pd7y/f2+cezZgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIzmdvAbtnp8fR19/+d8l6aqx7V/cvk+te/uSfJOtT/ulwsr79xhHJ+oh9+fuTG2/5z+S6XxyzOln/zh8+kKwv/738ix+//MXzk+v6CxuS9ZMRe3YgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIJx9iYYdMYZyfo5N2wr9Pp/3nVhbu2Zey9JrvvRJc8X2vaUZ+tf97/+emiy/vynb0/W2+9LX3f+7g/nj9NPX/jryXUn3pAsn5Rq7tnNbLGZdZnZxj7L2sxshZltyW7HNLZNAEUN5GP8EklXn7BsoaSV7j5V0srsMYAWVjPs7r5K0v4TFs+StDS7v1RS/hw/AFpCvd/Zx7v7nuz+Xkm5F1Ezs/mS5kvSCJ1e5+YAFFX413jvvWJl7pkY7t7h7u3u3j5Uw4tuDkCd6g37PjObIEnZbVd5LQFohHrDvkzS3Oz+XElPlNMOgEap+Z3dzB6WNFPSmWa2S9LXJN0l6ftmNk/STkmzG9nkyW77Vz+erG+ecl+yvuHwkWR94zX5150fs7fYOHqVtt10WrL+5Lj0/OxKXHf+na6RyTX90unJ+qDDR9Prt+D58DXD7u5zckrM9gCcRDhcFgiCsANBEHYgCMIOBEHYgSA4xbUJRv7miacWHK/WtMeLXvndZL17774P3FOz2PD8oybf+P1PJdd99oa7k/UepYfmft79bm7t/L/9RXLdX1zQlqy3/+maZP2n7clyJdizA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQjLM3gXv+qZYDMXpYetrld4fk/2f07u5C265lyNmTkvXXZk7MrT33V+mpqAfVuIxZreMTPvv0H+fWPro5PU6uCy5OlscOfTu9vobVqDcfe3YgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIJx9iYYvKzGJLc1zn3+x4/8OFmfu+ry3NraJ6cl1x32v+ltn3fjlmS949xHkvUPDRqR3kABN287cb7R453/lZ/n1tIXgpZG7co/F16S3jiSvhS1lL78dxXYswNBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEOaePie4TKOtzS8yJn890a5HP5asb7rkoWT9qPeU2U7LuGnbZ5P1Xy4Ym6z3rH+5zHaOc/Cmi5L1Uf+6umHbTlntK/WW7+/3Ago19+xmttjMusxsY59li8xst5mty/6uLbNhAOUbyMf4JZL6O1Tp2+4+PftbXm5bAMpWM+zuvkpSev4iAC2vyA90C8xsffYxP/fgbzObb2adZtZ5RIcKbA5AEfWG/X5JkyVNl7RH0rfynujuHe7e7u7tQ5U/yR+Axqor7O6+z92PunuPpAckzSi3LQBlqyvsZjahz8PrJW3Mey6A1lDzfHYze1jSTElnmtkuSV+TNNPMpktySTsk3dbAHk95E+8enKw/tDQ9njx7VFeZ7Rzn3975ULL+mdNeS9ZPt/qvn/7K96Yk62PXP1/3axdV1Th6ETXD7u5z+ln8YAN6AdBAHC4LBEHYgSAIOxAEYQeCIOxAEJziehIYPLYtWT/4W/lDVLtnpv89P+8H6Usmvz49PW3yjxfmHjwpKT309ukX+xvo+X9tv/OzZB3vV+gUVwCnBsIOBEHYgSAIOxAEYQeCIOxAEIQdCIIpm08CR99IXwLwtMf/O7c25fFi2+78l7XJeo/Sp7BuOtydWxt328Hkuvlroh7s2YEgCDsQBGEHgiDsQBCEHQiCsANBEHYgCMbZT3GDTk+fj/7mD36txiukx9kHqd9Tp9/zhb+8I7fWtru6S0FHxJ4dCIKwA0EQdiAIwg4EQdiBIAg7EARhB4JgnP0U5x+bnKz/5BNLarxCehz9njenJuvjfvRqbo3z1Zur5p7dzCaZ2TNmttnMNpnZl7LlbWa2wsy2ZLdjGt8ugHoN5GN8t6Qvu/s0SRdLut3MpklaKGmlu0+VtDJ7DKBF1Qy7u+9x97XZ/QOSXpJ0lqRZkpZmT1sq6bpGNQmguA/0nd3MzpH0SUmrJY139z1Zaa+k8TnrzJc0X5JGKH2cNoDGGfCv8WY2StKjku5w97f61rx3dsh+Z4h09w53b3f39qEaXqhZAPUbUNjNbKh6g/6Qu/8wW7zPzCZk9QmSuhrTIoAy1PwYb2Ym6UFJL7n7PX1KyyTNlXRXdvtEQzpETUPOnpRbu3LJTxq67ZWfuyRZ91c3NnT7GLiBfGe/VNKtkjaY2bps2Z3qDfn3zWyepJ2SZjemRQBlqBl2d39W+UdWXFFuOwAahcNlgSAIOxAEYQeCIOxAEIQdCIJTXE8B2+ZNzK09/ivF5mz+i67pybpt3Jqs93tYJSrBnh0IgrADQRB2IAjCDgRB2IEgCDsQBGEHgmCc/SRw4OaLk/VN8+5LVNOXgq7lxbnTkvWed18u9PpoHvbsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAE4+yngJ7EWeMHew4l173s/q8k6xPXP1dXT2g97NmBIAg7EARhB4Ig7EAQhB0IgrADQRB2IIiBzM8+SdJ3JY1X72XAO9z9XjNbJOkLkl7Lnnqnuy9vVKORjd56IFm/dceVubX1T/5Gct1J32AcPYqBHFTTLenL7r7WzM6QtMbMVmS1b7v73zSuPQBlGcj87Hsk7cnuHzCzlySd1ejGAJTrA31nN7NzJH1S0ups0QIzW29mi81sTM46882s08w6jyh96CaAxhlw2M1slKRHJd3h7m9Jul/SZEnT1bvn/1Z/67l7h7u3u3v7UA0voWUA9RhQ2M1sqHqD/pC7/1CS3H2fux919x5JD0ia0bg2ARRVM+xmZpIelPSSu9/TZ/mEPk+7XtLG8tsDUJaB/Bp/qaRbJW0ws3XZsjslzTGz6eodjtsh6baGdAj5mk3J+puX5tcmiaG1enRffmGyvnNeT7I++Q9eLLOdUgzk1/hn1f/FxxlTB04iHEEHBEHYgSAIOxAEYQeCIOxAEIQdCIJLSQP9GPL0mmR98tNNaqRE7NmBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAhzz5/ut/SNmb0maWefRWdKer1pDXwwrdpbq/Yl0Vu9yuztbHf/1f4KTQ37+zZu1unu7ZU1kNCqvbVqXxK91atZvfExHgiCsANBVB32joq3n9KqvbVqXxK91aspvVX6nR1A81S9ZwfQJIQdCKKSsJvZ1Wb2UzPbamYLq+ghj5ntMLMNZrbOzDor7mWxmXWZ2cY+y9rMbIWZbclu+51jr6LeFpnZ7uy9W2dm11bU2yQze8bMNpvZJjP7Ura80vcu0VdT3remf2c3s8GSfibpSkm7JL0gaY67b25qIznMbIekdnev/AAMM/ttSQclfdfdP54t+6ak/e5+V/YP5Rh3/2qL9LZI0sGqp/HOZiua0HeacUnXSfqcKnzvEn3NVhPetyr27DMkbXX37e5+WNIjkmZV0EfLc/dVkvafsHiWpKXZ/aXq/Z+l6XJ6awnuvsfd12b3D0g6Ns14pe9doq+mqCLsZ0l6tc/jXWqt+d5d0lNmtsbM5lfdTD/Gu/ue7P5eSeOrbKYfNafxbqYTphlvmfeununPi+IHuve7zN0/JekaSbdnH1dbkvd+B2ulsdMBTePdLP1MM/6eKt+7eqc/L6qKsO+WNKnP44nZspbg7ruz2y5Jj6n1pqLed2wG3ey2q+J+3tNK03j3N824WuC9q3L68yrC/oKkqWZ2rpkNk3SLpGUV9PE+ZjYy++FEZjZS0lVqvamol0mam92fK+mJCns5TqtM4503zbgqfu8qn/7c3Zv+J+la9f4iv03Sn1XRQ05f50n6n+xvU9W9SXpYvR/rjqj3t415ksZKWilpi6QfSWprod6+J2mDpPXqDdaEinq7TL0f0ddLWpf9XVv1e5foqynvG4fLAkHwAx0QBGEHgiDsQBCEHQiCsANBEHYgCMIOBPF/zDhToRPAk+sAAAAASUVORK5CYII=\n","text/plain":["<Figure size 432x288 with 1 Axes>"]},"metadata":{"tags":[],"needs_background":"light"}}]},{"cell_type":"markdown","metadata":{"id":"LGjZJlgaqpKz","colab_type":"text"},"source":["위 과정을 모두 완료하셨으면 오버피팅 현상을 경험해보기 위해 학습을 더 시키고 정확도를 확인해 봅시다."]}]}