TensorFlowBeginner.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d8421b9c-25cc-4ab9-9b7a-9c4feb1d9e10",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ed807198-a3af-4f48-9852-66f76ce5c763",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorFlow version: 2.12.0\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(\"TensorFlow version:\", tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e1a9f12-f663-451b-83f8-69d76077b7ed",
   "metadata": {},
   "source": [
    "# MNIST Example"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4aae2275-b10d-4b34-9606-9a2dadfa0477",
   "metadata": {},
   "source": [
    "The below uses the MNIST database of handwritten digits:\n",
    "\n",
    "http://yann.lecun.com/exdb/mnist/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "919e9bdb-13f8-44bf-ae50-e21a077f998d",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Load a dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fec4549-2e97-4794-b584-55177e1cad0e",
   "metadata": {},
   "source": [
    "Load and prepare the [MNIST dataset](http://yann.lecun.com/exdb/mnist/). \n",
    "\n",
    "The pixel values of the images range from 0 to 255. Scale these values to a range of 0 to 1 by dividing the values by `255.0`. This also converts the sample data from integers to floating-point numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1059b453-7f70-4092-9c66-96b443f266e4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81deeec3-6e16-4bea-b453-a431d34653e3",
   "metadata": {},
   "source": [
    "## Build a machine learning model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48e5c9e7-2e24-4ccc-b244-7730c3bfcd22",
   "metadata": {
    "tags": []
   },
   "source": [
    "Build a [tf.keras.Sequential](https://www.tensorflow.org/api_docs/python/tf/keras/Sequential) model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c205f9a6-2ccc-4c42-9a37-9cf333043e08",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dropout(0.2),\n",
    "  tf.keras.layers.Dense(10)\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f396067e-ed90-4852-b8a4-6d0e11701283",
   "metadata": {},
   "source": [
    "[Sequential](https://www.tensorflow.org/guide/keras/sequential_model) is useful for stacking layers where each layer has one input [tensor](https://www.tensorflow.org/guide/tensor) and one output tensor. Layers are functions with a known mathematical structure that can be reused and have trainable variables. Most TensorFlow models are composed of layers. This model uses the [Flatten](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Flatten), [Dense](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense), and [Dropout](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dropout) layers.\n",
    "\n",
    "For each example, the model returns a vector of [logits](https://developers.google.com/machine-learning/glossary#logits) or [log-odds](https://developers.google.com/machine-learning/glossary#log-odds) scores, one for each class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "483544cb-a06e-4d8e-a163-74f1241850bf",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.0907826 ,  0.02561883,  0.759544  , -0.01242453,  0.25221872,\n",
       "         0.04480809,  0.16548052,  0.39588165,  0.21627727, -0.63472855]],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions = model(x_train[:1]).numpy()\n",
    "predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aad76850-78be-4ca4-9f76-9e54e20b4bb3",
   "metadata": {},
   "source": [
    "The [tf.nn.softmax](https://www.tensorflow.org/api_docs/python/tf/nn/softmax) function converts these logits to probabilities for each class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "debb3f74-7e55-4356-8e64-f12521b163ec",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.09113245, 0.08538327, 0.17787398, 0.08219601, 0.10709862,\n",
       "        0.08703753, 0.09820054, 0.12364481, 0.10331769, 0.04411513]],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.nn.softmax(predictions).numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e742f1b0-21a5-4216-910d-17cfd6134901",
   "metadata": {},
   "source": [
    "Define a loss function for training using [losses.SparseCategoricalCrossentropy](https://www.tensorflow.org/api_docs/python/tf/keras/losses/SparseCategoricalCrossentropy):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "cebba92f-ce95-45e7-aa33-07555c6d7df9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1a305c9-f0d1-46e2-ad37-17f05f51c84f",
   "metadata": {},
   "source": [
    "The loss function takes a vector of ground truth values and a vector of logits and returns a scalar loss for each example. This loss is equal to the negative log probability of the true class: The loss is zero if the model is sure of the correct class.\n",
    "\n",
    "This untrained model gives probabilities close to random (1/10 for each class), so the initial loss should be close to `-tf.math.log(1/10) ~= 2.3`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e5d518b1-da95-4045-a114-c9bca38f6179",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.4414158"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loss_fn(y_train[:1], predictions).numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9073ce47-fab8-4274-99ed-eaf9639f1de1",
   "metadata": {},
   "source": [
    "Before you start training, configure and compile the model using Keras [Model.compile](https://www.tensorflow.org/api_docs/python/tf/keras/Model#compile). Set the [optimizer](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers) class to `adam`, set the `loss` to the `loss_fn` function you defined earlier, and specify a metric to be evaluated for the model by setting the `metrics` parameter to `accuracy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "fee4986c-b3cc-4b32-ad22-03bd1d98163c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam',\n",
    "              loss=loss_fn,\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5e67cb6-bd89-4ba2-8542-7cfe488d00c0",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Train and evaluate your model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5bd10de-1966-4f15-a431-4577b82c264a",
   "metadata": {
    "tags": []
   },
   "source": [
    "Use the [Model.fit](https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit) method to adjust your model parameters and minimize the loss:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "26bf3f70-34c5-49e2-a47f-bd9443cf4b0a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.2990 - accuracy: 0.9126\n",
      "Epoch 2/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.1437 - accuracy: 0.9572\n",
      "Epoch 3/5\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.1063 - accuracy: 0.9680\n",
      "Epoch 4/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.0854 - accuracy: 0.9728\n",
      "Epoch 5/5\n",
      "1875/1875 [==============================] - 3s 1ms/step - loss: 0.0740 - accuracy: 0.9762\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x243a1bbce50>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "807aed73-da82-4003-94a6-849cc43da6f6",
   "metadata": {},
   "source": [
    "The [Model.evaluate](https://www.tensorflow.org/api_docs/python/tf/keras/Model#evaluate) method checks the model's performance, usually on a [validation set](https://developers.google.com/machine-learning/glossary#validation-set) or [test set](https://developers.google.com/machine-learning/glossary#test-set)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d45f915a-0403-49a5-a40e-6e4295cb8b08",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 - 0s - loss: 0.0691 - accuracy: 0.9788 - 358ms/epoch - 1ms/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.06912174820899963, 0.9787999987602234]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(x_test,  y_test, verbose=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7447faa4-35c1-4831-a090-91bee434e0b3",
   "metadata": {},
   "source": [
    "The image classifier is now trained to ~98% accuracy on this dataset. To learn more, read the [TensorFlow tutorials](https://www.tensorflow.org/tutorials/).\n",
    "\n",
    "If you want your model to return a probability, you can wrap the trained model, and attach the softmax to it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "2a6371c9-dc2f-4217-b46f-3d0c2f4325d1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "probability_model = tf.keras.Sequential([\n",
    "  model,\n",
    "  tf.keras.layers.Softmax()\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "3dbdbf1b-5d7c-4ff0-a1f7-52a3cfba1ebd",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5, 10), dtype=float32, numpy=\n",
       "array([[2.0434395e-06, 2.2819911e-07, 1.7241711e-05, 3.1198311e-04,\n",
       "        9.0869022e-12, 8.2517471e-07, 4.4351111e-12, 9.9965262e-01,\n",
       "        1.6983607e-06, 1.3372014e-05],\n",
       "       [6.2869283e-09, 2.2103530e-03, 9.9778521e-01, 4.4468497e-06,\n",
       "        4.4686422e-15, 4.1234513e-10, 4.0277706e-10, 5.1603195e-14,\n",
       "        1.6946629e-08, 1.6961159e-14],\n",
       "       [4.4979103e-08, 9.9904364e-01, 2.7563493e-04, 3.1229686e-06,\n",
       "        8.9194167e-05, 4.8298311e-06, 1.7517121e-05, 4.9518654e-04,\n",
       "        7.0532558e-05, 2.4356277e-07],\n",
       "       [9.9992597e-01, 7.7108533e-09, 5.4130254e-05, 1.1515327e-07,\n",
       "        7.7532682e-09, 2.1377681e-07, 2.2265076e-06, 1.7082326e-05,\n",
       "        1.6351828e-08, 2.0325712e-07],\n",
       "       [1.0946256e-06, 1.9718034e-10, 1.5284771e-06, 5.8868638e-10,\n",
       "        9.9792743e-01, 4.8041797e-09, 8.1433456e-07, 2.5234202e-05,\n",
       "        7.5788272e-08, 2.0438107e-03]], dtype=float32)>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "probability_model(x_test[:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "545b272a-65a4-4bbd-9068-4987f93977d3",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a3e9a15-3110-4e24-952d-1a580f7d886c",
   "metadata": {
    "tags": []
   },
   "source": [
    "Congratulations! You have trained a machine learning model using a prebuilt dataset using the [Keras](https://www.tensorflow.org/guide/keras/sequential_model) API.\n",
    "\n",
    "For more examples of using Keras, check out the [tutorials](https://www.tensorflow.org/tutorials/keras/). To learn more about building models with Keras, read the [guides](https://www.tensorflow.org/guide/keras). If you want learn more about loading and preparing data, see the tutorials on [image data loading](https://www.tensorflow.org/tutorials/load_data/images) or [CSV data loading](https://www.tensorflow.org/tutorials/load_data/csv)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}