codes/ipython/1-basics/models.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "models.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MDevcewD85Im"
      },
      "source": [
        "## Models in TensorFlow\n",
        "\n",
        "In TensorFlow, you always need to define models to train a machine learning model. A model consists of layers that conduct operations and can be reused in the model's structure. Let's get started.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iJoLp_aUvBFR"
      },
      "source": [
        "# Loading necessary libraries\n",
        "import tensorflow as tf\n",
        "import numpy as np"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t2tybyJb7Vdf"
      },
      "source": [
        "### Layer\n",
        "\n",
        "In TensorFlow, we can implement layers using the high-level [tf.Module](https://www.tensorflow.org/api_docs/python/tf/Module) class."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oBvht7tVAX4I",
        "outputId": "820d740e-87b7-499e-cea0-f949c4e3bb4f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        }
      },
      "source": [
        "class SampleLayer(tf.Module):\n",
        "  \"\"\"\n",
        "  We define the layer with a class that inherited the structure of tf.Module class.\n",
        "  \"\"\"\n",
        "  def __init__(self, name=None):\n",
        "    super().__init__(name=name)\n",
        "\n",
        "    # Define a trainable variable\n",
        "    self.x = tf.Variable([[1.0, 3.0]], name=\"x_trainable\")\n",
        "\n",
        "    # Define a non-trainable variable\n",
        "    self.y = tf.Variable(2.0, trainable=False, name=\"y_non_trainable\")\n",
        "  def __call__(self, input):\n",
        "    return self.x * input + self.y\n",
        "\n",
        "# Initialize the layer\n",
        "# Here, __call__ function will not be called\n",
        "simple_layer = SampleLayer(name=\"my_layer\")\n",
        "\n",
        "# Call the layer and extract some information\n",
        "output = simple_layer(tf.constant(1.0))\n",
        "print(\"Output:\", output)\n",
        "print(\"Layer name:\", simple_layer.name)\n",
        "print(\"Trainable variables:\", simple_layer.trainable_variables)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Output: tf.Tensor([[3. 5.]], shape=(1, 2), dtype=float32)\n",
            "Layer name: my_layer\n",
            "Trainable variables: (<tf.Variable 'x_trainable:0' shape=(1, 2) dtype=float32, numpy=array([[1., 3.]], dtype=float32)>,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DFVkXDUK9tlj"
      },
      "source": [
        "### Model\n",
        "\n",
        "Now. let's define a model. A model consists of multiple layers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cp01Jsqg84ps",
        "outputId": "4107ee01-08c0-4617-fad7-8d1183bfef26",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 92
        }
      },
      "source": [
        "class Model(tf.Module):\n",
        "  def __init__(self, name=None):\n",
        "    super().__init__(name=name)\n",
        "\n",
        "    self.layer_1 = SampleLayer('layer_1')\n",
        "    self.layer_2 = SampleLayer('layer_2')\n",
        "\n",
        "  def __call__(self, x):\n",
        "    x = self.layer_1(x)\n",
        "    output = self.layer_2(x)\n",
        "    return output\n",
        "\n",
        "# Initialize the model\n",
        "custom_model = Model(name=\"model_name\")\n",
        "\n",
        "# Call the model\n",
        "# Call the layer and extract some information\n",
        "output = custom_model(tf.constant(1.0))\n",
        "print(\"Output:\", output)\n",
        "print(\"Model name:\", custom_model.name)\n",
        "print(\"Trainable variables:\", custom_model.trainable_variables)\n"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Output: tf.Tensor([[ 5. 17.]], shape=(1, 2), dtype=float32)\n",
            "Model name: model_name\n",
            "Trainable variables: (<tf.Variable 'x_trainable:0' shape=(1, 2) dtype=float32, numpy=array([[1., 3.]], dtype=float32)>, <tf.Variable 'x_trainable:0' shape=(1, 2) dtype=float32, numpy=array([[1., 3.]], dtype=float32)>)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jdpeI9wg-65r"
      },
      "source": [
        "### Keras Models\n",
        "\n",
        "Keras is a high-level API that is part of TensorFlow now. You can use [tf.keras.Model](https://www.tensorflow.org/api_docs/python/tf/keras/Model) to define a model. You can also use the collection of [tf.keras.layers](https://www.tensorflow.org/api_docs/python/tf/keras/layers) for your convenience. It's straightforward as below to define a model that has two fully-connected layers:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_D3TjQXm-p3y",
        "outputId": "59f662e1-fc73-4d39-c152-7c40d746b2f5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        }
      },
      "source": [
        "class CustomModel(tf.keras.Model):\n",
        "\n",
        "  def __init__(self):\n",
        "    super(CustomModel, self).__init__()\n",
        "    self.layer_1 = tf.keras.layers.Dense(16, activation=tf.nn.relu)\n",
        "    self.layer_2 = tf.keras.layers.Dense(32, activation=None)\n",
        "\n",
        "  def call(self, inputs):\n",
        "    x = self.layer_1(inputs)\n",
        "    out = self.layer_2(inputs)\n",
        "    return out\n",
        "\n",
        "# Create model\n",
        "custom_model = CustomModel()\n",
        "\n",
        "# Call the model\n",
        "# Call the layer and extract some information\n",
        "output = custom_model(tf.constant([[1.0, 2.0, 3.0]]))\n",
        "print(\"Output shape:\", output.shape)\n",
        "print(\"Model name:\", custom_model.name)\n",
        "\n",
        "# Count total trainable variables\n",
        "total_trainable_var = np.sum([tf.size(var).numpy() for var in custom_model.trainable_variables])\n",
        "print(\"Number of trainable variables:\", total_trainable_var)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Output shape: (1, 32)\n",
            "Model name: custom_model\n",
            "Number of trainable variables: 192\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}