code/neural_network_regression.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Neural network exercises: Regression\n",
        "\n",
        "In this notebook, we will have a quick look at a regression neural network. The exercises are very short, but can be useful just for understanding the basic mechanics of the framework we use. Here we run [Keras](keras.io), which one of the two major ones (the other being [PyTorch](https://pytorch.org/)). You are of course free to use either one."
      ],
      "metadata": {
        "id": "AmRsUCECPhfo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Imports\n",
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "import matplotlib.pyplot as plt"
      ],
      "metadata": {
        "id": "_4NJ3CgjPkKe"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Gererate and plot some random data."
      ],
      "metadata": {
        "id": "zSUyXIqWSKBG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "rng = np.random.default_rng()\n",
        "\n",
        "def random_data(num_samples=120):\n",
        "    x = rng.uniform(0, 2*np.pi, size=num_samples)\n",
        "    y = np.sin(x) + rng.normal(size=num_samples, scale=0.1)\n",
        "\n",
        "    x = np.expand_dims(x, axis=1)\n",
        "    y = np.expand_dims(y, axis=1)\n",
        "    return x, y\n",
        "\n",
        "\n",
        "X, Y = random_data()\n",
        "plt.scatter(X, Y)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 447
        },
        "id": "7Nio345MPpjN",
        "outputId": "8a4bba42-c853-40ea-a1b8-36ef6e6f173d"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.collections.PathCollection at 0x7d27d341a110>"
            ]
          },
          "metadata": {},
          "execution_count": 2
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGdCAYAAADaPpOnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA410lEQVR4nO3dfXhU9Z3//9ckkASQDISbzCCIEW0xjYKAQIRtK4YCuhS6++1aK5doW135wq6Ke1X5/apIu9vodl2x1R+I1lUvijftV0RqG5e76qUNZYVvqhGLkMZCIRMKyAxEuWlyfn/EE3IzN+dM5sycM/N8XNdcl5mcM/PJMM55z+fz/rzfPsMwDAEAAHhEXqYHAAAAYAfBCwAA8BSCFwAA4CkELwAAwFMIXgAAgKcQvAAAAE8heAEAAJ5C8AIAADylT6YHkGptbW06dOiQBg4cKJ/Pl+nhAAAACwzD0IkTJzRixAjl5cWfW8m64OXQoUMaNWpUpocBAACScODAAY0cOTLuMVkXvAwcOFBS+x9fXFyc4dEAAAArIpGIRo0a1XEdjyfrghdzqai4uJjgBQAAj7GS8kHCLgAA8BSCFwAA4CkELwAAwFMIXgAAgKcQvAAAAE8heAEAAJ5C8AIAADyF4AUAAHhK1hWpA9Kptc3QjsZjOnzilIYPLNLkshLl59FTCwCcRPCCtMm2C31NfZNWbNytpvCpjvuC/iItn1uu2RXBDI4MALIbwQvSItsu9DX1TVq0dpeMbveHwqe0aO0urVowwZN/FwB4ATkvcJx5oe8cuEjnLvQ19U0ZGllyWtsMrdi4u0fgIqnjvhUbd6u1LdoRAIDeIniBo7LxQr+j8ViPQKwzQ1JT+JR2NB5L36AAIIcQvMBR2XihP3wi9t+TzHEAAHsIXuCobLzQDx9YlNLjAAD2ELzAUdl4oZ9cVqKgv0ix9kn51J6MPLmsJJ3DAoCcQfACR2XjhT4/z6flc8slqcffZf68fG65p7eBA4CbEbzAUdl6oZ9dEdSqBRMU8HedMQr4iyxtk25tM1TbcFQb6g6qtuGopxKWASDTfIZhZNWnZiQSkd/vVzgcVnFxcaaHg8/EqvNy33XlGjygIOOF65ItoJfMedlW8wYAUsHO9ZvgBWnT/UL/cctp/eC1DyxfxJ2q0OtUMBFtvJt2h6IWtzP/CorbAchVrgle3nzzTf3oRz/Szp071dTUpPXr12v+/Plxz/nNb36jpUuX6v3339eoUaP0ve99TzfffLPl5yR48YZYFWpjXcSdCjDsjsPO43Yfb6C4SKf+2qrjn5yNeo5P7ctOb90zw3PLaADQW3au347mvLS0tGjcuHF6/PHHLR3f2Nio6667TldffbXq6up055136jvf+Y5ef/11J4eJNLNbuM6pCr1OFdCLOd7IqZiBi/mcXqt5AwCZ4Ghvozlz5mjOnDmWj1+9erXKysr08MMPS5IuvfRSvfXWW3rkkUc0a9Ysp4aJNLNTuG5yWUncAMOn9gBjZnnA9myFnXFUjhli6THjBURWvb3viOebVgKAk1y126i2tlZVVVVd7ps1a5Zqa2tjnnP69GlFIpEuN7ibncJ1TlbodaKAXqLxWvHYtn2a/tBWz/V8AoB0cVXwEgqFVFpa2uW+0tJSRSIRffrpp1HPqa6ult/v77iNGjUqHUNFL9gpXOdkhV4nCuilqlKwV5tWAkA6uCp4ScayZcsUDoc7bgcOHMj0kJCAncJ1TlboTWUBPbNuy97mE7bHEY1Xm1YCQDq4KngJBAJqbm7ucl9zc7OKi4vVr1+/qOcUFhaquLi4yw3uZqdwnZMVelNVQK+mvknTH9qqG57crse2NcQ91idpcP++ChQXJhwfCbwAEJ2rgpfKykpt2bKly32bNm1SZWVlhkYEp1itUOt0hd7eVsqNtbMoGnOE1X93md6+9xotuXqMpTF6qWklAKSDo7uNTp48qX379nX83NjYqLq6OpWUlOiCCy7QsmXLdPDgQT333HOSpNtvv12PPfaYvvvd7+pb3/qWtm7dqpdeekmvvfaak8NEhsyuCGpmeSBh4TkzwOhRNyVFVWmtjqM7uzuLuo932sXDEs7USN5qWgkA6eBo8PLOO+/o6quv7vh56dKlkqSFCxfqmWeeUVNTk/bv39/x+7KyMr322mu666679Oijj2rkyJF66qmn2CadxfLzfJa2IScbYKR6HJ1Z3Vk0oDBft/3NRVoy45Iu4zWXxELhU1EDILNonZeaVgJAOtAeAEjShrqDuuOFOkvH+hS9Wq+57CSpSwBDuwAAucbO9dvRmRdkL7NvTyhySsdOnlbJgAIF/P1cWVwtmZ5IVs6xu5wTrZie00tiAJCNCF5gW7S+PSa3dUdOpieS1XMSLft0Fq9ar9NLYgCQbVy12wju1tpm6NHNH+r2OLtrmlxUXC2Znkh2zum8E8qqWDuHzJybeePPV+WYIQQuABAHwQssqalv0rQHt+qRzXsTHmsoNcXVzMJvG+oOqrbhqK3HS6bpYjLnmMs+JQMKLI2LnUMA0HssGyEhczbCTihit6Fhd79695C+t6Fex1rOdWG2sySVTNPFZBs1zq4IasbYUk2t3txlvJ3F2jmUTD4OAOQ6ghfE1ZsuyckWV6v+1W498WZjj/vNJSkrO3CS6YnUmz5K+Xk+Laws0yObP+zxu1jF9JLJxwEAsGyEBHrTJTmZJZJfvdsUNXAxWV2SSqYnUrJ9lMz2ANECFyl6td5k8nEAAO0IXhBXsrMnyfQbam0z9L0N9QmPs9LvJ5meSMmck6g9wF1Vl+ite2Z0CVySya0BAJxD8IK4kpk98Sm5fkM7Go/pWMsZS8cmCqqS6Ylk95xES2o+SS/8T88u53ZyawAAPRG8IK5EsxHdBS02NIzGziyPlaAqmaaLds5JNgjpTW4NAICEXSRgzkYsWrtLPinqLMP/mnC+pl08tNcVdq3O8pQM6Gt5SSqZAnBWz0kmCGltM3TkxGlL57GtGgCiI3hBQrFK2Kd6Z4w5y5MoQfhf51XYCpCSabpo5Ry7Cb7xKhN3RkNGAIiP4AWWpKOEfedZnlh5JP/4xTJde/mIlD1nb9jpCm21Vk6sfBwAwDnkvMCydJSwN2d5gt1yToYMKND/980JWnatvXL8ToqX4Cu1L7FdWxHQ9j8e1QOvvm+pVk68fBwAQDufYRhZtR/TTkttuJeXKs9GWw7K80l2dzrfd92lunlamWv/TgBwkp3rN8tGcKVk8lQypfOS2qbdIT399ke2AxdJGjqwkMAFACxg2QhIgfw8nyaXlejX9aGkH8PO7qLeNK0EAK9j5gVIkWRbKdjdXURPJAC5jpkXIEWSKSpnd3cRPZEAgOAFSJlkisrZ2V1ETyQAaMeyEZAiVuu+/Mf/GqcjLadt76Ky047AK8nOAJAMghcgReK1Uui8PDTtkqGWH7PzlvG9zScsnUNPJADZjuAFSKFYrRQCSSTUWm0n0B09kQBkO4IXIMVS0UrBajuBaD5uOZPEWQDgHQQvyHqZqNbbmyJ78RJzrfjBa7s1qyJAwTsAWYvgBVnNizVRkq0XYyJpF0C2Y6s0spZXa6KkIuGWpF0A2YzgBVnJyzVRUpFwS9IugGxG8IKsZKcmituY9WKSzVgJ2mg1AABeRPCCrGR12cSNyytmvRhJSQUw/8+1l2pH4zGaNgLIWiTsIitZXTZx6/JKrHoxVix/9X0d67Rd2u0JygBgl88wjKz6WhaJROT3+xUOh1VcXJzp4SBDWtsMTX9oa8JS/W/dM8PVW4q7V9h9bFuD7ccwq/3eVXWJLhw6IG3bxQHADjvXb2Zeslxrm6HtDUdV+8cjktprj0y9aEjWX7islup3++vQuV5MbcPRpIIX829/ZPPejvuYjQHgZcy8ZLGa+ibd+/J7Ov7J2S73D+rfVw/+3WU5ceHyYp2XWBLNJtlhhmxWO1oDgNPsXL8JXrJUTX2Tbl+7K+4xq3PkwpWJCrtOMWvXSEpJAOOFpTMAucHO9ZvdRlmotc3QA6++n/A4t9Y5STVz6WXe+PNVOcZ7S2atbYZqG45qQ91B+fsV6PFvXqGAv2uiccmAvrYf183bxQEgHnJestCOxmMKRU4nPI4y8u4Xa9nrvuvKNXhAQcds0sTRg/WlH21LaknJjdvFASAeghcPM5dDQuFPdazljErOK1SguEihiPWLERcu94rVWToUPqXF63Zp1YIJmjf+/I77YyUoJ+LW7eIAEAvBi0dF+0ZuGlCQb/lxuHC5U6L2Bj61L/vNLD/XPdpubRgz54VqvAC8huDFg2J9Ize1nGm19DiUkXcvO+0NOi/7za4IamZ5oCNB+aMjn2jl5g87zjF5abs4AHRH8OIx8b6R28WFy716096gc20YSfp84LweszEBj24XBwCJ4MVzEn0jt2Jw/76qzpE6L16VyvYG3WdjvL5dHAAIXjymNwm2cyoCWjB1dE5U2PU6s7N0ovYGVpf9us/GAICXUefFY3qTYFt34DiBi0fE6yxNvgqAXEfw4jHmN/JkUJDMW8zdQ90L0gX8RY6X9e9cGK+24WhOFDME4B0sG3lMfp5P9113qf73uv+b1PnUdfGWTOSrZFM/KADZieDFY2rqm/SD1z5I+nzqunhPOvNV4hXGW7R2F40cAbgCy0YeYl5Yktlt5BN1XRBfosJ4Uu70wwLgbgQvHtGb+i4keMIKO4XxACCTWDZyObN/0dv7/pJ0fRcKksGK3hTGA4B0InhxsXj9ixJZcvXFuqT0PAqSwbJUFsYDACcRvLhUov5FiUy7eChFyWBLqgvjAYBTyHlxod72LyIxF91ZqdtCYTwAXsHMiwv1pn+RT1xg0JWdui1mYTwaOQJwM4IXF0o2IbL7BclM9qUZX+5Kpm4LjRwBuB3BiwtZTYhc/OUxKhlQoJIBBQr4+3W5wFAlFYnqtvjUXrdlZnmgR2BCI0cAbkbw4kJWEyeXfuXzUb8NUyUVkr26LQQqALyEhF0X6k3iJFVSYaJuC4BsRfDiUsl2FKZKKkzUbQGQrVg2crFkEif5tg0TdVsAZCuCF5ezmzjJt22Y8vN8uu+6S/W/1/3fHr+zsvzIbiMAbkXwkmX4tg1TTX2TfvDaB1F/F69uCzvVALgdOS8eYKU6qokqqZDO7TiLlf9033WxA5do55k71WrqmxwZLwDYkZbg5fHHH9eFF16ooqIiTZkyRTt27Ih57DPPPCOfz9flVlSUu0scNfVNmv7QVt3w5Hbd8UKdbnhyu6Y/tDXuRSTZZF9kh0TtJXySfvBazx1n7FQD4BWOLxu9+OKLWrp0qVavXq0pU6Zo5cqVmjVrlvbs2aPhw4dHPae4uFh79uzp+Nnny81Zgt7Ua6FKau5Ktr6L3fPIiwGQKY4HL//5n/+pW2+9VbfccoskafXq1Xrttdf09NNP69577416js/nUyAQcHportab6qgmqqTmpmR3nNk5j7wYAJnk6LLRmTNntHPnTlVVVZ17wrw8VVVVqba2NuZ5J0+e1OjRozVq1CjNmzdP77//vpPDdCXqtSBZye44s3reR0dayIsBkFGOBi9HjhxRa2urSktLu9xfWlqqUCgU9ZzPf/7zevrpp7VhwwatXbtWbW1tuuqqq/TnP/856vGnT59WJBLpcssG1GtBsswdZ7EWcHxqnyXpvuPM6nnP79hPXgyAjHLdbqPKykrddNNNGj9+vL70pS/p5Zdf1rBhw/TEE09EPb66ulp+v7/jNmrUqDSP2BnUa0Gykt1xZuW8b1x5gUKR0zGfmxlBAOngaPAydOhQ5efnq7m5ucv9zc3NlnNa+vbtqyuuuEL79u2L+vtly5YpHA533A4cONDrcbtBst+eASn5HWeJzrtwaH9Lz8+MIAAnOZqwW1BQoIkTJ2rLli2aP3++JKmtrU1btmzRkiVLLD1Ga2ur3nvvPV177bVRf19YWKjCwsJUDdk1zG/Bi9bukk/qMk1PvRZYkeyOs3jn1TYctfTczAgCcJLju42WLl2qhQsXatKkSZo8ebJWrlyplpaWjt1HN910k84//3xVV1dLkr7//e9r6tSpuvjii3X8+HH96Ec/0p/+9Cd95zvfcXqormN+C+6+qyNedVSgs2R3nMU6jwrOANzA8eDl+uuv11/+8hfdf//9CoVCGj9+vGpqajqSePfv36+8vHOrVx9//LFuvfVWhUIhDR48WBMnTtRvf/tblZeXOz1UV6JeC9yEGUEAbuAzDCOrtgVEIhH5/X6Fw2EVFxdnejhAVqLOC4BUs3P9pjEjANuYEQSQSQQvAJJCBWcAmeK6Oi8AAADxELwAAABPIXgBAACeQvACAAA8heAFAAB4CsELAADwFIIXAADgKQQvAADAUwheAACApxC8AAAAT6E9AICktLYZ9DYCkBEELwBso6s0gExi2QiALTX1TVq0dleXwEWSQuFTWrR2l2rqmzI0MgC5guAFgGWtbYZWbNwtI8rvzPtWbNyt1rZoRwBAahC8ALBsR+OxHjMunRmSmsKntKPxWPoGBSDnkPPiESRHwmlW3mOHT8QOXJI5DgCSQfCShHQHEiRHwmlW32PDBxZZejyrxwFAMghebEp3IGEmR3bPIDCTI1ctmEAAg16x8x6bXFaioL9IofCpqHkvPkkBf3tADwBOIefFhnTvsiA5Ek6z+x7Lz/Np+dxySe2BSmfmz8vnlrOkCcBRBC8WZSKQIDkSTkvmPTa7IqhVCyYo4O+6NBTwFzETCCAtWDayyM6HfOWYISl5TpIj4bRk32OzK4KaWR4giRxARhC8WJSJQILkSDitN++x/DxfygJ1ALCDZSOLMhFImMmRsb7L+tSeLExyJJLFewyAFxG8WJSJD3mSI+E03mMAvIjgxaJMfciTHAmn8R4D4DU+wzCyap9tJBKR3+9XOBxWcXFxyh8/UwXjqLALp/EeA5BJdq7fBC9J4EMeAIDUsnP9ZrdREthlAQBA5pDzAgAAPIWZFwCOiLa8KoklVwC9RvCSYuTDANET2wf17ytJOv7J2Y776I4OIBkELymUqZ1IgJvE6lLdOWgx0R0dQDLIeUmRdHecBtwoXgPTaOiODiAZBC8pkKjjtCHp/11frzN/bUvzyID02t5wNG4D02jojg7ALoKXFEjUcVqSjrac0dTqzczAIGvV1Ddp8bpdSZ9Pd3QAVhG8pIDVD91jLWdZQkJWMpdNj3/aM6/FKrqjA7CK4CUF7H7osr6PbGI3zyUaOlcDsIPgJQUmjh4sq7uhWd9HtrGybJrIV8cFKSkAwDKClxTY+aePZXcihfV9ZAur7+WCPrE/bta82chyKgDLCF5SIJlAhPV9ZAur7+XzCuOXlWI5FYBVBC8pYCcQ8Yn1fWSXyWUlCvqLFGvRxyepZEBfHWs5E/MxWE4FYAfBSwok+vDubvncctb3kTXy83xaPrdcknr8P2D+/LXx51t6LJZTAVhB8JIC8T68Owv6iyiDjqw0uyKoVQsmKODvOgsZ+Ow9X1UesPQ4LKcCsILeRilifnh37200ZECB5o0foZnlAZo0IqvNrghqZnkgamPS1jZDQX+RQuFTUbdU+9Qe6LCcCsAKn2EYWZUhF4lE5Pf7FQ6HVVxcnPbnp6s0EJ1ZyE5SlwDG/L+DWUkgt9m5fhO8AEgbOq8DiMXO9ZtlIwBpE29pCQCsIngBkFb5eT5VjhmS6WEA8DB2GwEAAE8heAEAAJ5C8AIAADyF4AUAAHgKwQsAAPAUdhsBcC2KPgKIhuAFgCtR0A5ALCwbAXAds5VA58BFkkLhU1q0dpdq6psyNDIAbkDwAsBVWtsMrdi4O2oDR/O+FRt3q7UtqzqbALCB4AWAq+xoPNZjxqUzQ1JT+JR2NB5L36AAuAo5LxaROAikx+ETsQOXZI4DkH0IXiwgcRBwRrQvBcMHFlk61+pxALIPwUsCZuJg99V1M3Fw1YIJBDBAEmJ9KbjvunIF/UUKhU9FzXvxSQr42wMdALmJnJc4SBwEnBFvN9Hidbv01XHtXwi6L8yaPy+fW86yLZDD0hK8PP7447rwwgtVVFSkKVOmaMeOHXGP//nPf66xY8eqqKhIl112mX71q1+lY5g9kDgIpF6iLwWGpA11h/T4N69QwN91aSjgL2K2E4Dzy0Yvvviili5dqtWrV2vKlClauXKlZs2apT179mj48OE9jv/tb3+rG264QdXV1frbv/1brVu3TvPnz9euXbtUUVHh9HC7IHEQSL1EXwokKRQ5rb2HW/TWPTNIlAfQg88wDEfXPKZMmaIrr7xSjz32mCSpra1No0aN0j/90z/p3nvv7XH89ddfr5aWFv3yl7/suG/q1KkaP368Vq9enfD5IpGI/H6/wuGwiouLezX22oajuuHJ7QmPe/7WqaocM6RXzwXkig11B3XHC3WWjl3NLAuQM+xcvx1dNjpz5ox27typqqqqc0+Yl6eqqirV1tZGPae2trbL8ZI0a9asmMefPn1akUikyy1VJpeVKOgv6rHubvKpPcGQxEHAOju7hMgpAxCNo8HLkSNH1NraqtLS0i73l5aWKhQKRT0nFArZOr66ulp+v7/jNmrUqNQMXlJ+nk/L55ZLInEQSBXzS4EV5JQBiMbzu42WLVumcDjccTtw4EBKH392RVCrFkwgcRBIkc5fCqwgpwxAd44m7A4dOlT5+flqbm7ucn9zc7MCgUDUcwKBgK3jCwsLVVhYmJoBxzC7IqiZ5QESB4EUmV0R1F1Vl+iRzXsTHpuqYnRUyQayh6PBS0FBgSZOnKgtW7Zo/vz5ktoTdrds2aIlS5ZEPaeyslJbtmzRnXfe2XHfpk2bVFlZ6eRQE8rP85GUC6TQkhmX6PkdBxSKRJ9ZSWUxOqpkA9nF8WWjpUuX6sknn9Szzz6rDz74QIsWLVJLS4tuueUWSdJNN92kZcuWdRx/xx13qKamRg8//LD+8Ic/6IEHHtA777wTM9gB4E35eT498NVy+eRsTlm8gniL1u5STX1Trx4fQPo5Hrxcf/31+o//+A/df//9Gj9+vOrq6lRTU9ORlLt//341NZ378Ljqqqu0bt06rVmzRuPGjdMvfvELvfLKK2mv8QLAeU7nlFElG8hOjtd5SbdU1nkBkB5O5aNQqwlILSdzx+xcv2nMCCDjUplT1vnDdW/zCUvnsKMJSMxNuWMELwCyRrQPVytStaMJyFZm7lj3pRozdyzdpUM8X+cFAKTYibmJ5Pmkj1vOODQqwPvcmDtG8ALAM1rbDNU2HNWGuoOqbTja8WEZ78M1kTZDWryOXUdALImaqRpKfzVslo0AeEK89XZ/vwLbMy7drdi4WzPLAxSuA7qxmhOWztwxZl4AuF6iWi2bdkfvfWZVJr45Al5hNScsnbljBC8AXM3KevuGukMpeS52HQE9mc1UY81J+tQ+C5qKathWEbwAcDUr6+1HW86oZEDfmB+uVg0dUBg1pwbIZZ2bqTpZDdsOcl4AuJrV2ZCvjT9fT7/9kXxSl1ka8+PU37+vwp+cjTqD4/vs93f//Pddei3R/whoZ1bD7p53FqDOCwD0ZHUdvao8oCvLSmJ+uErSorW7ogY3hqTjn5yVdLbLY2aqhgXgRrMrgppZHnBFd3aCFwCuZq63h8KnYm6FNtfb8/N8cT9co31zLO7XR61t0snTf+3xuIbagxt2IgHtUlkNuzfIeQHgavHW202fnm3t2HFkfrjOG3++KscM6RJwzK4I6q17Zuiuqks0qF9fSVL4079GDVxM7EQC3IfgBYDrmevt/v59o/4+/MlZLVprrdDcpt0hrdy8V8c/PZvw2M7YiQS4B8ELAE+YWR5QUZ/oH1lWS5T3phIv/Y8A9yB4AeAJOxqPKRQ5HfP3VpZ3Em27jiYTNSwAxEfwAsATrC7bvL3vLzFnX+wu/WSqhgWA+AheAHiC1WWbx7Y1aPpDW6Pmv9hd+gn4i9gmDbgQW6UBeIKVLdOmWPVZrDxGyYC+uu9vv6BAceZqWACIj5kXAJ5gZcu0KVYCb6Iy5z5JP/zaZfraFT23WQNwD4IXAJ5hbpkO+BMv/8RK4I31GCwRAd7BshEATzFLlD+y6UM9tm1fwuOjJem6qcw5APsIXgB4Tn6eT9MuHmopeImVpOuWMucA7GPZCIAnmcm3seZKqM8CZC+CFwCelCj5VqI+C5CtCF4AeBbJt0BuIucFgKeRfAvkHoIXAJ5H8i2QW1g2AgAAnkLwAgAAPIXgBQAAeArBCwAA8BSCFwAA4CkELwAAwFMIXgAAgKcQvAAAAE8heAEAAJ5C8AIAADyF4AUAAHgKvY0AAIBa2wzPNDgleAEAIMfV1Ddpxcbdagqf6rgv6C/S8rnlml0RzODIomPZCACAHFZT36RFa3d1CVwkKRQ+pUVrd6mmvilDI4uN4AUAUqy1zVBtw1FtqDuo2oajam0zMj0kIKrWNkMrNu5WtHeoed+Kjbtd9x5m2QgAUshr0+/IbTsaj/WYcenMkNQUPqUdjcdUOWZI+gaWADMvAJAiXpx+R247fCJ24JLMcelC8AIAKeDV6XfktuEDi1J6XLoQvABACtiZfgfcYnJZiYL+IsXaEO1T+7Ln5LKSdA4rIYIXADkrlYm1Xp1+R27Lz/Np+dxySeoRwJg/L59b7rp6LyTsAshJqU6s9er0OzC7IqhVCyb0+P8h4OJEc4IXADnHTKztPs9iJtauWjDB9ge2Of0eCp+KmvfiU/vFwG3T74DUHsDMLA94psIuy0YAcopTibVenX4HTPl5PlWOGaJ5489X5Zghrn6vErwAyClOJtaa0+8Bf9eloYC/KKnZHADRsWwEIKc4nVjrtel3wIsIXgDklHQk1prT7wCcwbIRgJySqK6F5M66FgDOIXgBkFM6J9bG8tVxQZZ5ABcjeAGQc2ZXBHXbF8ti/n7Nm430IQJcjOAFQM5pbTP06u/jByf0IQLci+AFQM6hDxHgbew2ApBznNou3dpmsEUaSAOCFwA5x4nt0qnulQQgNpaNAOScRNulfbK3XdrsldR9KcrslUTyL+xKZcfzbMTMC4CcY26XXrR2l3xSlz5HdvsQJeqV5FN78u/M8gBLSLCEWbzEmHkBkJN604eo87fiZ95uJPkXKcMsnjXMvADIWcn0IYr2rdiKZHslIXcwi2edozMvx44d04033qji4mINGjRI3/72t3Xy5Mm453z5y1+Wz+frcrv99tudHCaAHGb2IZo3/nxVjhmSMHCJ9q3YiqHnFZLDgLh6s4U/13JkHJ15ufHGG9XU1KRNmzbp7NmzuuWWW3Tbbbdp3bp1cc+79dZb9f3vf7/j5/79+zs5TABIKN634nh8kgb176u7X6pTKHK6435yGNBdslv4czFHxrGZlw8++EA1NTV66qmnNGXKFE2fPl0/+clP9MILL+jQoUNxz+3fv78CgUDHrbi42KlhAoAlib4VR2MmA3/8ydkugYtEDgN6SmYLf67myDgWvNTW1mrQoEGaNGlSx31VVVXKy8vT7373u7jn/uxnP9PQoUNVUVGhZcuW6ZNPPol57OnTpxWJRLrcACDVkslZCfiLNKh/36i/M2dwaEMAk90t/IlyZKTsfX85tmwUCoU0fPjwrk/Wp49KSkoUCoVinvfNb35To0eP1ogRI/Tuu+/qnnvu0Z49e/Tyyy9HPb66ulorVqxI6dgBoDur34rvu+5SDR1YqOEDi9TWZujGn8b+stY5h6FyzJAUjRReZXcLv50cmWx7f9meebn33nt7JNR2v/3hD39IekC33XabZs2apcsuu0w33nijnnvuOa1fv14NDQ1Rj1+2bJnC4XDH7cCBA0k/NwDEYvVb8c3TyjqSf4+0nI5xdFfsRILJzhZ+p9pceIHtmZe7775bN998c9xjLrroIgUCAR0+fLjL/X/961917NgxBQIBy883ZcoUSdK+ffs0ZsyYHr8vLCxUYWGh5ccDgGQkU9jOiTYEyH5Wt/Dn8vvLdvAybNgwDRs2LOFxlZWVOn78uHbu3KmJEydKkrZu3aq2traOgMSKuro6SVIwmJ0Z0wC8w/xW3H1nRyDGzg5ztiYUPhU1L8H32blW2xAgd5hb+OPJ5feXzzAMxzJ55syZo+bmZq1evbpjq/SkSZM6tkofPHhQ11xzjZ577jlNnjxZDQ0NWrduna699loNGTJE7777ru666y6NHDlSb7zxhqXnjEQi8vv9CofD7FIC4Ag73aPN3SBS9NmaRNV8gXiy6f1l5/rtaJG6n/3sZxo7dqyuueYaXXvttZo+fbrWrFnT8fuzZ89qz549HbuJCgoKtHnzZn3lK1/R2LFjdffdd+vv//7vtXHjRieHCQC22Cls15s2BEAiufr+cnTmJROYeQHgRnZmawC7suH9Zef6TW8jAEgDKzkMQLJy7f1FV2kAAOApBC8AAMBTCF4AAICnELwAAABPIXgBAACewm4jAAAyKBu2OacbwQsAABlSU9/Uo91EMEa7CZzDshEAABlglvbvHLhIUih8SovW7lJNfVOGRuZ+BC8AAKRZa5uhFRt3R22oaN63YuNutbZlVRH8lCF4AQAgzXY0Husx49KZIakpfEo7Go+lb1AeQvACAECaHT4RO3BJ5rhcQ8IuACDnZHqHz/CBRYkPsnFcriF4AQDklJr6Jj3w6m6FIudmNQLFRXrgq+nb4TO5rERBf5FC4VNR8158kgL+9qAKPbFsBADIGTX1Tbp97a4ugYskhSKndHsad/jk5/m0fG65pPZApTPz5+Vzy6n3EgPBCwAgJ7S2Gbr35ffiHrPs5ffStsNndkVQqxZMUMDfdWko4C/SqgUTqPMSB8tGAICcsP2PR3X8k7Nxj/n4k7Pa/sejmnbx0LSMaWZ5QAML+6r2j0ck+VQ5ZoimXjSEGZcECF4AwEGZTgzFObUNRy0fl8rgJdZ7IFp13f+z689U17WA4AUAHELpd7exuhyUumWjWO+Br44Las2bjT2eyayuy7JRfOS8AIADKP3uPpUXWZtNsXpcIvHeA09ECVwkqutaRfACAClG6Xd3mjpmiAb17xv3mEH9+2rqmCG9fi4r74FYqK6bGMELAKQYpd/dKT/Ppwf/7rK4xzz4d5elJCcp0XvACqrrxkbwAgApRul395pdEdTqBRMUKC7scn+guFCrU5hnkop/W6rrxkbCLgCkGKXf3W12RVAzywOO7gLrzb8t1XUTI3gBgBSj9Lv75ee111RxSqL3QCxU17WGZSMASDFKvyPRe8An6R+/WKYg1XWT4jMMI6vS3SORiPx+v8LhsIqLizM9HAA5jDovSPQeoIjhOXau3wQvAOCgWBcnLlq5g39ra+xcv8l5AQAHRcutYEYmtzidX5OLyHkBgDSyUnm3tc1QbcNRbag7qNqGoxSzA7ph2QgA0qS1zdD0h7bGLF7mU3uF18I+eQpFTnfcz6yMPXaWaVjScQ+WjQDAhaxU3v34k7M97qdZn3V2luRYvvMulo0AIE2SrbpKPyRr7DTDpHGmtxG8AECa9KbqKv2Q4rPTDJPGmd7HshEApEmyVVc7O3ziFHkaUdhthmn1WHYJuRPBCwCkiVl1ddHaXfJJSQUwHx1p6ZH0S56GtGl3yNJxdpbuaJzpXiwbAUAaza4IatWCCQp0Kwsf9BdpUP++PUrJm3ySBvfvq0c27yVPo5ua+iY9/fZHlo4dPrCIxplZgJkXAEizWF2NN+0ORZ2VMQOaWDM1xmfHrNi4WzPLAzm1hGTmr1gR7NQMk8aZ3sbMCwBkgFl1dd7481U5Zojy83wxZ2UC/iLdWfU5HY+yjdqUqwm9iXJdOjObYdI40/uYeQEAF4k1K/PLdw9ZOt9qnkbnpN+h5xVKhnSk5bTnEoCt/r3fnnZhl5wgM1DsXuclQP6QJxC8AIDLROuFYzX/Ym/zSdU2HI0bgEQrztaZlxKArb4uVeWBHvfFChS9ErjlMtoDAIAHmK0FrG6zjldVdtHaXXEfw7x0e6Gib6LXxcxfeeueGQQlLmfn+k3OCwB4QLw8jWii7UCKV5ytMy8VaiN/JTcRvACAR8RK6I0mWgBiJ7nVSwnA8RKdvTB7BPvIeQEAD+mcp/H2vr/osW0NMY/tXik2maJrm3aHPFFllvyV3ELwAgAeYyb0Wg1GzOOSKbr29NsfaXJZiSdmL6IlOsdDmwXvIngBAI+yWyk2md5K2Vr8LtqOq5IBBZo/foRmlgc6AhkCHHcieAEAj0oUjHSvFJtMb6VsbFIYa8fVsZYzevrtj/T02x8p6C/SV8cF9ervm+gj5UIk7AKARyWz08ZO0m9n2dKk0OqOq6bwKT3xZiN9pFyK4AUAPCyZnTazK4J6654Zev7WqVpy9cWWnsdNTQpb2wzVNhzVhrqDqm04ams7t50dV9F4aRt5NmPZCAA8LpmdNmZy6+SyEv2fXX/2TJPCaLkqdpZyUjGDlI1LaV7DzAsAZIFojR6tnueVIm9mrkpvlnKGDihM2XiyZSnNiwheACDHeaHIW7xcFatLOTX1Tbr7579P2ZjctJSWa1g2AgC4vshbolyVREs5Vno62ZHnkyaOHpyiR4NdzLwAQJbpTUKrW9ktyNeZ1R1GdrQZ0s4/fZzCR4QdzLwAQBZJNqG1t4mwThdzs1uQr7Pe7jCKhZyXzCF4AYAsEWtpxExojZW/kux5nc/vTeBjhd2CfJ05FWSQ85I5LBsBQBZINqG1t4mwqdgBZEVvdkWlOsjwqT04c8v28VxE8AIAWcBOQmsqzpNSswPIjmR3RZmzNqngtu3juYplIwDIAskmtG7aHbJ03q8/m0HpnMtidwdQKvJiki3It3xuuW5fu8vWcw3u31eGpOOfnO24L0BvI1cgeAGALJBMQmtNfZOefvsjS+c9V/snPVf7py65LHYCplTmxZgF+eyYXRHUt6ddqJ9a+HtvqhytORXBjmUht24fz2UsGwFAFjCXRmJdVrvnaZhLPnY1dcplsRowfXSkJS15MYlUlQcsHTenIthRpTjZysVwFsELAGQBuwmtvdk+bKg9l2Xi6MGWAqbnd+xPW15MPHYDPLgXwQsAZAk7Ca293T7cFD6lnX/6OGHA9I0rL1Aocjrm48RLCE41L/VxQnyOBS//9m//pquuukr9+/fXoEGDLJ1jGIbuv/9+BYNB9evXT1VVVdq7d69TQwSArDO7Iqi37pmh52+dqke/MV7P3zpVb90zo0deSSq2Dx8+cSphwHTh0P6WHysdvNDHCYk5lrB75swZff3rX1dlZaV++tOfWjrn3//93/XjH/9Yzz77rMrKynTfffdp1qxZ2r17t4qKKAYEAFZYSWhNVPTNCjMAircDqLbhqK3HSge393FCYo4FLytWrJAkPfPMM5aONwxDK1eu1Pe+9z3NmzdPkvTcc8+ptLRUr7zyir7xjW84NVQAyDnmEsqitbvkk2wHMEMGFHTJDYkVMPWmMq6TktmxBPdwTc5LY2OjQqGQqqqqOu7z+/2aMmWKamtrY553+vRpRSKRLjcAQGKxllB8FiYgfjCvomOmIl4jSPJM4ATX1HkJhdoLJZWWlna5v7S0tON30VRXV3fM8gAA7Om+hPLRkRY9sjl+ruE/frFM117enhtipX6LGSR1P46Cb0iWreDl3nvv1UMPPRT3mA8++EBjx47t1aDsWLZsmZYuXdrxcyQS0ahRo9L2/ADgdeYSSmuboekPbY177KB+ffTFS4ZrQ91BfXTkE63c/KGlho7kmSCVbAUvd999t26++ea4x1x00UVJDSQQaC8e1NzcrGDwXBTe3Nys8ePHxzyvsLBQhYWFST0nAOAcK7Vfjn/6V93409/FPcZQ+5LQio27NbM80BGguCHPJBUtCpB5toKXYcOGadiwYY4MpKysTIFAQFu2bOkIViKRiH73u99p0aJFjjwnAOCcVG5X7t7XyA1S2aIAmeVYwu7+/ftVV1en/fv3q7W1VXV1daqrq9PJkyc7jhk7dqzWr18vSfL5fLrzzjv1r//6r3r11Vf13nvv6aabbtKIESM0f/58p4YJAPiME9uV01W/JZGa+iZXtChAajiWsHv//ffr2Wef7fj5iiuukCRt27ZNX/7ylyVJe/bsUTgc7jjmu9/9rlpaWnTbbbfp+PHjmj59umpqaqjxAgBpkIraL92ls35LLGYfp1gtCqItccHdfIZhON9QIo0ikYj8fr/C4bCKi4szPRwA8BRzhkKyX/ulM7N+y1v3zOgSEGQi56S24ahueHJ7wuOev3Wqa5a4cpGd67drtkoDADIv1rbmZHSv35KpnBOrS1duWeJCYgQvAIAuum9rHnpeoe5+qU7NkdOWZ2PurPpcl4DEnNGxsq061awuXblhiQvWuKbCLgDAPcxtzfPGn69pFw/VA1/9gq3zI5+e6fjvRDknUnvOSefKvKlk5vLEWpzyqX0GKN0tCpA8ghcAQELmclLJgL6Wjv/p2x/p0c0fduS4xFuC6ryt2gm0KMg+BC8AAEtmVwS1fVmVSgYUWDr+kc17Ne3Brdq0O3aLl86czDmJ1ccp4C9ydMkKziDnBQBgWUGfPP3waxW6/bMdSYmEIqf09NsfWTrW6ZwTWhRkD4IXAIAtsyuC+ta0Cy0HJT61d6qOldJibqtOR86JG1oUoPdYNgIA2DazPGD5WEPnAhdyTpAKBC8AANsS7eCJ5parLtTgbvky5JwgGQQvAADbOu/gsWrD7w/qWMu5LdQlA/rqvusuzUjg0tpmqLbhqDbUHVRtw1HHtmnDGbQHAAAkraa+SQ+8+r5CkdO2zzVnbdI980J3aXeyc/1m5gUAkLTZFUG9fe81uqvqc1F/H29ZKR0F6rqju3R2IHgBAPRKfp5Pd1RdotULJijYrY5KopowTheo6yzTlX6ROmyVBgCkRLQ6KqHwp7rrpd8nPDcdTRHtVPplO7W7EbwAAFKmex2V2oajls7b23xStQ1HHS0aR3fp7MGyEQDAMVa3VD+2bZ9ueHK7pj+01bG8E7pLZw+CFwCAY+I1RYzGycRZuktnD4IXAICjYjVFjMbJxFm6S2cPghcAgONmVwT11j0z9PytU7Xk6jFxj3VyBxLdpbMDCbsAgLQwk3kznThLd2nvI3gBAKSVGxJn6S7tbSwbAQDSisRZ9BbBCwAgrUicRW8RvAAA0o7EWfQGOS8AgIwgcRbJIngBAGQMibNIBstGAADAUwheAACApxC8AAAATyF4AQAAnkLwAgAAPIXgBQAAeArBCwAA8BSCFwAA4CkELwAAwFOyrsKuYRiSpEgkkuGRAAAAq8zrtnkdjyfrgpcTJ05IkkaNGpXhkQAAALtOnDghv98f9xifYSXE8ZC2tjYdOnRIAwcOlM9nvblXJBLRqFGjdODAARUXFzs4Qm/jdbKG18kaXidreJ2s47Wyxo2vk2EYOnHihEaMGKG8vPhZLVk385KXl6eRI0cmfX5xcbFr/iHdjNfJGl4na3idrOF1so7Xyhq3vU6JZlxMJOwCAABPIXgBAACeQvDymcLCQi1fvlyFhYWZHoqr8TpZw+tkDa+TNbxO1vFaWeP11ynrEnYBAEB2Y+YFAAB4CsELAADwFIIXAADgKQQvAADAUwheJD3++OO68MILVVRUpClTpmjHjh2ZHpLrvPnmm5o7d65GjBghn8+nV155JdNDcqXq6mpdeeWVGjhwoIYPH6758+drz549mR6W66xatUqXX355R4GsyspK/frXv870sFzvwQcflM/n05133pnpobjKAw88IJ/P1+U2duzYTA/LlQ4ePKgFCxZoyJAh6tevny677DK98847mR6WbTkfvLz44otaunSpli9frl27dmncuHGaNWuWDh8+nOmhuUpLS4vGjRunxx9/PNNDcbU33nhDixcv1vbt27Vp0yadPXtWX/nKV9TS0pLpobnKyJEj9eCDD2rnzp165513NGPGDM2bN0/vv/9+pofmWv/zP/+jJ554Qpdffnmmh+JKX/jCF9TU1NRxe+uttzI9JNf5+OOPNW3aNPXt21e//vWvtXv3bj388MMaPHhwpodmn5HjJk+ebCxevLjj59bWVmPEiBFGdXV1BkflbpKM9evXZ3oYnnD48GFDkvHGG29keiiuN3jwYOOpp57K9DBc6cSJE8Yll1xibNq0yfjSl75k3HHHHZkekqssX77cGDduXKaH4Xr33HOPMX369EwPIyVyeublzJkz2rlzp6qqqjruy8vLU1VVlWprazM4MmSLcDgsSSopKcnwSNyrtbVVL7zwglpaWlRZWZnp4bjS4sWLdd1113X5rEJXe/fu1YgRI3TRRRfpxhtv1P79+zM9JNd59dVXNWnSJH3961/X8OHDdcUVV+jJJ5/M9LCSktPBy5EjR9Ta2qrS0tIu95eWlioUCmVoVMgWbW1tuvPOOzVt2jRVVFRkejiu89577+m8885TYWGhbr/9dq1fv17l5eWZHpbrvPDCC9q1a5eqq6szPRTXmjJlip555hnV1NRo1apVamxs1N/8zd/oxIkTmR6aq/zxj3/UqlWrdMkll+j111/XokWL9M///M969tlnMz0027KuqzTgFosXL1Z9fT1r7zF8/vOfV11dncLhsH7xi19o4cKFeuONNwhgOjlw4IDuuOMObdq0SUVFRZkejmvNmTOn478vv/xyTZkyRaNHj9ZLL72kb3/72xkcmbu0tbVp0qRJ+uEPfyhJuuKKK1RfX6/Vq1dr4cKFGR6dPTk98zJ06FDl5+erubm5y/3Nzc0KBAIZGhWywZIlS/TLX/5S27Zt08iRIzM9HFcqKCjQxRdfrIkTJ6q6ulrjxo3To48+mulhucrOnTt1+PBhTZgwQX369FGfPn30xhtv6Mc//rH69Omj1tbWTA/RlQYNGqTPfe5z2rdvX6aH4irBYLDHl4NLL73Uk0tsOR28FBQUaOLEidqyZUvHfW1tbdqyZQtr70iKYRhasmSJ1q9fr61bt6qsrCzTQ/KMtrY2nT59OtPDcJVrrrlG7733nurq6jpukyZN0o033qi6ujrl5+dneoiudPLkSTU0NCgYDGZ6KK4ybdq0HqUbPvzwQ40ePTpDI0pezi8bLV26VAsXLtSkSZM0efJkrVy5Ui0tLbrlllsyPTRXOXnyZJdvMY2Njaqrq1NJSYkuuOCCDI7MXRYvXqx169Zpw4YNGjhwYEfulN/vV79+/TI8OvdYtmyZ5syZowsuuEAnTpzQunXr9Jvf/Eavv/56pofmKgMHDuyRLzVgwAANGTKEPKpO/uVf/kVz587V6NGjdejQIS1fvlz5+fm64YYbMj00V7nrrrt01VVX6Yc//KH+4R/+QTt27NCaNWu0Zs2aTA/Nvkxvd3KDn/zkJ8YFF1xgFBQUGJMnTza2b9+e6SG5zrZt2wxJPW4LFy7M9NBcJdprJMn4r//6r0wPzVW+9a1vGaNHjzYKCgqMYcOGGddcc43x3//935keliewVbqn66+/3ggGg0ZBQYFx/vnnG9dff72xb9++TA/LlTZu3GhUVFQYhYWFxtixY401a9ZkekhJ8RmGYWQobgIAALAtp3NeAACA9xC8AAAATyF4AQAAnkLwAgAAPIXgBQAAeArBCwAA8BSCFwAA4CkELwAAwFMIXgAAgKcQvAAAAE8heAEAAJ5C8AIAADzl/wctME+IIecZDQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Activation functions\n",
        "\n",
        "Let's investigate the impact of selecting different activation functions. This snippet makes a plot of three common ones -- for more, have a look at https://keras.io/api/layers/activations/."
      ],
      "metadata": {
        "id": "7IuBKknUPyJg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "relu = tf.keras.activations.relu\n",
        "tanh = tf.keras.activations.tanh\n",
        "softplus = tf.keras.activations.softplus\n",
        "\n",
        "x_activ = np.linspace(-3, 3, 100)\n",
        "plt.figure()\n",
        "\n",
        "for activ_func in [relu, tanh, softplus]:\n",
        "    plt.plot(x_activ, activ_func(x_activ), label=activ_func.__name__)\n",
        "\n",
        "plt.legend()"
      ],
      "metadata": {
        "id": "njipm8U_Pw6S"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Train a simple regression model, like the one we saw in lectures. We use `Dense` layers, which are just the \"ordinary\" ones. If we want to process images or text, we need different types of layers."
      ],
      "metadata": {
        "id": "yMIrhc-KPwkH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# This is the model specification. We have two hidden layers, with ten nodes each.\n",
        "# To predict a single value, we need a single node in the output layer.\n",
        "model = tf.keras.Sequential(\n",
        "    [\n",
        "        tf.keras.layers.Dense(10, activation=tanh),     # modify here\n",
        "        tf.keras.layers.Dense(10, activation=tanh),     # modify here\n",
        "        tf.keras.layers.Dense(1, activation=None)       # for regression, don't apply activation in last layer\n",
        "    ]\n",
        ")\n",
        "\n",
        "# Compile the model. Necessary for the framework to figure out the computation graph\n",
        "model.compile(\n",
        "    loss='mse', # Loss function: mean squared error\n",
        "    optimizer=tf.keras.optimizers.Adam(learning_rate=0.01)\n",
        ")\n",
        "\n",
        "# The training step. `epochs` indicate the numer of iterations.\n",
        "model.fit(X, Y, epochs=250, verbose=0)\n",
        "\n",
        "# Plot the training data\n",
        "plt.scatter(X, Y)\n",
        "\n",
        "# Plot the predicted function, by making a grid og x-values and predicting the\n",
        "# y-value for each of them\n",
        "x_grid = np.linspace(0, 2*np.pi, num=200)[:, np.newaxis]\n",
        "preds = model.predict(x_grid)\n",
        "plt.plot(x_grid, preds, color=\"darkorange\", linewidth=2)"
      ],
      "metadata": {
        "id": "WPvHGbRuQaaf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Exercise 1\n",
        "\n",
        "Change the activation functions above, and see of the prediction line changes.\n",
        "\n",
        "## Exercise 2\n",
        "Vary the number of epochs, and see if a low number of iterations results in underfitting, and if a large number results in overfitting.\n",
        "\n",
        "## Exercise 3\n",
        "Change the model architure by adding or removing layers and nodes. Again check for under- and overfitting."
      ],
      "metadata": {
        "id": "5xF0Sa1HRN-d"
      }
    }
  ]
}