_notebooks/2021-08-11-Univariate-distribution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Univariate Distribution\n",
    "\n",
    "> In this post, we will show the basic usage of tensorflow probability (tfp), and how to make univariate distribution. This is the summary of lecture \"Probabilistic Deep Learning with Tensorflow 2\" from Imperial College London\n",
    "\n",
    "- toc: true \n",
    "- badges: true\n",
    "- comments: true\n",
    "- author: Chanseok Kang\n",
    "- categories: [Python, Coursera, Tensorflow_probability, ICL]\n",
    "- image: images/bernoulli.png"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Tensorflow Probability](https://www.tensorflow.org/probability) (tfp for short) is a library for probabilistic reasoning and statistical analysis in Tensorflow. It is a part of wide ecosystem of Tensorflow, so it can easily combined with Tensorflow core."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import tensorflow_probability as tfp\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "tfd = tfp.distributions\n",
    "plt.rcParams['figure.figsize'] = (10, 6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensorflow Version:  2.4.0\n",
      "Tensorflow Probability Version:  0.11.1\n"
     ]
    }
   ],
   "source": [
    "print(\"Tensorflow Version: \", tf.__version__)\n",
    "print(\"Tensorflow Probability Version: \", tfp.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Univariate Distribution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From wikipedia,\n",
    "\n",
    "> In In statistics, a univariate distribution is a probability distribution of only one random variable. This is in contrast to a multivariate distribution, the probability distribution of a random vector (consisting of multiple random variables)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normal Distribution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of simple univariate distributions is Normal Distribution (also known as Gaussian Distribution). We can create it with tensorflow probability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tfp.distributions.Normal 'Normal' batch_shape=[] event_shape=[] dtype=float32>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a normal distribution from tensorflow distributions\n",
    "normal = tfd.Normal(loc=0, scale=1)\n",
    "normal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `loc` stands for mean($\\mu$) of distribution, and `scale` is standard distribution ($\\sigma$) of distribution. After that, we create the normal distribution object. In order to generate the data from normal distribution, we need to `sample` from it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=0.9464462>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "normal.sample()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or it can generate multiple samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5,), dtype=float32, numpy=\n",
       "array([-0.73940355,  0.79343444,  0.29724854,  0.6113488 ,  0.02649165],\n",
       "      dtype=float32)>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "normal.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we generate 10000 samples and plot it, its shape will be bell-shaped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAAFlCAYAAADYqP0MAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAXSUlEQVR4nO3dfaxlV3kf4N+bcaxULggpHj7q8XSsxCq1UkzRxKQyagoNaMxEDDStYkKhCaCRJdwEqagMRaJqUdVBqdq0qpORRa0oSqgVKbgd4QHbTVP5D6CdMTXGBoxGzrS+DMgG0hBEhBl4+8c5g06ur7n72nfNuR/PI13ds9dea5/3bA2+P9beZ+3q7gAAsLl+ZNkFAADsREIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwACXLbuAtVx55ZV94MCBZZcBALCuBx544GvdvXd1+5YMWQcOHMiZM2eWXQYAwLqq6v+s1e5yIQDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAJctuwCArebAsbsn9Tt3/PDgSoDtzEwWAMAAQhYAwABCFgDAAEIWAMAAbnwHdo2pN7QDbAYzWQAAAwhZAAADCFkAAAMIWQAAAwhZAAADTApZVXWoqh6tqrNVdeyH9PvpqvpeVf39jY4FANhJ1g1ZVbUnyW1JbkpyXZI3V9V1z9DvQ0nu2ehYAICdZspM1g1Jznb3Y939VJI7kxxZo98/TvIHSZ54FmMBAHaUKYuRXpXk8YXtlSSvXOxQVVcleVOS1yT56Y2MXTjG0SRHk2T//v0TygKwwCiwdU2Zyao12nrV9m8keW93f+9ZjJ01dt/e3Qe7++DevXsnlAUAsHVNmclaSXL1wva+JOdX9TmY5M6qSpIrk7y+qi5MHAsAsONMCVmnk1xbVdck+XKSm5P80mKH7r7m4uuq+u0kH+vu/1JVl603FgBgJ1o3ZHX3haq6NbNvDe5Jckd3P1JVt8z3n9jo2M0pHQBg65oyk5XuPpXk1Kq2NcNVd//yemMBAHY6K74DAAwgZAEADCBkAQAMIGQBAAww6cZ3AJ5u6mrz544fHlwJsBWZyQIAGEDIAgAYQMgCABhAyAIAGEDIAgAYQMgCABhAyAIAGMA6WcCWNHUNKoCtykwWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAFZ8By4pK7kDu4WZLACAAYQsAIABhCwAgAGELACAAYQsAIABJoWsqjpUVY9W1dmqOrbG/iNV9VBVPVhVZ6rqVQv7zlXV5y7u28ziAQC2qnWXcKiqPUluS/LaJCtJTlfVye7+/EK3P0xysru7ql6W5PeTvHRh/6u7+2ubWDcAwJY2ZSbrhiRnu/ux7n4qyZ1Jjix26O5vdXfPN69I0gEA2MWmhKyrkjy+sL0yb/sLqupNVfXFJHcnefvCrk5yb1U9UFVHn+lNquro/FLjmSeffHJa9QAAW9SUkFVrtD1tpqq77+rulyZ5Y5IPLuy6sbtfkeSmJO+qqr+91pt09+3dfbC7D+7du3dCWQAAW9eUkLWS5OqF7X1Jzj9T5+6+P8lPVNWV8+3z899PJLkrs8uPAAA72pSQdTrJtVV1TVVdnuTmJCcXO1TVT1ZVzV+/IsnlSb5eVVdU1fPm7VckeV2ShzfzAwAAbEXrfruwuy9U1a1J7kmyJ8kd3f1IVd0y338iyS8keVtVfTfJnyf5xfk3DV+U5K55/rosyUe6+xODPgsAwJaxbshKku4+leTUqrYTC68/lORDa4x7LMn1z7FGAIBtx4rvAAADCFkAAAMIWQAAAwhZAAADCFkAAANM+nYhwHoOHLt72SUAbClmsgAABjCTBTDY1Fm+c8cPD64EuJTMZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAxgxXfgh/JMQoBnx0wWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAk0JWVR2qqker6mxVHVtj/5GqeqiqHqyqM1X1qqljAQB2onVDVlXtSXJbkpuSXJfkzVV13apuf5jk+u5+eZK3J/nwBsYCAOw4U2aybkhytrsf6+6nktyZ5Mhih+7+Vnf3fPOKJD11LADATnTZhD5XJXl8YXslyStXd6qqNyX510lemOTwRsbOxx9NcjRJ9u/fP6EsgJ3lwLG7J/U7d/zw+p2ApZsyk1VrtPXTGrrv6u6XJnljkg9uZOx8/O3dfbC7D+7du3dCWQAAW9eUkLWS5OqF7X1Jzj9T5+6+P8lPVNWVGx0LALBTTAlZp5NcW1XXVNXlSW5OcnKxQ1X9ZFXV/PUrklye5OtTxgIA7ETr3pPV3Req6tYk9yTZk+SO7n6kqm6Z7z+R5BeSvK2qvpvkz5P84vxG+DXHDvosAABbxpQb39Pdp5KcWtV2YuH1h5J8aOpYAICdzorvAAADCFkAAAMIWQAAAwhZAAADCFkAAANM+nYhAFuHx+/A9mAmCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYIDLll0AAGMcOHb35L7njh8eWAnsTpNmsqrqUFU9WlVnq+rYGvvfUlUPzX8+WVXXL+w7V1Wfq6oHq+rMZhYPALBVrTuTVVV7ktyW5LVJVpKcrqqT3f35hW5/nORnu/tPquqmJLcneeXC/ld399c2sW4AgC1tyuXCG5Kc7e7HkqSq7kxyJMkPQlZ3f3Kh/6eT7NvMIoHNt5FLSQBs3JTLhVcleXxhe2Xe9kzekeTjC9ud5N6qeqCqjj7ToKo6WlVnqurMk08+OaEsAICta8pMVq3R1mt2rHp1ZiHrVQvNN3b3+ap6YZL7quqL3X3/0w7YfXtmlxlz8ODBNY8PALBdTJnJWkly9cL2viTnV3eqqpcl+XCSI9399Yvt3X1+/vuJJHdldvkRAGBHmxKyTie5tqquqarLk9yc5ORih6ran+SjSd7a3V9aaL+iqp538XWS1yV5eLOKBwDYqta9XNjdF6rq1iT3JNmT5I7ufqSqbpnvP5HkA0l+PMlvVlWSXOjug0lelOSuedtlST7S3Z8Y8kkAALaQSYuRdvepJKdWtZ1YeP3OJO9cY9xjSa5f3Q4AsNN5rA4AwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAEIWAMAAQhYAwABCFgDAAJctuwBgcx04dveySwAgZrIAAIYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGsOI7AJOfFHDu+OHBlcDOYSYLAGAAIQsAYAAhCwBggEkhq6oOVdWjVXW2qo6tsf8tVfXQ/OeTVXX91LEAADvRuiGrqvYkuS3JTUmuS/LmqrpuVbc/TvKz3f2yJB9McvsGxgIA7DhTZrJuSHK2ux/r7qeS3JnkyGKH7v5kd//JfPPTSfZNHQsAsBNNCVlXJXl8YXtl3vZM3pHk4xsdW1VHq+pMVZ158sknJ5QFALB1TQlZtUZbr9mx6tWZhaz3bnRsd9/e3Qe7++DevXsnlAUAsHVNWYx0JcnVC9v7kpxf3amqXpbkw0lu6u6vb2QsAMBOM2Um63SSa6vqmqq6PMnNSU4udqiq/Uk+muSt3f2ljYwFANiJ1p3J6u4LVXVrknuS7ElyR3c/UlW3zPefSPKBJD+e5DerKkkuzC/9rTl20GcBANgyJj27sLtPJTm1qu3Ewut3Jnnn1LEAADudFd8BAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABpi0GCmwXAeO3b3sEgDYIDNZAAADCFkAAAMIWQAAA7gnC4DJpt4feO744cGVwNZnJgsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYAAhCwBgACELAGAAIQsAYIDLll0AADvPgWN3T+p37vjhwZXA8pjJAgAYYFLIqqpDVfVoVZ2tqmNr7H9pVX2qqr5TVe9Zte9cVX2uqh6sqjObVTgAwFa27uXCqtqT5LYkr02ykuR0VZ3s7s8vdPtGkl9N8sZnOMyru/trz7FWAIBtY8pM1g1Jznb3Y939VJI7kxxZ7NDdT3T36STfHVAjAMC2MyVkXZXk8YXtlXnbVJ3k3qp6oKqObqQ4AIDtasq3C2uNtt7Ae9zY3eer6oVJ7quqL3b3/U97k1kAO5ok+/fv38DhAQC2nikzWStJrl7Y3pfk/NQ36O7z899PJLkrs8uPa/W7vbsPdvfBvXv3Tj08AMCWNCVknU5ybVVdU1WXJ7k5yckpB6+qK6rqeRdfJ3ldkoefbbEAANvFupcLu/tCVd2a5J4ke5Lc0d2PVNUt8/0nqurFSc4keX6S71fVu5Ncl+TKJHdV1cX3+kh3f2LIJwEA2EImrfje3aeSnFrVdmLh9Vczu4y42jeTXP9cCgQA2I6s+A4AMIBnF8ISTX2+GwDbj5AFAwhPALhcCAAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgMfqALA0Ux9Bde744cGVwOYzkwUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwgJAFADCAkAUAMICQBQAwwKSQVVWHqurRqjpbVcfW2P/SqvpUVX2nqt6zkbEAADvRuiGrqvYkuS3JTUmuS/LmqrpuVbdvJPnVJP/mWYwFANhxpsxk3ZDkbHc/1t1PJbkzyZHFDt39RHefTvLdjY4FANiJpoSsq5I8vrC9Mm+b4rmMBQDYtqaErFqjrScef/LYqjpaVWeq6syTTz458fAAAFvTlJC1kuTqhe19Sc5PPP7ksd19e3cf7O6De/funXh4AICtaUrIOp3k2qq6pqouT3JzkpMTj/9cxgIAbFuXrdehuy9U1a1J7kmyJ8kd3f1IVd0y33+iql6c5EyS5yf5flW9O8l13f3NtcYO+iwAAFvGuiErSbr7VJJTq9pOLLz+amaXAieNhe3qwLG7l10CANuEFd8BAAaYNJMFAMs0dRb53PHDgyuB6cxkAQAMIGQBAAwgZAEADCBkAQAMIGQBAAwgZAEADGAJB4hFRgHYfGayAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABhCyAAAGELIAAAYQsgAABvCAaAB2jKkPez93/PDgSsBMFgDAEEIWAMAAQhYAwABCFgDAAEIWAMAAvl3Ijjb1m0YAsNkmzWRV1aGqerSqzlbVsTX2V1X9h/n+h6rqFQv7zlXV56rqwao6s5nFAwBsVevOZFXVniS3JXltkpUkp6vqZHd/fqHbTUmunf+8MslvzX9f9Oru/tqmVQ0AsMVNmcm6IcnZ7n6su59KcmeSI6v6HEnyOz3z6SQvqKqXbHKtAADbxpSQdVWSxxe2V+ZtU/t0knur6oGqOvpsCwUA2E6m3Phea7T1Bvrc2N3nq+qFSe6rqi929/1Pe5NZADuaJPv3759QFgDA1jVlJmslydUL2/uSnJ/ap7sv/n4iyV2ZXX58mu6+vbsPdvfBvXv3TqseAGCLmjKTdTrJtVV1TZIvJ7k5yS+t6nMyya1VdWdmN7z/aXd/paquSPIj3f1n89evS/IvN698ANi4jSzv4mHSPFvrhqzuvlBVtya5J8meJHd09yNVdct8/4kkp5K8PsnZJN9O8ivz4S9KcldVXXyvj3T3Jzb9UwAAbDGTFiPt7lOZBanFthMLrzvJu9YY91iS659jjQAA247H6gAADCBkAQAM4NmFbEueSQjAVmcmCwBgACELAGAAIQsAYAD3ZAHADzH1HlCLlrKamSwAgAGELACAAYQsAIAB3JPFlmHtK2A7c+8Wq5nJAgAYQMgCABhAyAIAGEDIAgAYwI3vAHAJuUF+9zCTBQAwgJkshrM0AwC7kZksAIABhCwAgAGELACAAdyTxbPmXisAeGZmsgAABjCTxdOYoQJYPutpbX9CFgBsY8LY1uVyIQDAAEIWAMAAQhYAwACT7smqqkNJ/n2SPUk+3N3HV+2v+f7XJ/l2kl/u7s9MGQsAbE/uB/vh1g1ZVbUnyW1JXptkJcnpqjrZ3Z9f6HZTkmvnP69M8ltJXjlxLM+RbwMCsFn8Tdk8U2aybkhytrsfS5KqujPJkSSLQelIkt/p7k7y6ap6QVW9JMmBCWN3Ff94AViGZf792a0zXlNC1lVJHl/YXslstmq9PldNHLsUwg4AbC2b/bd52aFtSsiqNdp6Yp8pY2cHqDqa5Oh881tV9eiE2qa4MsnXNulY25VzMOM8zDgPM86Dc3CR8zCz485DfehZDXs25+GvrtU4JWStJLl6YXtfkvMT+1w+YWySpLtvT3L7hHo2pKrOdPfBzT7uduIczDgPM87DjPPgHFzkPMw4DzObeR6mLOFwOsm1VXVNVV2e5OYkJ1f1OZnkbTXzM0n+tLu/MnEsAMCOs+5MVndfqKpbk9yT2TIMd3T3I1V1y3z/iSSnMlu+4WxmSzj8yg8bO+STAABsIZPWyeruU5kFqcW2EwuvO8m7po69xDb9EuQ25BzMOA8zzsOM8+AcXOQ8zDgPM5t2HmqWjwAA2EweqwMAMMCuCVlV9Z6q6qq6ctm1LENVfbCqHqqqB6vq3qr6K8uuaRmq6ter6ovzc3FXVb1g2TUtQ1X9g6p6pKq+X1W76ttEVXWoqh6tqrNVdWzZ9SxDVd1RVU9U1cPLrmWZqurqqvqjqvrC/H8Pv7bsmpahqn6sqv5XVX12fh7+xbJrWpaq2lNV/7uqPrYZx9sVIauqrs7s0T7/d9m1LNGvd/fLuvvlST6W5ANLrmdZ7kvyU939siRfSvK+JdezLA8n+XtJ7l92IZfSwqO+bkpyXZI3V9V1y61qKX47yaFlF7EFXEjyT7r7ryf5mSTv2qX/Hr6T5DXdfX2Slyc5NF8pYDf6tSRf2KyD7YqQleTfJfmneYaFUHeD7v7mwuYV2aXnorvv7e4L881PZ7Z2267T3V/o7s1a8Hc7+cFjwrr7qSQXH/W1q3T3/Um+sew6lq27v9Ldn5m//rPM/rhetdyqLr2e+dZ880fnP7vub0RV7UtyOMmHN+uYOz5kVdUbkny5uz+77FqWrar+VVU9nuQt2b0zWYvenuTjyy6CS+qZHgHGLldVB5L8zST/c8mlLMX8MtmDSZ5Icl9378bz8BuZTch8f7MOOGkJh62uqv5bkhevsev9Sf5Zktdd2oqW44edh+7+r939/iTvr6r3Jbk1yT+/pAVeIuudh3mf92d2qeD3LmVtl9KU87ALTX7UF7tHVf3lJH+Q5N2rZv13je7+XpKXz+9Tvauqfqq7d809e1X180me6O4HqurvbNZxd0TI6u6fW6u9qv5GkmuSfLaqktmloc9U1Q3d/dVLWOIl8UznYQ0fSXJ3dmjIWu88VNU/SvLzSf5u7+A1TDbw72E3mfKYMHaRqvrRzALW73X3R5ddz7J19/+rqv+R2T17uyZkJbkxyRuq6vVJfizJ86vqd7v7Hz6Xg+7oy4Xd/bnufmF3H+juA5n9B/YVOzFgraeqrl3YfEOSLy6rlmWqqkNJ3pvkDd397WXXwyXnUV/8QM3+3/d/SvKF7v63y65nWapq78VvWlfVX0ryc9llfyO6+33dvW+eFW5O8t+fa8BKdnjI4i84XlUPV9VDmV0+3ZVfVU7yH5M8L8l98+UsTqw3YCeqqjdV1UqSv5Xk7qq6Z9k1XQrzLz1cfNTXF5L8/m581FdV/eckn0ry16pqpareseyaluTGJG9N8pr5fw8enM9k7DYvSfJH878PpzO7J2tTljDY7az4DgAwgJksAIABhCwAgAGELACAAYQsAIABhCwAgAGELACAAYQsAIABhCwAgAH+P6CGRA/ntbNoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(samples.numpy(), bins=50, density=True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you're familiar with statistics, the probability of each sample can be expressed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=0.3989423>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "normal.prob(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or you can use log probability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=-0.9189385>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "normal.log_prob(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exponential distribution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another example of univariate distribution is exponential distribution. This distribution has controllable parameter called $\\lambda$, and can be expressed like this,\n",
    "\n",
    "$$ f(x; \\lambda) \\begin{cases} \\lambda e^{-\\lambda x} & x \\ge 0, \\\\ 0 & x < 0 \\end{cases} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "exponential = tfd.Exponential(rate=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5,), dtype=float32, numpy=\n",
       "array([0.23124246, 0.28650132, 0.10770323, 0.6426723 , 0.34070757],\n",
       "      dtype=float32)>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exponential.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAFlCAYAAADPim3FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAQHElEQVR4nO3dXYymd1nH8d/lro0CIsSuRratW5Py0hgquhQUXxBUWtbYmHjQohCIpGlCEY2JrCZ6wkkJaiChsGlqJUZiD6DRSlfqgW8HBNIWEGhryabUdimG1hdUPKgLlwczTYZhln2618w+z0w/n2TTve/n35mredLNd//PPfdd3R0AAM7Oty17AACA3UxMAQAMiCkAgAExBQAwIKYAAAbEFADAwP5lfePzzz+/Dx06tKxvDwCwsHvuuefx7j6w1WtLi6lDhw7l7rvvXta3BwBYWFX9y+le8zEfAMCAmAIAGBBTAAADYgoAYEBMAQAMiCkAgAExBQAwIKYAAAbEFADAgJgCABgQUwAAA2IKAGBATAEADOxf9gA77dDROxZa99ANR3Z4EgBgL7IzBQAwIKYAAAbEFADAgJgCABgQUwAAA2IKAGBATAEADIgpAIABMQUAMCCmAAAGxBQAwICYAgAYEFMAAANiCgBgQEwBAAyIKQCAATEFADAgpgAABsQUAMCAmAIAGBBTAAADYgoAYEBMAQAMiCkAgAExBQAwsFBMVdUVVfVAVZ2oqqNbvP7dVfVXVfVPVXVvVb1p+0cFAFg9Z4ypqtqX5MYkVya5NMk1VXXppmVvSXJfd1+W5JVJ/rCqztvmWQEAVs4iO1OXJznR3Q929xNJbk1y1aY1neS7qqqSPCvJvyc5ta2TAgCsoEVi6mCSRzYcn1w/t9F7k7woyaNJPpvkbd399W2ZEABghS0SU7XFud50/Jokn07yvCQ/nOS9VfXsb/pCVddW1d1Vdfdjjz32FEcFAFg9i8TUySQXbji+IGs7UBu9KcltveZEki8keeHmL9TdN3X34e4+fODAgbOdGQBgZSwSU3cluaSqLl6/qPzqJLdvWvNwklcnSVV9X5IXJHlwOwcFAFhF+8+0oLtPVdX1Se5Msi/JLd19b1Vdt/76sSTvSPKBqvps1j4WfHt3P76DcwMArIQzxlSSdPfxJMc3nTu24fePJvn57R3t3Dp09I6F1j10w5EdngQA2E3cAR0AYEBMAQAMiCkAgAExBQAwIKYAAAbEFADAgJgCABgQUwAAA2IKAGBATAEADIgpAIABMQUAMCCmAAAGxBQAwICYAgAYEFMAAANiCgBgQEwBAAyIKQCAATEFADAgpgAABsQUAMCAmAIAGBBTAAADYgoAYEBMAQAMiCkAgAExBQAwIKYAAAbEFADAgJgCABgQUwAAA2IKAGBATAEADIgpAIABMQUAMCCmAAAGxBQAwICYAgAYEFMAAANiCgBgQEwBAAyIKQCAATEFADAgpgAABsQUAMCAmAIAGBBTAAADYgoAYEBMAQAMiCkAgAExBQAwIKYAAAbEFADAwP5lD7DbHDp6x0LrHrrhyA5PAgCsAjtTAAADYgoAYEBMAQAMiCkAgAExBQAwIKYAAAbEFADAwEIxVVVXVNUDVXWiqo6eZs0rq+rTVXVvVf3D9o4JALCaznjTzqral+TGJD+X5GSSu6rq9u6+b8Oa5yR5X5IruvvhqvreHZoXAGClLLIzdXmSE939YHc/keTWJFdtWvO6JLd198NJ0t1f3t4xAQBW0yIxdTDJIxuOT66f2+j5SZ5bVX9fVfdU1Ru2+kJVdW1V3V1Vdz/22GNnNzEAwApZJKZqi3O96Xh/kh9NciTJa5L8XlU9/5v+pe6buvtwdx8+cODAUx4WAGDVLPKg45NJLtxwfEGSR7dY83h3fzXJV6vqH5NcluTz2zIlAMCKWmRn6q4kl1TVxVV1XpKrk9y+ac1fJvnJqtpfVc9I8rIk92/vqAAAq+eMO1Pdfaqqrk9yZ5J9SW7p7nur6rr114919/1V9dEkn0ny9SQ3d/fndnJwAIBVsMjHfOnu40mObzp3bNPxu5K8a/tGAwBYfe6ADgAwIKYAAAbEFADAgJgCABgQUwAAA2IKAGBATAEADIgpAIABMQUAMCCmAAAGxBQAwICYAgAYEFMAAAP7lz3AXnXo6B0LrXvohiM7PAkAsJPsTAEADIgpAIABMQUAMCCmAAAGxBQAwICYAgAYEFMAAANiCgBgQEwBAAyIKQCAATEFADAgpgAABsQUAMCAmAIAGBBTAAADYgoAYEBMAQAMiCkAgAExBQAwIKYAAAbEFADAgJgCABgQUwAAA2IKAGBATAEADIgpAIABMQUAMCCmAAAGxBQAwICYAgAYEFMAAANiCgBgQEwBAAyIKQCAATEFADAgpgAABsQUAMCAmAIAGNi/7AGe7g4dvWOhdQ/dcGSHJwEAzoadKQCAATEFADAgpgAABsQUAMCAmAIAGBBTAAADYgoAYEBMAQAMLBRTVXVFVT1QVSeq6ui3WPfSqvpaVf3y9o0IALC6zhhTVbUvyY1JrkxyaZJrqurS06x7Z5I7t3tIAIBVtcjO1OVJTnT3g939RJJbk1y1xbq3Jvlwki9v43wAACttkWfzHUzyyIbjk0letnFBVR1M8ktJXpXkpaf7QlV1bZJrk+Siiy56qrM+rXmGHwCspkV2pmqLc73p+N1J3t7dX/tWX6i7b+ruw919+MCBAwuOCACwuhbZmTqZ5MINxxckeXTTmsNJbq2qJDk/yWur6lR3/8V2DAkAsKoWiam7klxSVRcn+WKSq5O8buOC7r74yd9X1QeSfERIAQBPB2eMqe4+VVXXZ+2n9PYluaW7762q69ZfP7bDMwIArKxFdqbS3ceTHN90bsuI6u43zscCANgd3AEdAGBATAEADIgpAIABMQUAMCCmAAAGxBQAwICYAgAYEFMAAANiCgBgQEwBAAyIKQCAATEFADAgpgAABsQUAMCAmAIAGBBTAAADYgoAYEBMAQAMiCkAgAExBQAwIKYAAAbEFADAwP5lD8D2OnT0joXXPnTDkR2cBACeHuxMAQAMiCkAgAExBQAwIKYAAAbEFADAgJgCABgQUwAAA2IKAGBATAEADIgpAIABMQUAMCCmAAAGxBQAwICYAgAYEFMAAANiCgBgQEwBAAyIKQCAATEFADAgpgAABvYvewCW59DROxZa99ANR3Z4EgDYvexMAQAMiCkAgAExBQAwIKYAAAbEFADAgJgCABgQUwAAA2IKAGBATAEADLgDOmfkTukAcHp2pgAABsQUAMCAmAIAGBBTAAADYgoAYEBMAQAMiCkAgIGFYqqqrqiqB6rqRFUd3eL1X6mqz6z/+lhVXbb9owIArJ4zxlRV7UtyY5Irk1ya5JqqunTTsi8k+enufnGSdyS5absHBQBYRYvsTF2e5ER3P9jdTyS5NclVGxd098e6+z/WDz+e5ILtHRMAYDUtElMHkzyy4fjk+rnT+bUkfz0ZCgBgt1jk2Xy1xbnecmHVz2Qtpn7iNK9fm+TaJLnooosWHJHdwjP8AHg6WmRn6mSSCzccX5Dk0c2LqurFSW5OclV3/9tWX6i7b+ruw919+MCBA2czLwDASlkkpu5KcklVXVxV5yW5OsntGxdU1UVJbkvy+u7+/PaPCQCwms74MV93n6qq65PcmWRfklu6+96qum799WNJfj/J9yR5X1UlyanuPrxzYwMArIZFrplKdx9PcnzTuWMbfv/mJG/e3tEAAFafO6ADAAyIKQCAATEFADAgpgAABsQUAMCAmAIAGBBTAAADC91nCraTZ/gBsJfYmQIAGBBTAAADYgoAYEBMAQAMiCkAgAExBQAwIKYAAAbEFADAgJgCABgQUwAAA2IKAGDAs/lYWZ7hB8BuYGcKAGBATAEADIgpAIABMQUAMCCmAAAG/DQfu56f+gNgmexMAQAMiCkAgAExBQAw4JopnjYWvbYqcX0VAIuzMwUAMCCmAAAGxBQAwICYAgAYEFMAAANiCgBgQEwBAAyIKQCAATfthC14eDIAi7IzBQAwYGcKBuxgAWBnCgBgQEwBAAz4mA/OAR8HAuxddqYAAAbEFADAgJgCABgQUwAAAy5AhxXiQnWA3cfOFADAgJgCABgQUwAAA66Zgl1o0WurFuUaLICzZ2cKAGBATAEADPiYD3hKHxv6SBDgG9mZAgAYsDMFPCUufgf4RnamAAAG7EwBS+UROsBuJ6aAXUF0AatqoZiqqiuSvCfJviQ3d/cNm16v9ddfm+R/k7yxuz+5zbMCnJHoAs61M8ZUVe1LcmOSn0tyMsldVXV7d9+3YdmVSS5Z//WyJO9f/yfAStruC+mfikVDThjC7rDIztTlSU5094NJUlW3JrkqycaYuirJn3Z3J/l4VT2nqr6/u7+07RMD7HLLDLntJvhgsZg6mOSRDccn8827TlutOZhETAHssN0QZ7thxkXshijcS7cv2S2xvkhM1Rbn+izWpKquTXLt+uH/VNUDC3z/qfOTPH4Ovg/nhvdzb/F+7i17/v2sdy57gnPq/CSP74b/5nM04w+c7oVFYupkkgs3HF+Q5NGzWJPuvinJTQt8z21TVXd39+Fz+T3ZOd7PvcX7ubd4P/cW7+fiFrlp511JLqmqi6vqvCRXJ7l905rbk7yh1rw8yVdcLwUAPB2ccWequ09V1fVJ7szarRFu6e57q+q69dePJTmetdsinMjarRHetHMjAwCsjoXuM9Xdx7MWTBvPHdvw+07ylu0dbduc048V2XHez73F+7m3eD/3Fu/ngmqtgwAAOBsedAwAMLBnY6qqrqiqB6rqRFUdXfY8nL2qurCq/q6q7q+qe6vqbcueibmq2ldVn6qqjyx7FmbWb9T8oar65/X/T39s2TNx9qrqN9f/rP1cVf15VX3HsmdadXsypjY8AufKJJcmuaaqLl3uVAycSvJb3f2iJC9P8hbv557wtiT3L3sItsV7kny0u1+Y5LJ4X3etqjqY5NeTHO7uH8raD55dvdypVt+ejKlseAROdz+R5MlH4LALdfeXnnxwdnf/d9b+oD643KmYqKoLkhxJcvOyZ2Gmqp6d5KeS/HGSdPcT3f2fSx2Kqf1JvrOq9id5Rra4byTfaK/G1Okeb8MuV1WHkrwkySeWPAoz707y20m+vuQ5mPvBJI8l+ZP1j21vrqpnLnsozk53fzHJHyR5OGuPhPtKd//NcqdafXs1phZ6vA27S1U9K8mHk/xGd//Xsufh7FTVLyT5cnffs+xZ2Bb7k/xIkvd390uSfDWJ61R3qap6btY+ybk4yfOSPLOqfnW5U62+vRpTCz3eht2jqr49ayH1we6+bdnzMPKKJL9YVQ9l7SP4V1XVny13JAZOJjnZ3U/uFn8oa3HF7vSzSb7Q3Y919/8luS3Jjy95ppW3V2NqkUfgsEtUVWXteoz7u/uPlj0PM939O919QXcfytr/m3/b3f7mu0t1978meaSqXrB+6tVJ7lviSMw8nOTlVfWM9T97Xx0/UHBGC90Bfbc53SNwljwWZ+8VSV6f5LNV9en1c7+7fmd+YPnemuSD6395fTAeKbZrdfcnqupDST6ZtZ+k/lTcCf2M3AEdAGBgr37MBwBwTogpAIABMQUAMCCmAAAGxBQAwICYAgAYEFMAAANiCgBg4P8Bg1IkTIOXVt8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(exponential.sample(10000).numpy(), bins=50, density=True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bernoulli Distribution\n",
    "\n",
    "Bernoulli Distribution is also a family of univariate distribution. All we need to describe this distribution is the probabiltiy that 1 is occurred. Otherwise, 0 will be occurred.\n",
    "\n",
    "$$ f(x; p) = \\begin{cases} p & \\text{if } k=1, \\\\ q = 1 - p & \\text{if } k = 0 \\end{cases} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "bernoulli = tfd.Bernoulli(probs=0.8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5,), dtype=int32, numpy=array([1, 0, 1, 1, 1])>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bernoulli.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This distribution generates only two data, 0 and 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Probability result 0.20000000298023224 for k = 0\n",
      "Probability result 0.4000000059604645 for k = 0.5\n",
      "Probability result 0.800000011920929 for k = 1\n",
      "Probability result 0.05000000074505806 for k = -1\n"
     ]
    }
   ],
   "source": [
    "for k in [0, 0.5, 1, -1]:\n",
    "    print('Probability result {} for k = {}'.format(bernoulli.prob(k), k))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We already define the probability of 1 to 0.8, so the probability of 0 will be 0.2. You can see that the probability of unexpected data will be strange probability."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Batch Distributions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The advantage of tensorflow distribution is that it can easily make batch data from specific distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "bernoulli_batch = tfd.Bernoulli(probs=[0.1, 0.25, 0.5, 0.75, 0.9])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tfp.distributions.Bernoulli 'Bernoulli' batch_shape=[5] event_shape=[] dtype=int32>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bernoulli_batch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5, 5), dtype=int32, numpy=\n",
       "array([[0, 1, 1, 1, 1],\n",
       "       [0, 0, 0, 1, 1],\n",
       "       [0, 1, 1, 0, 1],\n",
       "       [0, 0, 1, 1, 1],\n",
       "       [0, 0, 1, 0, 1]])>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bernoulli_batch.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can make 2D batch samples using higher rank as probs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tfp.distributions.Bernoulli 'Bernoulli' batch_shape=[1, 3, 2] event_shape=[] dtype=int32>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "probs = [[[0.5, 0.5], \n",
    "          [0.8, 0.3], \n",
    "          [0.25, 0.75]]]\n",
    "bernoulli_batch_2D = tfd.Bernoulli(probs=probs)\n",
    "bernoulli_batch_2D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5, 1, 3, 2), dtype=int32, numpy=\n",
       "array([[[[1, 1],\n",
       "         [1, 0],\n",
       "         [0, 1]]],\n",
       "\n",
       "\n",
       "       [[[1, 1],\n",
       "         [0, 1],\n",
       "         [0, 1]]],\n",
       "\n",
       "\n",
       "       [[[0, 1],\n",
       "         [1, 0],\n",
       "         [0, 0]]],\n",
       "\n",
       "\n",
       "       [[[1, 1],\n",
       "         [1, 0],\n",
       "         [0, 0]]],\n",
       "\n",
       "\n",
       "       [[[1, 0],\n",
       "         [1, 0],\n",
       "         [0, 1]]]])>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bernoulli_batch_2D.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(1, 3, 2), dtype=float32, numpy=\n",
       "array([[[0.5 , 0.5 ],\n",
       "        [0.2 , 0.7 ],\n",
       "        [0.25, 0.75]]], dtype=float32)>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bernoulli_batch_2D.prob([[[1, 0], \n",
    "                         [0, 0], \n",
    "                         [1, 1]]])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}