notebooks/ProbabilisticReasoning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Probabilistic Reasoning\n",
    "\n",
    "This notebook serves as the supporting material for chapter 14**Probabilistic Reasoning**. In this notebook, we will learn how to use the code respository to build network models to reason under uncertainty according to the laws of probability theory. In the previous notebook, we briefly explained what Bayes' Rule is and how it can be utilised in Probabilistic Inference. This notebook introduces a systematic way to represent such conditional relationships in the form of **Bayesian Networks**. We will also have a look at a variety of approximate inference algorithms and will also explore ways in which probability theory can be applied to worlds with objects and relations(worlds represented in first order logic).\n",
    "\n",
    "## Representing knowledge in an uncertain domain\n",
    "\n",
    "We saw in the previous notebooks that the full joint probability distribution models can answer any question about the domain but they are computationally expensive as the space complexity grows exponentially. However, we saw that independence and conditional independence relationships among variables can be of great help in defining a full joint distribution. Owing to these shortcomings of full joint distributions and to the merits of conditional relations between random variables, AI researchers have come up with a clever data structure called Bayesian Networks. Bayesian networks can represent essentially any full joint probability distribution and in many cases can do so very concisely.\n",
    "\n",
    "A Bayesian network is a directed graph in which each node is annotated with quantitative probability information. The full specification is as follows:\n",
    "* Each node corresponds to a random variable, which may be discrete or continuous.\n",
    "* A set of directed links or arrows connects pairs of nodes. If there is an arrow from node X to node Y , X is said to be a parent of Y. The graph has no directed cycles (and hence is a directed acyclic graph, or DAG).\n",
    "* Each node $X_i$ has a conditional probability distribution $P(X_i \\mid Parents(X_i ))$ that quantifies the effect of the parents on the node.\n",
    "\n",
    "The topology of the network—the set of nodes and links—specifies the conditional independence relationships that hold in the domain. The\n",
    "intuitive meaning of an arrow is typically that X has a direct influence on Y, which suggests\n",
    "that causes should be parents of effects. It is usually easy for a domain expert to decide what\n",
    "direct influences exist in the domain—much easier, in fact, than actually specifying the probabilities themselves. Once the topology of the Bayesian network is laid out, we need only\n",
    "specify a conditional probability distribution for each variable, given its parents.\n",
    "\n",
    "The Bayesian Networks possess many interesting properties which are quite brilliantly explained in the text. Readers are advised to go through the text to get a feel for the Bayesian Networks. In this notebook, our main focus will be to utilise Bayesian Networks to model various problems.\n",
    "\n",
    "To work with the Bayesian Networks let us first load the aima jar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "05dc6698-2418-49a5-89a2-606bc131197f",
       "version_major": 2,
       "version_minor": 0
      },
      "method": "display_data"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%classpath add jar ../out/artifacts/aima_core_jar/aima-core.jar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The tooth cavity catch structure\n",
    "\n",
    "Consider the simple world described in the text, consisting of variables *Toothache, Cavity, Catch* and *Weather*. It is easy to see that *Weather* is independent of other variables. Furthermore it can be argued that *Toothache* and *Catch* are conditionally independent given *Cavity*. These relationships can be represented by a Bayesian Network structure shown below. Formally, the conditional independence of *Toothache* and *Catch*, given\n",
    "Cavity, is indicated by the absence of a link between *Toothache* and *Catch*. Intuitively, the\n",
    "network represents the fact that *Cavity* is a direct cause of *Toothache* and *Catch*, whereas\n",
    "no direct causal relationship exists between Toothache and Catch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## APIs from the code repository"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a look at the structure of APIs from the code repository. We will understand the APIs by considering the three defining points of the Bayesian Networks.\n",
    "\n",
    "**Each node corresponds to a random variable, which may be discrete or continuous.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `Node` interface represents the node in a Bayesian Network. Given below is a description of the Node interface.\n",
    "````java\n",
    "public interface Node {\n",
    "\n",
    "\t/**\n",
    "\t * \n",
    "\t * @return the Random Variable this Node is for/on.\n",
    "\t */\n",
    "\tRandomVariable getRandomVariable();\n",
    "\n",
    "\t/**\n",
    "\t * \n",
    "\t * @return true if this Node has no parents.\n",
    "\t * \n",
    "\t * @see Node#getParents()\n",
    "\t */\n",
    "\tboolean isRoot();\n",
    "\n",
    "\t/**\n",
    "\t * \n",
    "\t * @return the parent Nodes for this Node.\n",
    "\t */\n",
    "\tSet<Node> getParents();\n",
    "\n",
    "\t/**\n",
    "\t * \n",
    "\t * @return the children Nodes for this Node.\n",
    "\t */\n",
    "\tSet<Node> getChildren();\n",
    "\n",
    "\t/**\n",
    "\t * Get this Node's Markov Blanket:<br>\n",
    "\t * 'A node is conditionally independent of all other nodes in the network,\n",
    "\t * given its parents, children, and children's parents - that is, given its\n",
    "\t * <b>MARKOV BLANKET</b> (AIMA3e pg, 517).\n",
    "\t * \n",
    "\t * @return this Node's Markov Blanket.\n",
    "\t */\n",
    "\tSet<Node> getMarkovBlanket();\n",
    "\n",
    "\t/**\n",
    "\t * \n",
    "\t * @return the Conditional Probability Distribution associated with this\n",
    "\t *         Node.\n",
    "\t */\n",
    "\tConditionalProbabilityDistribution getCPD();\n",
    "````"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This interface can be implemented to obtain customised nodes. A default implementation is provided in the repository via the `FullCPTNode` class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second specification of the Bayesian Networks tells about the hierarchy of the network.\n",
    "\n",
    "**A set of directed links or arrows connects pairs of nodes. If there is an arrow from node X to node Y , X is said to be a parent of Y. The graph has no directed cycles (and hence is a directed acyclic graph, or DAG).**\n",
    "\n",
    "These links are stored as a set and can be obtained by the `getParents()` method of the `Node` interface."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The third specification states the data contained in each node.\n",
    "\n",
    "**Each node $X_i$ has a conditional probability distribution $P(X_i \\mid Parents(X_i ))$ that quantifies the effect of the parents on the node.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This information is stored in the form of a ConditionalProbabilityDistribution inside a node. After we have defined the nodes for our network, we can use the `BayesNet` class from the repository and then construct a Bayesian Network. Let's work with the ToothAcheCavityAndCatch example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random Variables = [Cavity, Toothache, Catch]\n",
      "The cavity Node: Cavity\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "aima.core.probability.bayes.impl.BayesNet@3e549709"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "package aima.notebooks.probabilisticreasoning;\n",
    "\n",
    "import aima.core.probability.bayes.*;\n",
    "import aima.core.probability.*;\n",
    "import aima.core.probability.bayes.impl.*;\n",
    "import aima.core.probability.util.*;\n",
    "import aima.core.probability.domain.*;\n",
    "\n",
    "// First let us define the Random Variables which make up our Bayes Network\n",
    "RandVar cavityRv = new RandVar(\"Cavity\", new BooleanDomain());\n",
    "RandVar toothacheRv = new RandVar(\"Toothache\", new BooleanDomain());\n",
    "RandVar catchRv = new RandVar(\"Catch\", new BooleanDomain());\n",
    "\n",
    "// Now we will define the nodes that make up the network and represent the above network\n",
    "// the order of the doubles in CPT is as follows\n",
    "// If A,B and C are the three random variables then first the possibilities of C will be exhausted, then B and then A.\n",
    "// For example if A, B and C are Boolean random variables then the doubles will be mentioned in the given order\n",
    "//    A    B    C\n",
    "//    1    1    1\n",
    "//    1    1    0\n",
    "//    1    0    1\n",
    "//    1    0    0\n",
    "//    0    1    1\n",
    "//    0    1    0\n",
    "//    0    0    1\n",
    "//    0    0    0\n",
    "FullCPTNode cavity = new FullCPTNode(cavityRv, new double[] {\n",
    "                                    // True\t\t\t\n",
    "                                    0.2,\n",
    "                                    // False\n",
    "                                    0.8 });\n",
    "FullCPTNode toothache = new FullCPTNode(toothacheRv,\n",
    "\t\t\t\tnew double[] {\n",
    "\t\t\t\t\t\t// C=true, T=true\n",
    "\t\t\t\t\t\t0.6,\n",
    "\t\t\t\t\t\t// C=true, T=false\n",
    "\t\t\t\t\t\t0.4,\n",
    "\t\t\t\t\t\t// C=false, T=true\n",
    "\t\t\t\t\t\t0.1,\n",
    "\t\t\t\t\t\t// C=false, T=false\n",
    "\t\t\t\t\t\t0.9\n",
    "\n",
    "\t\t\t\t}, cavity);\n",
    "\n",
    "FiniteNode catchNode = new FullCPTNode(catchRv, new double[] {\n",
    "\t\t\t\t// C=true, Catch=true\n",
    "\t\t\t\t0.9,\n",
    "\t\t\t\t// C=true, Catch=false\n",
    "\t\t\t\t0.1,\n",
    "\t\t\t\t// C=false, Catch=true\n",
    "\t\t\t\t0.2,\n",
    "\t\t\t\t// C=false, Catch=false\n",
    "\t\t\t\t0.8 }, cavity);\n",
    "\n",
    "// Now let us consider the Bayesian network\n",
    "// We need to specify only the root nodes from the nerwork\n",
    "BayesNet cavityBayesNet = new BayesNet(cavity);\n",
    "\n",
    "// Now let's extract whatever we can from the BayesNet\n",
    "System.out.println(\"Random Variables = \"+ cavityBayesNet.getVariablesInTopologicalOrder().toString());\n",
    "System.out.println(\"The cavity Node: \"+ cavityBayesNet.getNode(cavityRv).toString());\n",
    "return cavityBayesNet;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above block describes how to construct a Bayesian Network. However, a Bayesian Network itself is of little use. Hence,we need inference algorithms which can extract information from the network. Before, introducing various inference algorithms, we must note that a **Bayesian Network** is capable of describing a **Full Joint Distribution** by itself. This can be shown by considering the fact that a generic entry in the joint distribution is the probability of a conjunction of particular\n",
    "assignments to each variable, such as $P (X_1 = x_1 \\land . . . \\land X_n = x_n )$. We use the notation\n",
    "$P (x_1 , . . . , x_n )$ as an abbreviation for this. Now, in terms of conditional probability, using the product rule\n",
    "\n",
    "$$P (x_1 , . . . , x_n ) = P (x_n | x_{n−1} , . . . , x_1 )P (x_{n−1} , . . . , x_1 )$$\n",
    "\n",
    "Then we repeat the process, reducing each conjunctive probability to a conditional probability\n",
    "and a smaller conjunction. We end up with one big product:\n",
    "$$P (x_1 , . . . , x_n ) = P (x_n | x_{n−1} , . . . , x_1 )P (x_{n−1} | x_{n−2} , . . . , x_1 ) · · · P (x_2 | x_1 )P (x_1 )$$\n",
    "$$ = \\prod_{i = 1}^{n}P(x_i|x_{i-1},...,x_1)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This identity is called the chain rule. It holds for any set of random variables. We can conclude that the specification of the joint distribution is equivalent to the\n",
    "general assertion that, for every variable $X_i$ in the network\n",
    "\n",
    "$$\\textbf{P}(X_i | X_{i−1} , . . . , X_1 ) = \\textbf{P}(X_i | Parents(X_i ))$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "provided that $Parents(X_i) \\subseteq {X_{i−1} , . . . , X_1 }$. This last condition is satisfied by numbering\n",
    "the nodes in a way that is consistent with the partial order implicit in the graph structure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we can represent the full joint distribution of a particular system using Bayesian Networks, therefore we can consider the BayesianNetworks as Probability Models and can use them for inference procedures in the same way as we used the `FullJointProbabilityModel`. This model can be created using the `FiniteBayesModel` class from the repository. this class asks for a Bayesian Network and an Inference Procedure. If no inference procedure is mentioned, the `EnumerationAsk` algorithm is used as the default inference procedure. From now on, I will be using the `BayesNetExampleFactory` to creste the ToothAcheCavityCatch example. Let's have a look at how we can manipulate the `BayesNetModel` to perform inference using uncertain knowledge."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The random variables in the model = [Cavity, Toothache, Catch]\n",
      "The prior probability of having a toothache = 0.2\n",
      "The prior probability of having a cavity = 0.2\n",
      "The probability of having a cavity and toothache simultaneously is = 0.12\n",
      "The probability of having a toothache given that the person has a cavity(causal direction) is = 0.6\n",
      "The probability of having a cavity given that the person is experiencing toothache(diagnostic direction) is = 0.6\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "null"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "package aima.notebooks.probabilisticreasoning;\n",
    "\n",
    "import aima.core.probability.example.*;\n",
    "import aima.core.probability.bayes.*;\n",
    "import aima.core.probability.bayes.impl.*;\n",
    "import aima.core.probability.bayes.impl.*;\n",
    "import aima.core.probability.bayes.model.*;\n",
    "import aima.core.probability.proposition.*;\n",
    "\n",
    "// Load the network from the network factory.\n",
    "BayesianNetwork cavityNet = BayesNetExampleFactory.constructToothacheCavityCatchNetwork();\n",
    "// Construct the BayesModel from the BayesNet\n",
    "// We have not passed any inference procedure. Hence, the default inference procedure will be used.\n",
    "FiniteBayesModel model = new FiniteBayesModel(cavityNet);\n",
    "// Now we are ready to answer all sorts of questions.\n",
    "\n",
    "// Let's define a few assignments\n",
    "AssignmentProposition toothache = new AssignmentProposition(ExampleRV.TOOTHACHE_RV,true);\n",
    "AssignmentProposition cavity = new AssignmentProposition(ExampleRV.CAVITY_RV,true);\n",
    "// Now let's have a look at what we can do with the model.\n",
    "// To print the random variables in the model\n",
    "System.out.println(\"The random variables in the model = \" + model.getRepresentation());\n",
    "// We can calculate the prior probabilities of a variety of combinations of random variables\n",
    "System.out.println(\"The prior probability of having a toothache = \"+ model.prior(toothache));\n",
    "System.out.println(\"The prior probability of having a cavity = \"+ model.prior(cavity));\n",
    "System.out.println(\"The probability of having a cavity and toothache simultaneously is = \"+ model.prior(toothache, cavity));\n",
    "// We can also calculate a variety of posterior probabilities from the model as follows\n",
    "System.out.println(\"The probability of having a toothache given that the person has a cavity(causal direction) is = \"+ \n",
    "                  model.posterior(toothache,cavity));\n",
    "System.out.println(\"The probability of having a cavity given that the person is experiencing toothache(diagnostic direction) is = \"\n",
    "                  + model.posterior(cavity,toothache));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exact Inference in Bayesian Networks\n",
    "The basic task for any probabilistic inference system is to compute the posterior probability\n",
    "distribution for a set of query variables, given some observed event—that is, some assignment of values to a set of evidence variables.We will use the notation from the previous notebook: X denotes the query variable; **E**\n",
    "denotes the set of evidence variables $E_1 , . . . , E_m$ , and **e** is a particular observed event; Y will\n",
    "denote the nonevidence, nonquery variables $Y_1 , . . . , Y_l$ (called the hidden variables). Thus,\n",
    "the complete set of variables is $X = \\{X\\} \\cup E \\cup Y$. A typical query asks for the posterior\n",
    "probability distribution $P(X | \\textbf{e})$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inference by enumeration\n",
    "We proved in the previous notebook that any conditional probability can be computed by using the full joint distribution. Mathematically:\n",
    "$$ \\textbf{P}(X|\\textbf{e}) = \\alpha \\textbf{P}(X,\\textbf{e}) = \\alpha \\sum_{y}\\textbf{P}(X|\\textbf{e},\\textbf{y})$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, we know that a Bayesian Network can give a complete representation of a full joint distribution. Hence, the above sum is calculable from a Bayesian Network. Now, let's have a look at the entire process. Let b, j and m be particular values of random variables *B*, *J* and *M*. Let *E* and *A* be the hidden variables. Then by using the sum and product rules of probability we get:\n",
    "$$P(b|j,m) = \\alpha P(b) \\sum_{e}P(e)\\sum_{a}P(a|b,e)P(j|a)P(m|a)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, the above calculation can be represented in the form of a calculation tree shown below. The order of calculation brings with itself an intuition of a depth first tree. In fact, the enumeration ask algorithm employs a depth first approach to solve the inference problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "### AIMA3e\n",
       "__function__ ENUMERATION-ASK(_X_, __e__, _bn_) __returns__ a distribution over _X_  \n",
       "&emsp;__inputs__: _X_, the query variable  \n",
       "&emsp;&emsp;&emsp;&emsp;&emsp;__e__, observed values for variables __E__  \n",
       "&emsp;&emsp;&emsp;&emsp;&emsp;_bn_, a Bayes net with variables \\{_X_\\} &Union; __E__ &Union; __Y__ /\\* __Y__ = hidden variables \\*/  \n",
       "\n",
       "&emsp;__Q__(_X_) &larr; a distribution over _X_, initially empty  \n",
       "&emsp;__for each__ value _x<sub>i</sub>_ of _X_ __do__  \n",
       "&emsp;&emsp;&emsp;__Q__(_x<sub>i</sub>_) &larr; ENUMERATE\\-ALL(_bn_.VARS, __e__<sub>_x_<sub>_i_</sub></sub>)  \n",
       "&emsp;&emsp;&emsp;&emsp;&emsp;where __e__<sub>_x_<sub>_i_</sub></sub> is __e__ extended with _X_ = _x<sub>i</sub>_  \n",
       "&emsp;__return__ NORMALIZE(__Q__(_X_))  \n",
       "\n",
       "---\n",
       "__function__ ENUMERATE\\-ALL(_vars_, __e__) __returns__ a real number  \n",
       "&emsp;__if__ EMPTY?(_vars_) __then return__ 1.0  \n",
       "&emsp;_Y_ &larr; FIRST(_vars_)  \n",
       "&emsp;__if__ _Y_ has value _y_ in __e__  \n",
       "&emsp;&emsp;&emsp;__then return__ _P_(_y_ &vert; _parents_(_Y_)) &times; ENUMERATE\\-ALL(REST(_vars_), __e__)  \n",
       "&emsp;&emsp;&emsp;__else return__ &sum;<sub>_y_</sub> _P_(_y_ &vert; _parents_(_Y_)) &times; ENUMERATE\\-ALL(REST(_vars_), __e__<sub>_y_</sub>)  \n",
       "&emsp;&emsp;&emsp;&emsp;&emsp;where __e__<sub>_y_</sub> is __e__ extended with _Y_ = _y_  \n",
       "\n",
       "---\n",
       "__Figure__ ?? The enumeration algorithm for answering queries on Bayesian networks."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%python\n",
    "from notebookUtils import *\n",
    "pseudocode('Enumeration Ask')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above algorithm calculates the desired conditional distribution. It is implemented in the [`EnumerationAsk`](/aima-core/src/main/java/aima/core/probability/bayes/exact/EnumerationAsk.java) class in the repository. The algorithm takes as input a query variable, a few evidence variables and a Bayesian Network. However, to use the algorithm we will not directly call the algorithm. Instead, we will pass the `EnumerationAsk` as a parameter in the form of an Inference Procedure to our `BayesNetModel`. The cell below shows the steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The prior distribution for toothache is <0.2, 0.8>\n",
      "The prior distribution for cavity is <0.2, 0.8>\n",
      "The prior distribution for catch is <0.34, 0.66>\n",
      "The posterior distribution for toothache given cavity is \n",
      " \t <0.6, 0.10000000000000002, 0.4000000000000001, 0.9>\n",
      "The posterior distribution for catch given cavity is \n",
      " \t <0.9, 0.19999999999999998, 0.09999999999999999, 0.7999999999999999>\n",
      "The prior probability of having a cavity is 0.2\n",
      "The posterior probability of having a cavity given a toothache is 0.6\n",
      "The prior probability of having a cavity or a toothache is 0.28\n",
      "The posterior probability of not having a cavity given a toothache is 0.4000000000000001\n",
      "The prior probability of not having a cavity but having a toothache is 0.08000000000000002\n",
      "The prior probability of having a cavity or a toothache is 0.28\n",
      "The posterior probability of having a cavity given that the patient has a cavity or a toothache is  0.7142857142857143\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "null"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "package aima.notebooks.probabilisticreasoning;\n",
    "\n",
    "import aima.core.probability.example.*;\n",
    "import aima.core.probability.bayes.*;\n",
    "import aima.core.probability.bayes.exact.*;\n",
    "import aima.core.probability.bayes.impl.*;\n",
    "import aima.core.probability.bayes.impl.*;\n",
    "import aima.core.probability.bayes.model.*;\n",
    "import aima.core.probability.proposition.*;\n",
    "\n",
    "// Load the network from the network factory.\n",
    "BayesianNetwork cavityNet = BayesNetExampleFactory.constructToothacheCavityCatchNetwork();\n",
    "// Construct the BayesModel from the BayesNet\n",
    "// We will pass EnumerationAsk as the new inference procedure\n",
    "FiniteBayesModel model = new FiniteBayesModel(cavityNet, new EnumerationAsk());\n",
    "\n",
    "// Now we will fully exhaust this model to extract as much information as we can\n",
    "\n",
    "// First let us define some assignment propositions\n",
    "AssignmentProposition atoothache = new AssignmentProposition(\n",
    "\t\t\t\tExampleRV.TOOTHACHE_RV, true);\n",
    "\t\tAssignmentProposition anottoothache = new AssignmentProposition(\n",
    "\t\t\t\tExampleRV.TOOTHACHE_RV, false);\n",
    "\t\tAssignmentProposition acavity = new AssignmentProposition(\n",
    "\t\t\t\tExampleRV.CAVITY_RV, true);\n",
    "\t\tAssignmentProposition anotcavity = new AssignmentProposition(\n",
    "\t\t\t\tExampleRV.CAVITY_RV, false);\n",
    "\t\tAssignmentProposition acatch = new AssignmentProposition(\n",
    "\t\t\t\tExampleRV.CATCH_RV, true);\n",
    "\t\tAssignmentProposition anotcatch = new AssignmentProposition(\n",
    "\t\t\t\tExampleRV.CATCH_RV, false);\n",
    "\n",
    "// Now let us define some propositions which are conjunctions and/or disjunctions of the above propositions\n",
    "ConjunctiveProposition toothacheAndNotCavity = new ConjunctiveProposition(\n",
    "\t\t\t\tatoothache, anotcavity);\n",
    "DisjunctiveProposition cavityOrToothache = new DisjunctiveProposition(\n",
    "\t\t\t\tacavity, atoothache);\n",
    "\n",
    "// First let us calculate the prior probabilities of our random variables\n",
    "// The probabilities in the distribution are returned in the order <True, False>\n",
    "System.out.println(\"The prior distribution for toothache is \"+ model.priorDistribution(ExampleRV.TOOTHACHE_RV));\n",
    "System.out.println(\"The prior distribution for cavity is \"+ model.priorDistribution(ExampleRV.CAVITY_RV));\n",
    "System.out.println(\"The prior distribution for catch is \"+ model.priorDistribution(ExampleRV.CATCH_RV));\n",
    "// Now let us calculate the posterior distribution is\n",
    "// Posterior distribution first exhausts all the possibilities of the evidence variables\n",
    "System.out.println(\"The posterior distribution for toothache given cavity is \\n \\t \"+ model.posteriorDistribution(ExampleRV.TOOTHACHE_RV,\n",
    "                                                                                                           ExampleRV.CAVITY_RV).toString());\n",
    "\n",
    "System.out.println(\"The posterior distribution for catch given cavity is \\n \\t \"+ model.posteriorDistribution(ExampleRV.CATCH_RV,\n",
    "                                                                                                           ExampleRV.CAVITY_RV).toString());\n",
    "\n",
    "// Now let us have a look at some individual probabilities\n",
    "System.out.println(\"The prior probability of having a cavity is \"+model.prior(acavity));\n",
    "System.out.println(\"The posterior probability of having a cavity given a toothache is \"+ model.posterior(acavity, atoothache));\n",
    "System.out.println(\"The prior probability of having a cavity or a toothache is \"+model.prior(cavityOrToothache));\n",
    "System.out.println(\"The posterior probability of not having a cavity given a toothache is \"+model.posterior(anotcavity, atoothache));\n",
    "System.out.println(\"The prior probability of not having a cavity but having a toothache is \"+model.prior(toothacheAndNotCavity));\n",
    "System.out.println(\"The prior probability of having a cavity or a toothache is \"+model.prior(cavityOrToothache));\n",
    "System.out.println(\"The posterior probability of having a cavity given that the patient has a cavity or a toothache is  \"+\n",
    "                          model.posterior(acavity,cavityOrToothache));\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a large number of inferences that can be derived from a probability model. For the sake of conciseness, we will focus only on a few prior and posterior distributions in the upcoming examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Java",
   "language": "java",
   "name": "java"
  },
  "language_info": {
   "codemirror_mode": "text/x-java",
   "file_extension": ".java",
   "mimetype": "",
   "name": "Java",
   "nbconverter_exporter": "",
   "version": "1.8.0_161"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": false,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}