Bayes_learn_code_1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 计算Bayes\n",
    "\n",
    "请先阅读<http://blog.genesino.com/2016/09/bayes/>。\n",
    "\n",
    "代码为学习《Think Bayes》的记录，为了方便理解，我对代码做了解构，力求从0开始，逐步实现整个类的构建。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1 定义一个新的类 Bayes\n",
    "\n",
    "这个类实际上是一个字典，以hypo-prob为键值对。对象的初始化参数可以是一个含所有假设的列表，如`['H1','H2','H3]`, 初始化之后，所有的假设的起始概率相等； 另外也可以用一个字典或其它Bayes类初始化，如`{'H1':1, 'H2':5, 'H3':4}`。\n",
    "\n",
    "同时内建`Set`函数用于设置单个假设的概率值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class Bayes(object):\n",
    "    \"\"\"A bayes class, mainly a dictionary\"\"\"\n",
    "    def __init__(self, hypos=None, name=''):\n",
    "        \"\"\"\n",
    "        Initialize the distribution.\n",
    "        \n",
    "        hypos: sequence of hypotheses\n",
    "        \"\"\"\n",
    "        self.name = name\n",
    "        self.pmf = {}\n",
    "        if hypos is None:\n",
    "            return\n",
    "        \n",
    "        # Initiate the class object\n",
    "        # Three initalize methods are used to deal with different types of input\n",
    "        # \n",
    "        init_methods = [\n",
    "            self.InitPmf,\n",
    "            self.InitMapping,  #A dict\n",
    "            self.InitSequence, #equal probability for all hypos\n",
    "            self.InitFailure,\n",
    "        ]\n",
    "        \n",
    "        for method in init_methods:\n",
    "            try:\n",
    "                method(hypos)\n",
    "                break\n",
    "            except AttributeError:\n",
    "                continue\n",
    "        \n",
    "        if len(self):\n",
    "            self.Normalize()\n",
    "    \n",
    "    def __str__(self):\n",
    "        '''\n",
    "        Stringlize self.pmf\n",
    "        '''\n",
    "        tmpL = [\"Probability table\"]\n",
    "        for hypo, prob in sorted(self.pmf.iteritems()):\n",
    "            tmpL.append('\\t'.join([str(hypo), str(prob)]))\n",
    "        return '\\n'.join(tmpL)\n",
    "    \n",
    "    def InitSequence(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a sequence of hypos with equal probabilities.\n",
    "        \n",
    "        hypos: ['H1','H2','H3',...]\n",
    "        \"\"\"\n",
    "        for hypo in hypos:\n",
    "            self.Set(hypo, 1)\n",
    "    \n",
    "    def InitMapping(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a map from value to probablity (a dict).\n",
    "        \n",
    "        hypos = {'H1':1, 'H2':5, 'H3':4}\n",
    "        \"\"\"\n",
    "        for hypo, prob in hypos.iteritems():\n",
    "            self.Set(hypo, prob)\n",
    "    \n",
    "    def InitPmf(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a Bayes object.\n",
    "        \n",
    "        hypos = Bayes()\n",
    "        \"\"\"\n",
    "        for hypo, prob in hypos.iteritems():\n",
    "            self.Set(hypo, prob)\n",
    "    \n",
    "    def InitFailure(self, hypos):\n",
    "        \"\"\"Raise an errot\"\"\"\n",
    "        raise ValueError(\"None of the initialization methods works.\")\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.pmf)\n",
    "    \n",
    "    def Set(self, hypo, prob=0):\n",
    "        \"\"\"\n",
    "        Set hypo-prob pair\n",
    "        \"\"\"\n",
    "        self.pmf[hypo] = prob\n",
    "    \n",
    "    def Print(self):\n",
    "        \"\"\"Print the values and freqs in asending order.\"\"\"\n",
    "        for hypo, prob in sorted(self.pmf.iteritems()):\n",
    "            print hypo, prob\n",
    "    \n",
    "    def Normalize(self):\n",
    "        \"\"\"\n",
    "        Normalize probability\n",
    "        \"\"\"\n",
    "        total = float(sum(self.pmf.values()))\n",
    "        if total == 0.0:\n",
    "            raise ValueError('total probability is zero.')\n",
    "            logging.warning('Normalize: total probability is zero.')\n",
    "            return total\n",
    "        \n",
    "        factor = 1 / total\n",
    "        \n",
    "        for hypo in self.pmf:\n",
    "            self.pmf[hypo] *= factor\n",
    "            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2 初始化Bayes类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1 使用假设列表初始化Bayes类\n",
    "\n",
    "假如有一个质地均匀的筛子，其初始概率初始化如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Probability table\n",
      "1\t0.166666666667\n",
      "2\t0.166666666667\n",
      "3\t0.166666666667\n",
      "4\t0.166666666667\n",
      "5\t0.166666666667\n",
      "6\t0.166666666667\n"
     ]
    }
   ],
   "source": [
    "hypos = [1,2,3,4,5,6]\n",
    "\n",
    "die = Bayes(hypos)\n",
    "\n",
    "#die.Print()\n",
    "print die"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before normalize\n",
      "1 1\n",
      "2 1\n",
      "3 1\n",
      "4 1\n",
      "\n",
      "After normalize\n",
      "Probability table\n",
      "1\t0.25\n",
      "2\t0.25\n",
      "3\t0.25\n",
      "4\t0.25\n"
     ]
    }
   ],
   "source": [
    "hypos = [1,2,3,4]\n",
    "\n",
    "die2 = Bayes()\n",
    "\n",
    "for hypo in hypos:\n",
    "    die2.Set(hypo, 1)\n",
    "\n",
    "print \"Before normalize\"\n",
    "die2.Print()\n",
    "\n",
    "die2.Normalize()\n",
    "print \"\\nAfter normalize\"\n",
    "print die2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 使用假设-概率字典初始化Bayes类\n",
    "\n",
    "假如有一个质地不均匀的筛子，其初始概率初始化如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Probability table\n",
      "1\t0.1\n",
      "2\t0.2\n",
      "3\t0.3\n",
      "4\t0.4\n"
     ]
    }
   ],
   "source": [
    "hypos = {1:0.1,2:0.2,3:0.3,4:0.4}\n",
    "die3 = Bayes(hypos)\n",
    "print die3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Probability table\n",
      "1\t0.1\n",
      "2\t0.2\n",
      "3\t0.3\n",
      "4\t0.4\n"
     ]
    }
   ],
   "source": [
    "hypos = {1:1,2:2,3:3,4:4}\n",
    "die3 = Bayes(hypos)\n",
    "print die3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 利用Bayes类解决我们在<http://blog.genesino.com/2016/09/bayes/>中提到的问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在开始之前，我们需要在Bayes类中再定义一个方法`Mult`，用于计算先验概率与似然值的乘机以获得后验概率（其实是与后验概率成正比的数值）。\n",
    "\n",
    "除此之外再定义一个Prob函数，用于提取某个假设的概率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class Bayes(object):\n",
    "    \"\"\"A bayes class, mainly a dictionary\"\"\"\n",
    "    def __init__(self, hypos=None, name=''):\n",
    "        \"\"\"\n",
    "        Initialize the distribution.\n",
    "        \n",
    "        hypos: sequence of hypotheses\n",
    "        \"\"\"\n",
    "        self.name = name\n",
    "        self.pmf = {}\n",
    "        if hypos is None:\n",
    "            return\n",
    "        \n",
    "        # Initiate the class object\n",
    "        # Three initalize methods are used to deal with different types of input\n",
    "        # \n",
    "        init_methods = [\n",
    "            self.InitPmf,\n",
    "            self.InitMapping,  #A dict\n",
    "            self.InitSequence, #equal probability for all hypos\n",
    "            self.InitFailure,\n",
    "        ]\n",
    "        \n",
    "        for method in init_methods:\n",
    "            try:\n",
    "                method(hypos)\n",
    "                break\n",
    "            except AttributeError:\n",
    "                continue\n",
    "        \n",
    "        if len(self):\n",
    "            self.Normalize()\n",
    "    \n",
    "    def __str__(self):\n",
    "        '''\n",
    "        Stringlize self.pmf\n",
    "        '''\n",
    "        tmpL = [\"Probability table\"]\n",
    "        for hypo, prob in sorted(self.pmf.iteritems()):\n",
    "            tmpL.append('\\t'.join([str(hypo), str(prob)]))\n",
    "        return '\\n'.join(tmpL)\n",
    "    \n",
    "    def InitSequence(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a sequence of hypos with equal probabilities.\n",
    "        \n",
    "        hypos: ['H1','H2','H3',...]\n",
    "        \"\"\"\n",
    "        for hypo in hypos:\n",
    "            self.Set(hypo, 1)\n",
    "    \n",
    "    def InitMapping(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a map from value to probablity (a dict).\n",
    "        \n",
    "        hypos = {'H1':1, 'H2':5, 'H3':4}\n",
    "        \"\"\"\n",
    "        for hypo, prob in hypos.iteritems():\n",
    "            self.Set(hypo, prob)\n",
    "    \n",
    "    def InitPmf(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a Bayes object.\n",
    "        \n",
    "        hypos = Bayes()\n",
    "        \"\"\"\n",
    "        for hypo, prob in hypos.iteritems():\n",
    "            self.Set(hypo, prob)\n",
    "    \n",
    "    def InitFailure(self, hypos):\n",
    "        \"\"\"Raise an errot\"\"\"\n",
    "        raise ValueError(\"None of the initialization methods works.\")\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.pmf)\n",
    "    \n",
    "    def Set(self, hypo, prob=0):\n",
    "        \"\"\"\n",
    "        Set hypo-prob pair\n",
    "        \"\"\"\n",
    "        self.pmf[hypo] = prob\n",
    "    \n",
    "    def Print(self):\n",
    "        \"\"\"Print the values and freqs in asending order.\"\"\"\n",
    "        for hypo, prob in sorted(self.pmf.iteritems()):\n",
    "            print hypo, prob\n",
    "    \n",
    "    def Normalize(self):\n",
    "        \"\"\"\n",
    "        Normalize probability\n",
    "        \"\"\"\n",
    "        total = float(sum(self.pmf.values()))\n",
    "        if total == 0.0:\n",
    "            raise ValueError('total probability is zero.')\n",
    "            logging.warning('Normalize: total probability is zero.')\n",
    "            return total\n",
    "        \n",
    "        factor = 1 / total\n",
    "        \n",
    "        for hypo in self.pmf:\n",
    "            self.pmf[hypo] *= factor\n",
    "    \n",
    "    def Mult(self, hypo, likelihood):\n",
    "        '''\n",
    "        Update hypo probability by given likelihood\n",
    "        '''\n",
    "        self.pmf[hypo] = self.pmf.get(hypo,0) * likelihood\n",
    "    \n",
    "    def Prob(self, hypo, default=0):\n",
    "        \"\"\"\n",
    "        Get the probability of given hypo.\n",
    "        \"\"\"\n",
    "        return self.pmf.get(hypo, default)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1 假设有袋圆球，罐1中有30个黑球和10个白球，罐2中黑球和白球各20个。某人随机的从一个罐子中取出一粒球，发现是黑球，问这个黑球从罐1中取出的概率有多大？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Prior probability\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.5\n",
      "Bowl 2\t0.5\n",
      "\n",
      "Posterior probability\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.6\n",
      "Bowl 2\t0.4\n",
      "\n",
      "Probabiliy for 黑球来源于罐1: 0.6\n"
     ]
    }
   ],
   "source": [
    "# 我们用 \n",
    "#    Bowl 1表示黑球来源于罐1\n",
    "#    Bowl 2表示黑球来源于罐2\n",
    "\n",
    "hypos = [\"Bowl 1\", \"Bowl 2\"]\n",
    "\n",
    "bowl = Bayes(hypos)\n",
    "\n",
    "print \"\\nPrior probability\\n\"\n",
    "print bowl\n",
    "\n",
    "bowl.Mult(\"Bowl 1\", 0.75)\n",
    "bowl.Mult(\"Bowl 2\", 0.5)\n",
    "\n",
    "# 注意这个Normalize；\n",
    "#   因为我们的穷举了所有独立的假设，因此所有假设的概率和为1.\n",
    "#   我们依据此进行Normalize.\n",
    "bowl.Normalize()\n",
    "\n",
    "print \"\\nPosterior probability\\n\"\n",
    "print bowl\n",
    "\n",
    "print \"\\nProbabiliy for 黑球来源于罐1:\", bowl.Prob('Bowl 1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.1.1 上面的代码中对后验概率的计算为每个假设分布进行，我们定义了`Update`函数，使得可以同时对所有假设的后验概率进行更新。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class Bayes(object):\n",
    "    \"\"\"A bayes class, mainly a dictionary\"\"\"\n",
    "    def __init__(self, hypos=None, name=''):\n",
    "        \"\"\"\n",
    "        Initialize the distribution.\n",
    "        \n",
    "        hypos: sequence of hypotheses\n",
    "        \"\"\"\n",
    "        self.name = name\n",
    "        self.pmf = {}\n",
    "        if hypos is None:\n",
    "            return\n",
    "        \n",
    "        # Initiate the class object\n",
    "        # Three initalize methods are used to deal with different types of input\n",
    "        # \n",
    "        init_methods = [\n",
    "            self.InitPmf,\n",
    "            self.InitMapping,  #A dict\n",
    "            self.InitSequence, #equal probability for all hypos\n",
    "            self.InitFailure,\n",
    "        ]\n",
    "        \n",
    "        for method in init_methods:\n",
    "            try:\n",
    "                method(hypos)\n",
    "                break\n",
    "            except AttributeError:\n",
    "                continue\n",
    "        \n",
    "        if len(self):\n",
    "            self.Normalize()\n",
    "    \n",
    "    def __str__(self):\n",
    "        '''\n",
    "        Stringlize self.pmf\n",
    "        '''\n",
    "        tmpL = [\"Probability table\"]\n",
    "        for hypo, prob in sorted(self.pmf.iteritems()):\n",
    "            tmpL.append('\\t'.join([str(hypo), str(prob)]))\n",
    "        return '\\n'.join(tmpL)\n",
    "    \n",
    "    def InitSequence(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a sequence of hypos with equal probabilities.\n",
    "        \n",
    "        hypos: ['H1','H2','H3',...]\n",
    "        \"\"\"\n",
    "        for hypo in hypos:\n",
    "            self.Set(hypo, 1)\n",
    "    \n",
    "    def InitMapping(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a map from value to probablity (a dict).\n",
    "        \n",
    "        hypos = {'H1':1, 'H2':5, 'H3':4}\n",
    "        \"\"\"\n",
    "        for hypo, prob in hypos.iteritems():\n",
    "            self.Set(hypo, prob)\n",
    "    \n",
    "    def InitPmf(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a Bayes object.\n",
    "        \n",
    "        hypos = Bayes()\n",
    "        \"\"\"\n",
    "        for hypo, prob in hypos.iteritems():\n",
    "            self.Set(hypo, prob)\n",
    "    \n",
    "    def InitFailure(self, hypos):\n",
    "        \"\"\"Raise an errot\"\"\"\n",
    "        raise ValueError(\"None of the initialization methods works.\")\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.pmf)\n",
    "    \n",
    "    def Set(self, hypo, prob=0):\n",
    "        \"\"\"\n",
    "        Set hypo-prob pair\n",
    "        \"\"\"\n",
    "        self.pmf[hypo] = prob\n",
    "    \n",
    "    def Print(self):\n",
    "        \"\"\"Print the values and freqs in asending order.\"\"\"\n",
    "        for hypo, prob in sorted(self.pmf.iteritems()):\n",
    "            print hypo, prob\n",
    "    \n",
    "    def Normalize(self):\n",
    "        \"\"\"\n",
    "        Normalize probability\n",
    "        \"\"\"\n",
    "        total = float(sum(self.pmf.values()))\n",
    "        if total == 0.0:\n",
    "            raise ValueError('total probability is zero.')\n",
    "            logging.warning('Normalize: total probability is zero.')\n",
    "            return total\n",
    "        \n",
    "        factor = 1 / total\n",
    "        \n",
    "        for hypo in self.pmf:\n",
    "            self.pmf[hypo] *= factor\n",
    "    \n",
    "    def Mult(self, hypo, likelihood):\n",
    "        '''\n",
    "        Update given hypo probability by given likelihood\n",
    "        '''\n",
    "        self.pmf[hypo] = self.pmf.get(hypo,0) * likelihood\n",
    "    \n",
    "    def Prob(self, hypo, default=0):\n",
    "        \"\"\"\n",
    "        Get the probability of given hypo.\n",
    "        \"\"\"\n",
    "        return self.pmf.get(hypo, default)\n",
    "    \n",
    "    def Update(self):\n",
    "        '''\n",
    "        Update all hypo probability by given likelihood\n",
    "        '''\n",
    "        for hypo, prob in self.pmf.iteritems():\n",
    "            self.pmf[hypo] = prob * self.Likelihood(hypo)\n",
    "        self.Normalize()\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.1.2 上面定义的类`Bayes`中的`Update`函数依赖另一个函数Likelihood。对于不同的问题，`Prior probability`和`likelihood`不同；`Prior probability`可以在初始化时给定，对于`likelihood`我们也可以在类的外面定义函数计算`likelihood`并在初始化时给定，但是如果观察数据较多，我们就得多次初始化，操作起来比较繁琐。这儿采用的是类继承的方式，继承基类`Bayes`，在新的类中定义`Likelihood`方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Prior probability\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.5\n",
      "Bowl 2\t0.5\n",
      "\n",
      "Posterior probability\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.6\n",
      "Bowl 2\t0.4\n"
     ]
    }
   ],
   "source": [
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class Bowl(Bayes):\n",
    "    likelihood = {\"Bowl 1\": 0.75, \"Bowl 2\": 0.5}\n",
    "    def Likelihood(self, hypo):\n",
    "        return self.likelihood[hypo]\n",
    "\n",
    "hypos = [\"Bowl 1\", \"Bowl 2\"]\n",
    "\n",
    "bowl = Bowl(hypos)\n",
    "print \"\\nPrior probability\\n\"\n",
    "print bowl\n",
    "\n",
    "bowl.Update()\n",
    "print \"\\nPosterior probability\\n\"\n",
    "print bowl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.1.3 上面的代码中，似然值是固化在里面的，且只针对黑球，我们再做进一步的改变，使得程序可以处理多次取出不同球判断球来自于Bowl1的概率（取出后放回）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class Bayes(object):\n",
    "    \"\"\"A bayes class, mainly a dictionary\"\"\"\n",
    "    def __init__(self, hypos=None, name=''):\n",
    "        \"\"\"\n",
    "        Initialize the distribution.\n",
    "        \n",
    "        hypos: sequence of hypotheses\n",
    "        \"\"\"\n",
    "        self.name = name\n",
    "        self.pmf = {}\n",
    "        if hypos is None:\n",
    "            return\n",
    "        \n",
    "        # Initiate the class object\n",
    "        # Three initalize methods are used to deal with different types of input\n",
    "        # \n",
    "        init_methods = [\n",
    "            self.InitPmf,\n",
    "            self.InitMapping,  #A dict\n",
    "            self.InitSequence, #equal probability for all hypos\n",
    "            self.InitFailure,\n",
    "        ]\n",
    "        \n",
    "        for method in init_methods:\n",
    "            try:\n",
    "                method(hypos)\n",
    "                break\n",
    "            except AttributeError:\n",
    "                continue\n",
    "        \n",
    "        if len(self):\n",
    "            self.Normalize()\n",
    "    \n",
    "    def __str__(self):\n",
    "        '''\n",
    "        Stringlize self.pmf\n",
    "        '''\n",
    "        tmpL = [\"Probability table\"]\n",
    "        for hypo, prob in sorted(self.pmf.iteritems()):\n",
    "            tmpL.append('\\t'.join([str(hypo), str(prob)]))\n",
    "        return '\\n'.join(tmpL)\n",
    "    \n",
    "    def InitSequence(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a sequence of hypos with equal probabilities.\n",
    "        \n",
    "        hypos: ['H1','H2','H3',...]\n",
    "        \"\"\"\n",
    "        for hypo in hypos:\n",
    "            self.Set(hypo, 1)\n",
    "    \n",
    "    def InitMapping(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a map from value to probablity (a dict).\n",
    "        \n",
    "        hypos = {'H1':1, 'H2':5, 'H3':4}\n",
    "        \"\"\"\n",
    "        for hypo, prob in hypos.iteritems():\n",
    "            self.Set(hypo, prob)\n",
    "    \n",
    "    def InitPmf(self, hypos):\n",
    "        \"\"\"\n",
    "        Initialize with a Bayes object.\n",
    "        \n",
    "        hypos = Bayes()\n",
    "        \"\"\"\n",
    "        for hypo, prob in hypos.iteritems():\n",
    "            self.Set(hypo, prob)\n",
    "    \n",
    "    def InitFailure(self, hypos):\n",
    "        \"\"\"Raise an errot\"\"\"\n",
    "        raise ValueError(\"None of the initialization methods works.\")\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.pmf)\n",
    "    \n",
    "    def Set(self, hypo, prob=0):\n",
    "        \"\"\"\n",
    "        Set hypo-prob pair\n",
    "        \"\"\"\n",
    "        self.pmf[hypo] = prob\n",
    "    \n",
    "    def Print(self):\n",
    "        \"\"\"Print the values and freqs in asending order.\"\"\"\n",
    "        for hypo, prob in sorted(self.pmf.iteritems()):\n",
    "            print hypo, prob\n",
    "    \n",
    "    def Normalize(self):\n",
    "        \"\"\"\n",
    "        Normalize probability\n",
    "        \"\"\"\n",
    "        total = float(sum(self.pmf.values()))\n",
    "        if total == 0.0:\n",
    "            raise ValueError('total probability is zero.')\n",
    "            logging.warning('Normalize: total probability is zero.')\n",
    "            return total\n",
    "        \n",
    "        factor = 1 / total\n",
    "        \n",
    "        for hypo in self.pmf:\n",
    "            self.pmf[hypo] *= factor\n",
    "    \n",
    "    def Mult(self, hypo, likelihood):\n",
    "        '''\n",
    "        Update given hypo probability by given likelihood\n",
    "        '''\n",
    "        self.pmf[hypo] = self.pmf.get(hypo,0) * likelihood\n",
    "    \n",
    "    def Prob(self, hypo, default=0):\n",
    "        \"\"\"\n",
    "        Get the probability of given hypo.\n",
    "        \"\"\"\n",
    "        return self.pmf.get(hypo, default)\n",
    "    \n",
    "    def Update(self,dataL):\n",
    "        '''\n",
    "        Update all hypo probability by given obervation.\n",
    "        \n",
    "        dataL: A list of observations.\n",
    "        '''\n",
    "        for data in dataL:\n",
    "            for hypo, prob in self.pmf.iteritems():\n",
    "                self.pmf[hypo] = prob * self.Likelihood(hypo, data)\n",
    "        self.Normalize()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Prior probability\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.5\n",
      "Bowl 2\t0.5\n",
      "\n",
      "Posterior probability after get black ball.\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.6\n",
      "Bowl 2\t0.4\n",
      "\n",
      "Posterior probability after get white ball.\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.428571428571\n",
      "Bowl 2\t0.571428571429\n",
      "\n",
      "Posterior probability after get black ball.\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.529411764706\n",
      "Bowl 2\t0.470588235294\n"
     ]
    }
   ],
   "source": [
    "## 黑白球，取出后放回\n",
    "\n",
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class Bowl(Bayes):\n",
    "    state = {'Bowl 1': {\"black\": 0.75, \"white\": 0.25}, 'Bowl 2': {\"black\":0.5, \"white\":0.5}}\n",
    "    def Likelihood(self, hypo, data):\n",
    "        return self.state[hypo][data]\n",
    "\n",
    "hypos = [\"Bowl 1\", \"Bowl 2\"]\n",
    "\n",
    "bowl = Bowl(hypos)\n",
    "print \"\\nPrior probability\\n\"\n",
    "print bowl\n",
    "\n",
    "#dataL = [\"black\",'white','black']\n",
    "dataL = ['black']\n",
    "\n",
    "bowl.Update(dataL)\n",
    "print \"\\nPosterior probability after get %s ball.\\n\" % ','.join(dataL)\n",
    "print bowl\n",
    "\n",
    "dataL = ['white']\n",
    "\n",
    "bowl.Update(dataL)\n",
    "print \"\\nPosterior probability after get %s ball.\\n\" % ','.join(dataL)\n",
    "print bowl\n",
    "\n",
    "dataL = ['black']\n",
    "\n",
    "bowl.Update(dataL)\n",
    "print \"\\nPosterior probability after get %s ball.\\n\" % ','.join(dataL)\n",
    "print bowl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Prior probability\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.5\n",
      "Bowl 2\t0.5\n",
      "\n",
      "Posterior probability after get <black,white,black> ball.\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.529411764706\n",
      "Bowl 2\t0.470588235294\n"
     ]
    }
   ],
   "source": [
    "## 黑白球，取出后放回\n",
    "\n",
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class Bowl(Bayes):\n",
    "    state = {'Bowl 1': {\"black\": 0.75, \"white\": 0.25}, 'Bowl 2': {\"black\":0.5, \"white\":0.5}}\n",
    "    def Likelihood(self, hypo, data):\n",
    "        return self.state[hypo][data]\n",
    "\n",
    "hypos = [\"Bowl 1\", \"Bowl 2\"]\n",
    "\n",
    "bowl = Bowl(hypos)\n",
    "print \"\\nPrior probability\\n\"\n",
    "print bowl\n",
    "\n",
    "dataL = [\"black\",'white','black']\n",
    "\n",
    "bowl.Update(dataL)\n",
    "print \"\\nPosterior probability after get <%s> ball.\\n\" % ','.join(dataL)\n",
    "print bowl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.1.4 黑白球问题，如果是取出之后不放，这个问题应该怎么计算？ 这里我们做一个假设，取出的球来自于当次取出后，计算出的后验概率大的罐子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Prior probability\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.5\n",
      "Bowl 2\t0.5\n",
      "\n",
      "Current state: {u'Bowl 2': {u'white': 20, u'black': 20}, u'Bowl 1': {u'white': 10, u'black': 30}}\n",
      "Get <black> ball\n",
      "Bowl 2 0.4\n",
      "Bowl 1 0.6\n",
      "We assume <black> ball from <Bowl 1>.\n",
      "\n",
      "Current state: {u'Bowl 2': {u'white': 20, u'black': 20}, u'Bowl 1': {u'white': 10, u'black': 29}}\n",
      "Get <white> ball\n",
      "Bowl 2 0.565217391304\n",
      "Bowl 1 0.434782608696\n",
      "We assume <white> ball from <Bowl 2>.\n",
      "\n",
      "Current state: {u'Bowl 2': {u'white': 19, u'black': 20}, u'Bowl 1': {u'white': 10, u'black': 29}}\n",
      "Get <black> ball\n",
      "Bowl 2 0.472727272727\n",
      "Bowl 1 0.527272727273\n",
      "We assume <black> ball from <Bowl 1>.\n",
      "\n",
      "\n",
      "Posterior probability after get <black,white,black> ball.\n",
      "\n",
      "Probability table\n",
      "Bowl 1\t0.527272727273\n",
      "Bowl 2\t0.472727272727\n"
     ]
    }
   ],
   "source": [
    "## 黑白球，取出后不放回\n",
    "\n",
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class Bowl(Bayes):\n",
    "    state = {'Bowl 1': {\"black\": 30, \"white\": 10}, 'Bowl 2': {\"black\":20, \"white\":20}}\n",
    "    def Likelihood(self, hypo, data):\n",
    "        return self.state[hypo][data]/sum(self.state[hypo].values())\n",
    "    \n",
    "    def Update_state(self, data):\n",
    "        choose = ''\n",
    "        prob  = 0\n",
    "        for hypo, post_prob in self.pmf.items():\n",
    "            if post_prob > prob:\n",
    "                prob = post_prob\n",
    "                choose = hypo\n",
    "        self.state[choose][data] -= 1\n",
    "        print \"We assume <%s> ball from <%s>.\" % (data, choose)\n",
    "    \n",
    "    def Update(self, dataL):\n",
    "        '''\n",
    "        Update probability using observed data <dataL>\n",
    "        '''\n",
    "        for data in dataL:\n",
    "            print \"Current state:\", self.state\n",
    "            print \"Get <%s> ball\" % data\n",
    "            for hypo, prob in self.pmf.items():\n",
    "                self.pmf[hypo] = prob * self.Likelihood(hypo,data)\n",
    "            self.Normalize() #可以不要，只是测试用\n",
    "            for hypo, prob in self.pmf.items():\n",
    "                print hypo, prob\n",
    "                \n",
    "            self.Update_state(data)\n",
    "            print\n",
    "            \n",
    "        \n",
    "hypos = [\"Bowl 1\", \"Bowl 2\"]\n",
    "\n",
    "bowl = Bowl(hypos)\n",
    "print \"\\nPrior probability\\n\"\n",
    "print bowl\n",
    "print \n",
    "\n",
    "dataL = [\"black\",'white','black']\n",
    "\n",
    "bowl.Update(dataL)\n",
    "print \"\\nPosterior probability after get <%s> ball.\\n\" % ','.join(dataL)\n",
    "print bowl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "#### 3.2 Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?”"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Prior probability\n",
      "\n",
      "Probability table\n",
      "Door 1\t0.333333333333\n",
      "Door 2\t0.333333333333\n",
      "Door 3\t0.333333333333\n",
      "\n",
      "Posterior probability for each door after the host picking <Door 3>.\n",
      "\n",
      "Probability table\n",
      "Door 1\t0.333333333333\n",
      "Door 2\t0.666666666667\n",
      "Door 3\t0.0\n"
     ]
    }
   ],
   "source": [
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class Monty(Bayes):\n",
    "    def Likelihood(self, hypo, data):\n",
    "        if hypo == \"Door 1\" and data == \"Door 3\":\n",
    "            return 0.5\n",
    "        elif hypo == \"Door 2\" and data == \"Door 3\":\n",
    "            return 1\n",
    "        elif hypo == \"Door 3\" and data == \"Door 3\":\n",
    "            return 0\n",
    "        else:\n",
    "            raise ValueError('Unknown hypos or data')\n",
    "            logging.warning('Likelihood: Unknown hypos or data')\n",
    "            \n",
    "\n",
    "hypos = [\"Door 1\", \"Door 2\", \"Door 3\"]\n",
    "\n",
    "game = Monty(hypos)\n",
    "print \"\\nPrior probability\\n\"\n",
    "print game\n",
    "\n",
    "dataL = [\"Door 3\"]\n",
    "\n",
    "game.Update(dataL)\n",
    "print \"\\nPosterior probability for each door after the host picking <%s>.\\n\" % ','.join(dataL)\n",
    "print game"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3 公司在不同年份生产的M&M豆包含的不同颜色的豆的比例不同， 1994年产的M&M豆包装中，棕色30%，黄色20%，红色20%，绿色10%，橙色10%， 茶色10%；1996年产的M&M豆包装中，棕色13%，黄色14%，红色13%，绿色20%， 橙色16%，蓝色24%。假设手中有两粒M&M豆，分别是橙色和绿色， 一个来自1994年包装，一个来自1996年包装，求算橙色来源于1994年包装的概率？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.3.1 初始解法 （出发点错误）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Prior probability\n",
      "\n",
      "Probability table\n",
      "A\t0.5\n",
      "B\t0.5\n",
      "1994 orange 0.1\n",
      "1996 green 0.02\n",
      "1994 green 0.1\n",
      "1996 orange 0.016\n",
      "\n",
      "Posterior probability\n",
      "\n",
      "Probability table\n",
      "A\t0.555555555556\n",
      "B\t0.444444444444\n"
     ]
    }
   ],
   "source": [
    "##一个解法，有点乱，没有理顺关系\n",
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class MM(Bayes):\n",
    "    \n",
    "    mm_state = {'1994':dict(brown=30,yellow=20,red=20,green=10,orange=10,tan=10),\n",
    "                '1996':dict(blue=24,green=20,orange=16,yellow=14,red=13,brown=13)}\n",
    "    \n",
    "    hyposD = {\"A\": {'1994':'orange', '1996':'green'}, \"B\":{'1994':'green','1996':'orange'}}\n",
    "    #hyposD = {\"A\": {'1994':'yellow', '1996':'green'}, \"B\":{'1994':'green','1996':'yellow'}}\n",
    "    \n",
    "    def Likelihood(self, hypo, data):        \n",
    "        likelihood = 1\n",
    "        for hypo_cur, data_cur in self.hyposD[hypo].items():\n",
    "            likelihood *= self.mm_state[hypo_cur][data_cur] / sum(self.mm_state[hypo_cur].values())\n",
    "            #print hypo_cur, data_cur,likelihood\n",
    "        return likelihood\n",
    "            \n",
    "            \n",
    "## hypo A: 橙色1994，绿色1996\n",
    "## hypo B: 橙色1996，绿色1994\n",
    "\n",
    "hypos = [\"A\", \"B\"]\n",
    "\n",
    "mm = MM(hypos)\n",
    "print \"\\nPrior probability\\n\"\n",
    "print mm\n",
    "\n",
    "data = ['green-orange']\n",
    "\n",
    "mm.Update(data)\n",
    "print \"\\nPosterior probability\\n\"\n",
    "print mm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.3.2 正确解法 （合理表示观察数据和假设）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Probability table\n",
      "A\t0.259259259259\n",
      "B\t0.740740740741\n"
     ]
    }
   ],
   "source": [
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class MM(Bayes):\n",
    "    \n",
    "    mix1994 = dict(brown=30,yellow=20,red=20,green=10,orange=10,tan=10)\n",
    "    mix1996 = dict(blue=24,green=20,orange=16,yellow=14,red=13,brown=13)\n",
    "    \n",
    "    hyposD = {'A':{'bag1':mix1994,'bag2':mix1996}, 'B':{'bag1':mix1996,'bag2':mix1994}}\n",
    "    #hyposD = {\"A\": {'1994':'yellow', '1996':'green'}, \"B\":{'1994':'green','1996':'yellow'}}\n",
    "    \n",
    "    def Likelihood(self, hypo, data):        \n",
    "        color,bag = data\n",
    "        likelihood = self.hyposD[hypo][bag][color]/sum(self.hyposD[hypo][bag].values())\n",
    "        return likelihood\n",
    "        \n",
    "## 引入两个变量bag1, bag2代表两个袋子\n",
    "## 列出我们的观察数据，绿色的来源于其中一个袋子bag1，红色来源于另一个袋子bag2\n",
    "\n",
    "dataL = [(\"green\",\"bag1\"),(\"yellow\",\"bag2\")]\n",
    "\n",
    "## 列出假设\n",
    "## hypo A: bag1是1994，bag2是1996\n",
    "## hypo B: bag1是1996，bag2是1994\n",
    "\n",
    "## 这里我们利用中间变量bag1和bag2来提出假设，判断bag1是1994还是1996，而不是根据颜色提出假设。\n",
    "## 这么做的好处是方便处理观察到的数据。\n",
    "\n",
    "hypos = ['A','B']\n",
    "\n",
    "mm = MM(hypos)\n",
    "\n",
    "mm.Update(dataL)\n",
    "\n",
    "print mm\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.3.3 正确解法 （合理表示观察数据和假设，同时把观察数据定义为一次观察的结果）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A: green from mix1994, yellow from mix1996\n",
      "B: green from mix1996, yellow from mix1994\n",
      "Probability table\n",
      "A\t0.259259259259\n",
      "B\t0.740740740741\n"
     ]
    }
   ],
   "source": [
    "from __future__ import division, unicode_literals\n",
    "\n",
    "import logging\n",
    "\n",
    "class MM(Bayes):\n",
    "    \n",
    "    mix1994 = dict(brown=30,yellow=20,red=20,green=10,orange=10,tan=10)\n",
    "    mix1996 = dict(blue=24,green=20,orange=16,yellow=14,red=13,brown=13)\n",
    "    \n",
    "    hyposD = {'A':{'bag1':mix1994,'bag2':mix1996}, 'B':{'bag1':mix1996,'bag2':mix1994}}\n",
    "    #hyposD = {\"A\": {'1994':'yellow', '1996':'green'}, \"B\":{'1994':'green','1996':'yellow'}}\n",
    "    \n",
    "    def Likelihood(self, hypo, data):     \n",
    "        likelihood = 1\n",
    "        for color,bag in data.items():\n",
    "            likelihood *= self.hyposD[hypo][bag][color]/sum(self.hyposD[hypo][bag].values())\n",
    "        return likelihood\n",
    "\n",
    "#mix1994 = dict(brown=30,yellow=20,red=20,green=10,orange=10,tan=10)\n",
    "#mix1996 = dict(blue=24,green=20,orange=16,yellow=14,red=13,brown=13)\n",
    "    \n",
    "## 引入两个变量bag1, bag2代表两个袋子\n",
    "## 列出我们的观察数据，绿色的来源于其中一个袋子bag1，红色来源于另一个袋子bag2\n",
    "\n",
    "dataL = [{\"green\":\"bag1\",\"yellow\":\"bag2\"}]\n",
    "\n",
    "## 列出假设\n",
    "## hypo A: bag1是1994，bag2是1996\n",
    "## hypo B: bag1是1996，bag2是1994\n",
    "hyposD = {'A':{'bag1':\"mix1994\",'bag2':\"mix1996\"}, 'B':{'bag1':\"mix1996\",'bag2':\"mix1994\"}}\n",
    "\n",
    "## 这里我们利用中间变量bag1和bag2来提出假设，判断bag1是1994还是1996，而不是根据颜色提出假设。\n",
    "## 这么做的好处是方便处理观察到的数据。\n",
    "\n",
    "hypos = ['A','B']\n",
    "\n",
    "mm = MM(hypos)\n",
    "\n",
    "mm.Update(dataL)\n",
    "\n",
    "for hypo, describD in hyposD.items():\n",
    "    print \"%s: %s from %s, %s from %s\" % (hypo, 'green',describD[dataL[0]['green']],'yellow',describD[dataL[0]['yellow']])\n",
    "\n",
    "print mm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**MM问题需要再次消化**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.4 假设一项药物测试的假阳性率（非特异性）和假阴性率（不敏感性）都是1%。 已知人群中服用过该药物的个体约占0.5%。如果随机选择一个个体检测为阳性， 那么他服药的概率是多少？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prior probability:\n",
      "\n",
      "Probability table\n",
      "negative\t0.995\n",
      "positive\t0.005\n",
      "Posterior probability:\n",
      "\n",
      "Probability table\n",
      "negative\t0.667785234899\n",
      "positive\t0.332214765101\n"
     ]
    }
   ],
   "source": [
    "class Test(Bayes):\n",
    "    \n",
    "    testSta = {'positive':{'test_positive':0.99,'test_negative':0.01},\n",
    "               'negative':{'test_positive':0.01,'test_nagative':0.99}}\n",
    "    \n",
    "    def Likelihood(self, hypo, data):\n",
    "        return self.testSta[hypo][data]\n",
    "\n",
    "# 观察到的数据\n",
    "data = [\"test_positive\"]\n",
    "\n",
    "# 假设\n",
    "hyposD = {'positive':0.005, 'negative':0.995}\n",
    "\n",
    "test = Test(hyposD)\n",
    "print \"Prior probability:\\n\"\n",
    "print test\n",
    "\n",
    "test.Update(data)\n",
    "print \"Posterior probability:\\n\"\n",
    "print test\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.4.1 提高检测敏感性，降低假阴性率，对更准确表示检测结果帮助不大。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prior probability:\n",
      "\n",
      "Probability table\n",
      "negative\t0.995\n",
      "positive\t0.005\n",
      "Posterior probability:\n",
      "\n",
      "Probability table\n",
      "negative\t0.665551839465\n",
      "positive\t0.334448160535\n"
     ]
    }
   ],
   "source": [
    "class Test(Bayes):\n",
    "    \n",
    "    # 提高检测敏感性，降低假阴性率\n",
    "    testSta = {'positive':{'test_positive':1,'test_negative':0},\n",
    "               'negative':{'test_positive':0.01,'test_nagative':0.99}}\n",
    "    \n",
    "    def Likelihood(self, hypo, data):\n",
    "        return self.testSta[hypo][data]\n",
    "\n",
    "# 观察到的数据\n",
    "data = [\"test_positive\"]\n",
    "\n",
    "# 假设\n",
    "hyposD = {'positive':0.005, 'negative':0.995}\n",
    "\n",
    "test = Test(hyposD)\n",
    "print \"Prior probability:\\n\"\n",
    "print test\n",
    "\n",
    "test.Update(data)\n",
    "print \"Posterior probability:\\n\"\n",
    "print test\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.4.2 提高检测特异性，降低假阳性率，有助于更准确表示检测结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prior probability:\n",
      "\n",
      "Probability table\n",
      "negative\t0.995\n",
      "positive\t0.005\n",
      "Posterior probability:\n",
      "\n",
      "Probability table\n",
      "negative\t0.501259445844\n",
      "positive\t0.498740554156\n"
     ]
    }
   ],
   "source": [
    "class Test(Bayes):\n",
    "    \n",
    "    # 提高检测特异性，降低假阳性率\n",
    "    testSta = {'positive':{'test_positive':0.99,'test_negative':0.01},\n",
    "               'negative':{'test_positive':0.005,'test_nagative':0.995}}\n",
    "    \n",
    "    def Likelihood(self, hypo, data):\n",
    "        return self.testSta[hypo][data]\n",
    "\n",
    "# 观察到的数据\n",
    "data = [\"test_positive\"]\n",
    "\n",
    "# 假设\n",
    "hyposD = {'positive':0.005, 'negative':0.995}\n",
    "\n",
    "test = Test(hyposD)\n",
    "print \"Prior probability:\\n\"\n",
    "print test\n",
    "\n",
    "test.Update(data)\n",
    "print \"Posterior probability:\\n\"\n",
    "print test\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}