Skip to content

Commit

Permalink
Split Jeffrey's prior bit into its own cells
Browse files Browse the repository at this point in the history
  • Loading branch information
zaxtax committed Jan 2, 2014
1 parent 2d606b0 commit 6e48334
Showing 1 changed file with 36 additions and 11 deletions.
47 changes: 36 additions & 11 deletions Chapter6_Priorities/Priors.ipynb
Original file line number Original file line Diff line number Diff line change
Expand Up @@ -1389,20 +1389,45 @@
"\n", "\n",
"2. There typically exist conjugate priors for simple, one dimensional problems. For larger problems, involving more complicated structures, hope is lost to find a conjugate prior. For smaller models, Wikipedia has a nice [table of conjugate priors](http://en.wikipedia.org/wiki/Conjugate_prior#Table_of_conjugate_distributions).\n", "2. There typically exist conjugate priors for simple, one dimensional problems. For larger problems, involving more complicated structures, hope is lost to find a conjugate prior. For smaller models, Wikipedia has a nice [table of conjugate priors](http://en.wikipedia.org/wiki/Conjugate_prior#Table_of_conjugate_distributions).\n",
"\n", "\n",
"Really, conjugate priors are only useful for their mathematical convenience: it is simple to go from prior to posterior. I personally see conjugate priors as only a neat mathematical trick, and offer little insight into the problem at hand. \n", "Really, conjugate priors are only useful for their mathematical convenience: it is simple to go from prior to posterior. I personally see conjugate priors as only a neat mathematical trick, and offer little insight into the problem at hand. "
"\n", ]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Jefferys Priors\n", "## Jefferys Priors\n",
"\n", "\n",
"Earlier, we talked about objective priors rarely being *objective*. Partly what we mean by this is that we want a prior that doesn't bias our posterior estimates. The flat prior seems like a reasonable choice as it assigns equal probability to all values. \n", "Earlier, we talked about objective priors rarely being *objective*. Partly what we mean by this is that we want a prior that doesn't bias our posterior estimates. The flat prior seems like a reasonable choice as it assigns equal probability to all values. \n",
"\n", "\n",
"But the flat prior is not transformation invariant. What does this mean? Suppose we have a random variable $ \\bf X $ from Bernoulli($\\theta$). We define the prior on $p(\\theta) = 1$. \n", "But the flat prior is not transformation invariant. What does this mean? Suppose we have a random variable $ \\bf X $ from Bernoulli($\\theta$). We define the prior on $p(\\theta) = 1$. "
"\n", ]
"PUT PLOT OF THETA HERE\n", },
"\n", {
"Now, let's transform $\\theta$ with the function $\\psi = log \\frac{\\theta}{1-\\theta}$. This is just a function to stretch $\\theta$ across the real line. Now how likely are different values of $\\psi$ under our transformation.\n", "cell_type": "markdown",
"\n", "metadata": {},
"PUT PLOT OF PSI HERE\n", "source": [
"\n", "PUT PLOT OF THETA HERE"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, let's transform $\\theta$ with the function $\\psi = log \\frac{\\theta}{1-\\theta}$. This is just a function to stretch $\\theta$ across the real line. Now how likely are different values of $\\psi$ under our transformation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"PUT PLOT OF PSI HERE"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Oh no! Our function is no longer flat. It turns out flat priors do carry information in them after all. The point of Jeffreys Priors is to create priors that don't accidentally become informative when you transform the variables you placed them originally on." "Oh no! Our function is no longer flat. It turns out flat priors do carry information in them after all. The point of Jeffreys Priors is to create priors that don't accidentally become informative when you transform the variables you placed them originally on."
] ]
}, },
Expand Down Expand Up @@ -1695,4 +1720,4 @@
"metadata": {} "metadata": {}
} }
] ]
} }

0 comments on commit 6e48334

Please sign in to comment.