final version

mfvb.tex

@@ -4,7 +4,7 @@ \section{Mean-Field Variational Bayes}\label{sec:mfvb}
 
 Mean-field variational Bayes (MFVB) is a method for approximating the posterior distribution.  In general, we have unknown parameters $w_1, w_2, \ldots, w_n$ that we have priors on, and our objective is to find the joint distribution $p(w_1, w_2, \ldots, w_n)$.  Assuming that our approximate distribution is in the family $Q = \{q : q(w_1, w_2, \ldots, w_n) = q(w_1)q(w_2) \ldots q(w_n)\}$, we find $q^* \in Q$ that minimizes the KL-divergence with $p$, i.e. $q^* = \min KL(q || p)$. 
 In particular, for logistic regression, the analytical form of the posterior is unknown and has been approximated with MFVB in the literature. We use local variational bounds on the conditional probability using the convexity of the logarithm function. In particular, we use a variational treatment based on the approach of Jaakkola and Jordan (2000). 
-This approach consists of approximating the likelihood function of the logistic regression, governed by the sigmoid function, by the exponential of the a quadratic form, leading to a gaussian approximation of the posterior distribution. More explicitly, if $y\in \{-1,1\}$ is a target variable for a data vector $x$ then the likelihood function of the target variable $y$ is: 
+This approach consists of approximating the likelihood function of the logistic regression, governed by the sigmoid function, by the exponential of the quadratic form, leading to a gaussian approximation of the posterior distribution. More explicitly, if $y\in \{-1,1\}$ is a target variable for a data vector $x$ then the likelihood function of the target variable $y$ is: 
 \begin{equation}
 p(y | x, w)=\sigma(y w^T x)
 \end{equation}