Some issue about Problem 4.26 solution

[Heng-Zhou]

Hi, Zhengqi, as usual, I learnt a lot from your solution, like the one to exercise 4.26 which uses a different approach, simple and elegant compared to taking differentiation and completing a square as hinted. But there are some problems in the exposition, which I will amend as follows.

First, a small suggestion for $P(X\le x)=\Phi\left(\frac{x-\mu}{\sigma}\right)$ if $X\sim\mathcal N(X|\mu,\sigma^2)$. We can arrive at this statement directly by properties of random variable in theory of probability, instead of integrals and change of variable. This can be seen by noting that $\frac{X-\mu}{\sigma}$ conforms to standard Gaussian distribution. As a result, $P(X\le x)=P(\frac{X-\mu}{\sigma}\le\frac{x-\mu}{\sigma})$ which is equal to $\Phi\left(\frac{x-\mu}{\sigma}\right)$ by definition of $\Phi$.

Then come the major problem of the solution where Bayesian Formula is applied. After defining two auxiliary random variables $X\sim\mathcal N(0,\lambda^{-2})$ and $Y\sim\mathcal N(\mu,\sigma^2)$, we need to further assume that these two random variables are independent. Otherwise, the equation $$P(X\le Y|Y=a)=P(X\le a)$$ would be dubious. This is because the left hand is actually a probability based on conditional pdf $p_{X|Y}(x|y)$ while the right hand is a probability based on marginal pdf $p_X(x)$. They are in general not equal. In other words, if $X$ and $Y$ are not uncorrelated, given $Y=a$, $X$ is another (Gaussian) distribution whose parameters (mean and covariance) are not necessarily still 0 and $\lambda^{-2}$, respectively. Consequently, the above equation does not hold in general. We have only the latter part of the line: $P(X\le a)=\Phi(\lambda a)$. Such an independence assumption is valid because we can make $X$ and $Y$ independent by linear transformations if they are not.

Next, we use multiple integral to write $$P(X\le Y)=\int_{-\infty}^{+\infty}\left(\int_{-\infty}^yp(x,y)dx\right)dy=\int_{-\infty}^{+\infty}\left(\int_{-\infty}^yp(x|y)p(y)dx\right)dy=\int_{-\infty}^{+\infty}\left(\int_{-\infty}^yp(x|y)dx\right)p(y)dy.$$

Since $X$ and $Y$ are independent, we have $p(x|y)=p(x)$. So, the internal integral reduces to $$\int_{-\infty}^yp(x|y)dx=\int_{-\infty}^yp(x)dx=P(X\le y)=\Phi(\lambda y),$$ and in turn the above integral is $$P(X\le Y)=\int_{-\infty}^{+\infty}\Phi(\lambda y)p(y)dy=\int_{-\infty}^{+\infty}\Phi(\lambda y)\mathcal N(y|\mu,\sigma^2)dy.$$ Changing integral variable from $y$ to $a$, we will obtain the left side of equation (4.152).

For the right side of equation (4.152), it should be noted in particular that $X-Y$ is Gaussian only if $X$ and $Y$ are independent. See, e.g., this StackExchange thread. This is the second place to demonstrate the importance of independence assumption. Under this assumption, the argument of the original solution follows and we get $P(X-Y\le0)=\Phi\left(\frac{\mu}{(\lambda^{-2}+\sigma^2)^{1/2}}\right)$, which concludes the whole proof.