2010-44.md

course	course_year	question_number	tags	title	year
Markov Chains	IB	44	IB 2010 Markov Chains	Paper 4, Section I, E	2010

Consider a Markov chain $\left(X_{n}\right)_{n} \geqslant 0$ with state space ${a, b, c, d}$ and transition probabilities given by the following table.

\begin{tabular}{c|cccc} & $a$ & $b$ & $c$ & $d$ \ \hline$a$ & $1 / 4$ & $1 / 4$ & $1 / 2$ & 0 \ $b$ & 0 & $1 / 4$ & 0 & $3 / 4$ \ $c$ & $1 / 2$ & 0 & $1 / 4$ & $1 / 4$ \ $d$ & 0 & $1 / 2$ & 0 & $1 / 2$ \end{tabular}

By drawing an appropriate diagram, determine the communicating classes of the chain, and classify them as either open or closed. Compute the following transition and hitting probabilities:

• $\mathbb{P}\left(X_{n}=b \mid X_{0}=d\right)$ for a fixed $n \geqslant 0$

• $\mathbb{P}\left(X_{n}=c\right.$ for some $\left.n \geqslant 1 \mid X_{0}=a\right)$.