2016-45.md

course	course_year	question_number	tags	title	year
Markov Chains	IB	45	IB 2016 Markov Chains	Paper 2, Section II, H	2016

(a) Prove that every open communicating class of a Markov chain is transient. Prove that every finite transient communicating class is open. Give an example of a Markov chain with an infinite transient closed communicating class.

(b) Consider a Markov chain $\left(X_{n}\right)_{n \geqslant 0}$ with state space ${a, b, c, d}$ and transition probabilities given by the matrix

$$P=\left(\begin{array}{cccc} 1 / 3 & 0 & 1 / 3 & 1 / 3 \\ 0 & 1 / 4 & 0 & 3 / 4 \\ 1 / 2 & 1 / 2 & 0 & 0 \\ 0 & 2 / 3 & 0 & 1 / 3 \end{array}\right)$$

(i) Compute $\mathbb{P}\left(X_{n}=b \mid X_{0}=d\right)$ for a fixed $n \geqslant 0$.

(ii) Compute $\mathbb{P}\left(X_{n}=c\right.$ for some $\left.n \geqslant 1 \mid X_{0}=a\right)$.

(iii) Show that $P^{n}$ converges as $n \rightarrow \infty$, and determine the limit.

[Results from lectures can be used without proof if stated carefully.]