2021-41.md

course	course_year	question_number	tags	title	year
Markov Chains	IB	41	IB 2021 Markov Chains	Paper 3 , Section I, H	2021

Consider a Markov chain $\left(X_{n}\right)_{n \geqslant 0}$ on a state space $I$.

(a) Define the notion of a communicating class. What does it mean for a communicating class to be closed?

(b) Taking $I={1, \ldots, 6}$, find the communicating classes associated with the transition matrix $P$ given by

$$P=\left(\begin{array}{cccccc} 0 & 0 & 0 & 0 & \frac{1}{4} & \frac{3}{4} \\ \frac{1}{4} & 0 & 0 & 0 & \frac{1}{2} & \frac{1}{4} \\ 0 & \frac{1}{2} & 0 & \frac{1}{2} & 0 & 0 \\ 0 & \frac{1}{2} & 0 & 0 & \frac{1}{2} & 0 \\ \frac{1}{4} & \frac{1}{2} & 0 & 0 & 0 & \frac{1}{4} \\ 1 & 0 & 0 & 0 & 0 & 0 \end{array}\right)$$

and identify which are closed.

(c) Find the expected time for the Markov chain with transition matrix $P$ above to reach 6 starting from 1 .