2004-84.md

course	course_year	question_number	tags	title	year
Markov Chains	IB	84	IB 2004 Markov Chains	1.II.22H	2004

Consider the Markov chain with state space ${1,2,3,4,5,6}$ and transition matrix

$$\left(\begin{array}{cccccc} 0 & 0 & \frac{1}{2} & 0 & 0 & \frac{1}{2} \\ \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & 0 \\ \frac{1}{3} & 0 & \frac{1}{3} & 0 & 0 & \frac{1}{3} \\ \frac{1}{6} & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} \\ 0 & 0 & 0 & 0 & 1 & 0 \\ \frac{1}{4} & 0 & \frac{1}{2} & 0 & 0 & \frac{1}{4} \end{array}\right) \text {. }$$

Determine the communicating classes of the chain, and for each class indicate whether it is open or closed.

Suppose that the chain starts in state 2 ; determine the probability that it ever reaches state 6 .

Suppose that the chain starts in state 3 ; determine the probability that it is in state 6 after exactly $n$ transitions, $n \geqslant 1$.