2011-44.md

course	course_year	question_number	tags	title	year
Markov Chains	IB	44	IB 2011 Markov Chains	Paper 4, Section I, H	2011

Let $\left(X_{n}\right){n \geqslant 0}$ be a Markov chain on a state space $S$, and let $p{i j}(n)=\mathbb{P}\left(X_{n}=j \mid X_{0}=i\right)$.

(i) What does the term communicating class mean in terms of this chain?

(ii) Show that $p_{i i}(m+n) \geqslant p_{i j}(m) p_{j i}(n)$.

(iii) The period $d_{i}$ of a state $i$ is defined to be

$$d_{i}=\operatorname{gcd}\left{n \geqslant 1: p_{i i}(n)>0\right}$$

Show that if $i$ and $j$ are in the same communicating class and $p_{j j}(r)>0$, then $d_{i}$ divides $r$.