2014-45.md

course	course_year	question_number	tags	title	year
Markov Chains	IB	45	IB 2014 Markov Chains	Paper 1, Section II, 20H	2014

Consider a homogeneous Markov chain $\left(X_{n}: n \geqslant 0\right)$ with state space $S$ and transition $\operatorname{matrix} P=\left(p_{i, j}: i, j \in S\right)$. For a state $i$, define the terms aperiodic, positive recurrent and ergodic.

Let $S={0,1,2, \ldots}$ and suppose that for $i \geqslant 1$ we have $p_{i, i-1}=1$ and

$$p_{0,0}=0, p_{0, j}=p q^{j-1}, j=1,2, \ldots,$$

where $p=1-q \in(0,1)$. Show that this Markov chain is irreducible.

Let $T_{0}=\inf \left{n \geqslant 1: X_{n}=0\right}$ be the first passage time to 0 . Find $\mathbb{P}\left(T_{0}=n \mid X_{0}=0\right)$ and show that state 0 is ergodic.

Find the invariant distribution $\pi$ for this Markov chain. Write down:

(i) the mean recurrence time for state $i, i \geqslant 1$;

(ii) $\lim {n \rightarrow \infty} \mathbb{P}\left(X{n} \neq 0 \mid X_{0}=0\right)$.

[Results from the course may be quoted without proof, provided they are clearly stated.]