2010-43.md

course	course_year	question_number	tags	title	year
Markov Chains	IB	43	IB 2010 Markov Chains	Paper 3, Section I, E	2010

An intrepid tourist tries to ascend Springfield's famous infinite staircase on an icy day. When he takes a step with his right foot, he reaches the next stair with probability $1 / 2$, otherwise he falls down and instantly slides back to the bottom with probability $1 / 2$. Similarly, when he steps with his left foot, he reaches the next stair with probability $1 / 3$, or slides to the bottom with probability $2 / 3$. Assume that he always steps first with his right foot when he is at the bottom, and alternates feet as he ascends. Let $X_{n}$ be his position after his $n$th step, so that $X_{n}=i$ when he is on the stair $i, i=0,1,2, \ldots$, where 0 is the bottom stair.

(a) Specify the transition probabilities $p_{i j}$ for the Markov chain $\left(X_{n}\right)_{n} \geqslant 0$ for any $i, j \geqslant 0$.

(b) Find the equilibrium probabilities $\pi_{i}$, for $i \geqslant 0$. [Hint: $\left.\pi_{0}=5 / 9 .\right]$

(c) Argue that the chain is irreducible and aperiodic and evaluate the limit

$$\lim {n \rightarrow \infty} \mathbb{P}\left(X{n}=i\right)$$

for each $i \geqslant 0$.