| course |
course_year |
question_number |
tags |
title |
year |
Markov Chains |
IB |
44 |
|
Paper 2, Section II, 18H |
2021 |
Let $P$ be a transition matrix on state space $I$. What does it mean for a distribution $\pi$ to be an invariant distribution? What does it mean for $\pi$ and $P$ to be in detailed balance? Show that if $\pi$ and $P$ are in detailed balance, then $\pi$ is an invariant distribution.
(a) Assuming that an invariant distribution exists, state the relationship between this and
(i) the expected return time to a state $i$;
(ii) the expected time spent in a state $i$ between visits to a state $k$.
(b) Let $\left(X_{n}\right){n \geqslant 0}$ be a Markov chain with transition matrix $P=\left(p{i j}\right)_{i, j \in I}$ where $I={0,1,2, \ldots}$. The transition probabilities are given for $i \geqslant 1$ by
$$p_{i j}= \begin{cases}q^{-(i+2)} & \text { if } j=i+1, \ q^{-i} & \text { if } j=i-1 \ 1-q^{-(i+2)}-q^{-i} & \text { if } j=i\end{cases}$$
where $q \geqslant 2$. For $p \in(0,1]$ let $p_{01}=p=1-p_{00}$. Compute the following, justifying your answers:
(i) The expected time spent in states ${2,4,6, \ldots}$ between visits to state 1 ;
(ii) The expected time taken to return to state 1 , starting from 1 ;
(iii) The expected time taken to hit state 0 starting from $1 .$