2015-45.md

course	course_year	question_number	tags	title	year
Markov Chains	IB	45	IB 2015 Markov Chains	Paper 2, Section II, H	2015

(a) What does it mean for a transition matrix $P$ and a distribution $\lambda$ to be in detailed balance? Show that if $P$ and $\lambda$ are in detailed balance then $\lambda=\lambda P$.

(b) A mathematician owns $r$ bicycles, which she sometimes uses for her journey from the station to College in the morning and for the return journey in the evening. If it is fine weather when she starts a journey, and if there is a bicycle available at the current location, then she cycles; otherwise she takes the bus. Assume that with probability $p$, $0<p<1$, it is fine when she starts a journey, independently of all other journeys. Let $X_{n}$ denote the number of bicycles at the current location, just before the mathematician starts the $n$th journey.

(i) Show that $\left(X_{n} ; n \geqslant 0\right)$ is a Markov chain and write down its transition matrix.

(ii) Find the invariant distribution of the Markov chain.

(iii) Show that the Markov chain satisfies the necessary conditions for the convergence theorem for Markov chains and find the limiting probability that the mathematician's $n$th journey is by bicycle.

[Results from the course may be used without proof provided that they are clearly stated.]