2011-43.md

course	course_year	question_number	tags	title	year
Markov Chains	IB	43	IB 2011 Markov Chains	Paper 3, Section I, H	2011

Let $\left(X_{n}\right)_{n \geqslant 0}$ be a Markov chain with state space $S$.

(i) What does it mean to say that $\left(X_{n}\right)_{n} \geqslant 0$ has the strong Markov property? Your answer should include the definition of the term stopping time.

(ii) Show that

$$\mathbb{P}\left(X_{n}=i \text { at least } k \text { times } \mid X_{0}=i\right)=\left[\mathbb{P}\left(X_{n}=i \text { at least once } \mid X_{0}=i\right)\right]^{k}$$

for a state $i \in S$. You may use without proof the fact that $\left(X_{n}\right)_{n \geqslant 0}$ has the strong Markov property.