Question

Consider a two state Markov chain with one-step transition matrix on the states 1,21, , 0<p+q<2. 91-9 ' Show, by induction or otherwise, that the n-step transition matrix is Ptg -99 Based upon the above equation, what is lim-x P(Xn-2K-1). How about limn→x P(Xn-

Answer 1

Using induction, we first prove that the equation holds for n = 1. Then we assume that if the equation holds for n = k, then it also holds for n = k+1.

$P^n = \frac{1}{p&plus;q} \begin{pmatrix} q & p\\ q & p \end{pmatrix} &plus; \frac{(1 - p - q)^n}{p&plus;q}\begin{pmatrix} p &-p \\ -q & q \end{pmatrix}$

As, $n \rightarrow \infty$ , $(1 - p - q )^n \rightarrow 0$ because, p + q < 2 => -1 < (1 - p - q) < 1

Thus,

$\lim_{n \to \infty }P^n = \frac{1}{p&plus;q} \begin{pmatrix} q & p\\ q & p \end{pmatrix}$

So,

$\lim_{n \to \infty }P(X_n = 2 | X_0 = 1) = p/(p&plus;q)$

$\lim_{n \to \infty }P(X_n = 2 | X_0 = 2) = p/(p&plus;q)$

We were unable to transcribe this image

linn Fn = p+q(q P) 1

lim P(X,-21Xp = 1) = p/(p+ q)

lim P(X,-21Xp = 2) = p/(p+ q)