Question

BTGE2033 ENGINEERING STATISTICS Q4. (Continued) (d) A stochastic process can be in one of three possible states: state 1, state 2 and state 3. The transition probability from state i to state j is represented by the entry at row i column j of the following 1-step transition matrix 0.4 0.3 0.3 P0 0.4 0.6 0.6 0 0.4, (0.7, 0, 0.3) and the probability The initial probability distribution of the states is vo distribution of the states after n steps of transition is represented by Vn (3 marks) (i) Calculate v2 (ii) Calculate the steady-state probability distribution of the states, ve, which is (4 marks) attained after a very big number of transitions. Total: 25 marks

Answer 1

(i)

$P = \begin{pmatrix} 0.4 & 0.3 & 0.3\\ 0 & 0.4&0.6 \\ 0.6& 0& 0.4 \end{pmatrix}$

$P^2 = \begin{pmatrix} 0.4 & 0.3 & 0.3\\ 0 & 0.4&0.6 \\ 0.6& 0& 0.4 \end{pmatrix}\begin{pmatrix} 0.4 & 0.3 & 0.3\\ 0 & 0.4&0.6 \\ 0.6& 0& 0.4 \end{pmatrix} = \begin{pmatrix} 0.34& 0.24& 0.42 \\ 0.36 &0.16 &0.48 \\ 0.48 &0.18 &0.34 \end{pmatrix}$

$v_2 = v_0 P^2 = \begin{pmatrix} 0.7 & 0 & 0.3 \end{pmatrix}\begin{pmatrix} 0.34& 0.24& 0.42 \\ 0.36 &0.16 &0.48 \\ 0.48 &0.18 &0.34 \end{pmatrix} = \begin{pmatrix} 0.382 &0.222 &0.396 \end{pmatrix}$

(ii)

For steady state probability distributions,

$v_{\infty } P = v_{\infty }$

Let $v_{\infty }$ = (a, b, c)

then,

a + b + c = 1

0.4a + 0.6c = a => 0.6c = 0.6a => a = c

0.3a + 0.4b = b => 0.3a = 0.6b => a = 2b => c = 2b

0.3a + 0.6b + 0.4c = c => 0.3a + 0.6b - 0.6c = 0

Now,

a + b + c = 1

=> 2b + b + 2b = 1

=> 5b = 1

=> b = 0.2

a = c = 2b = 2 * 0.2 = 0.4

Thus, the steady state probability distributions is,

$v_{\infty }$ = (0.4, 0.2, 0.4)