Question

The transition matrix of a Markow chan s={1, 2, 3, 4, 5} is given by: ro.5 0.5 ooo7 1 0.25 0.75 0 0 0 1 0.2 0.2 0.2 0.2 0.2 10.25 0.25 0.25 0.25 LO 0 0 0 1 A) Determine equivalence classes B) For each class determine its perlod city, and if their states are cument or transient c) Calculate P{Xi2 =2 / X 16-4}

Answer 1

خا ن م Transition matrix of a Markov Chain for states 1, 2, 3, 4, 5 is given by r0.5 0.5 o o o re 2 0.25 0.75 o ooo 3 0.2 0.2 0.2 0.2 0.2 4 0.25 o 0.25 0.25 0.25 SL o o o o la Diagram of the given transition mateix یک ۱ 0.5 0.5 o s 2015 20.75 0.25 0.25 0. to.2 X0.25 0 .2 9 0.2 0125 ie, there is a way from 12 and 2 1, 151 and 242, 31, 32, 33, 34, 3-75, 471, 43, 44, 45, and sts. low, equivalence classes C(1) = {1, 2} = C(2) as 12 and 2-1, 1712-2 C(3) = {3, 43=C(4) as 3 4 and 4 3,3 33,474. (15= {5} , as 505 ②

(8) Periodicity. The god of no. of steps to come ith state from th state For state 1, l, = ged { 1, 2, 3, 4,...} = 1 for state 2, de-ged {1, 2, 3, 4,... 3 = 1 for state 3, ₂ - Jed {1,2,3,... 3 = 1 for state 4, 4-ged & 1, 2, 3 ... ? = 1 for states, es gold 1,2,.. 3 = 1 Period of each state is 1. Chain is a periodic. Trasient and recurrent condition - In a simple way, from diagram, if we go to another state from a state and we are able to come back from state, then that state is called recurrent state, otherwise transient state. Here, if we go to 2 from 1 then we are able to comes back to 1. Then 1 and 2 are recurrent states Now, take 3, if we go to 1 from 3 then there is a way to go 1 from 3 but we are not able to Come back from 1 to 3. Hence, 3 is a transient state, Also, 4 is a transient state, as if we go to 1 from 4 than we are not able to come back from 1 to 4. Now, s is a recurrent state , as we are not not able. to go outside. If there is a way to 1 from 5 and no way to come 5, then in that condition, 5 is transient. But here, howay is going outside to other states from Lence, Sis a recurrent state.

(c) P{X/2 = 2/X10 = 44 means probability of going to state 2 from 4 in (12-10)=2 steps, There are only two ways to go state 2 from 4.. @ 4 12 and 6 4 3 2 here, probability of condition 1 = 0-25 X0.5 = x probability of condition 2= 0.25X0-2 = 4x Jo Now, P[ Xyz=2410-43= 670 Dots - 28 P& Xx=24o=1} = (4.) = 0,175 X 20 160