Question

Consider the Markov chains with the following probability transition matrices: ar-(032) OP=(0503) a) P = 0.5 0.5 0.5 0.5 b) P = 0.5 1 1 0.5 0 OPEL c) P = 0 1 = 0 ( e) P = 0 d) P = WI-NI-NI- 11 Draw the transition diagram for each case and explain whether the Markov chain is irreducible and/or aperiodic.

Answer 1

Refer the attached pictures for the solution i gave as much information as possible for better understanding...

Thank you...

0.5 0.5 ca e.. a) pe Piz 0.5 0.5 J 21 P22 ) Consider a Markov chain with two possible ) arcs 1 and 2 Ex: P = Pix P12 = Piaz 0.5 00.5.bol 0.5 Transition bou Diagram: O 12 0.5 A Markov chain is irredecide if all states, Communicate with each other ald since states ① and ② communicate with each other they belong to same clan class = { State 1, State 280) hence it is irreducible Class, is acriodic since it has two self- Transitions p o re probability is more to enter into the state nos of Pr P127 P 22 ] 67 0.5 1 2 1 o Joint Pa I ht. Considee a possible states markov chain having two and to bor 0.5 Tondition 0.5 B Pransition ora Diagram :

and states has a Here also since Communicating class C Class 12 & state, State 2} this is an irreducible matrise And since state has a self transition the whole clars becomes, aperiodic Obel O pas 13 0 2 7 y Pu Piz Piz TOTO in 2015 415 [PB1 P2 P33 Consider this matrix with three possible States 1, 2 and 3 , 13 Pranaition Dieglan Here no estates belongs to 2 115 0 45 same class that means no State Communicate with others this is reducible And here all states have since they belong to individual make their classes apaiodic self transition clauses they causes they Tom

d) palio o ] » Tu Pelo [Pi For this matrix let en draw transition diogram with two possible states 0 and Imamilion ♡ ♡ Diogram Herc class 1 = { State, State 2} that Communicate with each other. Hence this f n matrix or markov's chain is irreducible since State 1 of state 2 does not contain Beef transitions This is periodic with period 2 e) pr (1207 10 / / 1 1/2 1/3 13 ) Consider three states possible for given markov's chain Stater, State2 , Sfate 3 12 12 12 Transition Coro Diogtam 113 e

Jo divide Poto classes we consider two States belonging to same clan if they Communicate with each other Class = { State I} Class 2 = { state 2, state 3} o smic Since class i containing state is not Communicating this maukov's Chain is - bio reducible When we talk about periodic or not du three states contain self transitions hence to they make their classes apauodec