Question

I need to know how to find out the following properties of a markov chain
1.Does this Markov chain have any absorbing states? Justify your answer.
2.Is state i accessible from state j (for all i and j)? Justify your answer.
3. Is this Markov chain irreducible? Justify your answer.
4. Is this Markov chain ergodic? Justify your answer.

Also I need help to simulate the markov chain. Can you send me a written response?

Answer 1

state hn Absorbing statii a Markovd chain such that Leave state thet it ismpasetble to example of Examples Anlabsarking Markor Chain the Drunkard' watk of lengte +2 the dnumkand i at In the drumknd halk ! intensection between thir Pub howe anol 1/½ 1Preb 2 Hom Pub and Home are abtorbing state Here State A ahoby tate Since it imporuble to leane state

Accessible We accesible that state i stati j written a if fens for some D. Excer ple Since There fore tates are accemble Since 2 3 2 12 So all tater are acceihle

to be A maskov chain said tredueible f all stutes communicate with each ether Communicate Two states omd arr sad to be communicate are Whitten as frem eact otten accessible Sn ether words means i 1 and 4i (C) Stete-AiB, C all tutes Since A BB A A C hen chaun sireduchle e

A markav chain is Af A)ALL etates are recwnint Chats diajran ls strengly conmected net teuddic Lie. aperiadic) Periad Note f 0 then abericdic Example 213 the chain i recurrent Becauie tatiog from Since that do LE1,213, amel from hererer yen can to refuring there Since 2 33 and Cteaperoe State 2 Engodie Markov chain is So

A markav chain is Af A)ALL etates are recwnint Chats diajran ls strengly conmected net teuddic Lie. aperiadic) Periad Note f 0 then abericdic Example 213 the chain i recurrent Becauie tatiog from Since that do LE1,213, amel from hererer yen can to refuring there Since 2 33 and Cteaperoe State 2 Engodie Markov chain is So

0 A markov chain is said to be irreducible if all states communicate with each other Communicate t Two states i and 2 are said to be communicate written as i o j, if they are accessible from each other. In other words. it i means i j and fi other words State - A,B,C since all states A B , B A - AC, CA then the chain is irreduable.

i To 2 Accessible : We say that state is accessible from state je written as j p i if foo for some n. Excemple since Since tho , tij Therefore states are accessible, since fill = 2 E = 0 b = x 20 are accessible. So all states

An Absorbing state is a state Markov chain such that PC Xx+= ixt=i) = 1 that is it is impossible to leave state & example of Example: 6 An absorbing Markor chain is the Drunkard's walk of length 12. In the drunkard's walk, the drunkard is at one of of an intersection between their house and Puki 3 Home Pub and Home are absorbing state. Be . There Since State es absorbing it is impossible to leave state. state &

& A markow chain hain is is a ergodic All state are recurrent that is diagram is strongly connected ) is not periodic. Cic. aperiodic) Period 1 Note: If k o Example: Yes then 3. aberiodic - since the chain is recurrent that is because strating from e = 1, 2, 3, and from wherever you can go, there is a way to returing 0- seis Since since krb. o - = 1 > 0 133 2 So state and are aperiat Markor chain is Ergodic. S.