Question

ematics of Discrete-Time Markov Chaill Develop a Markov chain model for each of the following situations. Assume that the process is oh after each play and that Pw 0.4. Find the transient probabilities for 10 plays as well as the state and absorbing state probabilities when appropriate. (a) For steady- the given situation, let the states be the cash supply: S0, 10, 20, 30, and 40. In addition , find the first passage probabilities from the initial state to the state $0, and also to the state $40, her distraught daughter. $40. (b) Change the model such that whenever Ms. Kitchell reaches S0, she borrows another $ 20 from (c) Change the model as in part (b) but with the provision that the most Ms. Kitchell can borrow is

Answer 1

normal markov chain implies that the likelihood of going from one state to each other state is non - zero in some limited number of steps

so to demonstrate that a given markov chain is normal, we are going to ascertain p^2,p^3, etc until the point when we see each component of the network non zero.

p^2=p.p=[ 0.5000 0.3333 0.1667

0.3750 0.5000 0.1250

0.7500 0 0.2500]

still one term is zero

along these lines, we should attempt once more

p^3=p.p^2= [0.5000 0.3333 0.1667

0.5625 0.2500 0.1875

0.3750 0.5000 0.1250]

so each term is non zero, henceforth we can say that the markov chain is standard for 3 stages.

b.)

to figure likelihood of moving between various states state in 2 stages , we should discover the 2 stage change framework which is p^2.

so p^2 is as appeared

the likelihood of going from state 1 t 3 of every two stages will be(p^2)13 ie the component in line 1 and segment 3.

c.) the constraining vector

the constraining vector is a line vector which upon augmentation with the tansition likelihood grid gives indistinguishable vector from itself and entirety of its components ought to be equivalent to 1.

in this way, w.P=w

or then again [w1 w2 w3].[1/2 1/3 1/6

3/4 0 1/4

0 1 0]=[w1 w2 w3]

we get three direct conditions from this network condition which are homogenius

furthermore, the fourth condition is w1+w2+w3=1

after settling these fourth condition, you will get the outcome.

this is it.

I surmise you were not requesting that how unravel these direct conditions or discover framework square or shape cuz that should be possible utilizing any logical adding machine.