Question

4 A Markov chain with state space {0, 1, 2, 3, 4, 5} has the following transition matrix: P= 1/2 1/2 0 0 0 0 1/4 3/4 0 0 0 0 0 0 1/3 2/3 0 0 0 0 1/2 1/2 0 0 1/16 1/16 1/4 1/4 1/4 1/8 1/6 1/6 1/6 1/12 1/12 1/3 1/6) (1) Starting from state 4, find the mean time spent in state 5 before absorption. (5 marks) (ii) Starting from state 4, find the absorption probability into {2, 3). (5 marks) (iii) Starting from state 5, find the absorption probability into {0,1}. (5 marks)

Answer 1

(i) Starting from 4 let the mean time spent in state 5 before absorptio be X. Also when in 5 let the mean time spent in 5 be Y before apsorption.

Now we have from state 4, the probability to get absorbed as the probability of going to any of the 4 states ( as all of them are absorbing )

1/16 + 1/16 + 1/4 + 1/4 = 1/8 + 1/2 = 5/8

Therefore, X = (5/8)*0 + (1/4)*X + (1/8)*(1 + Y)

(3/4)X = (1/8)*(1 + Y)

6X = 1 + Y

Also from state 5, we have here:

Abosrption probabilty = (1/6)*2 + (1/12)*2 = 1/2

Therefore Y = (1/2)*0 + (1/3)*X + (1/6)*(1 + Y)

Y = (X/3) + (1/6)*(1 + Y)

6Y = 2X + 1 + Y

5Y = 2X + 1

Putting Y = 6X - 1 in the above equation, we get here:

5(6X - 1) = 6X - 1

24X = 4

X = 1/6

Therefore the mean time spent in state 5 before absorption is given here as: 1/6

(ii) Starting from state 4, let the absorption probability into {2, 3} be X. Also starting from state 5 let the same probability be Y.

Then, from 4 we have here:
X = (1/8)*0 + (1/2)*1 + (1/4)*X + (1/8)*Y
8X = 4 + 2X + Y
Y = 6X - 4

Also from state 5, we have here:
Y = (1/3)*0 + (1/6)*1 + (1/3)*X + (1/6)*Y
6Y = 1 + 2X + Y
5Y = 1 + 2X
5(6X - 4) = 1 + 2X
28X = 1 + 20
X = 3/4 = 0.75

Therefore 0.75 is the required probability here.

(iii) From previous part, we saw X = 0.75, therefore
Y = 6X - 4 = 6*0.75 - 4 = 0.5

This is the probability that from state 5, it is abosrbed in {2, 3}

Therefore the probability of being absorbed in state {0, 1} from state 5 would be computed as: 1 - Y = 0.5

Therefore 0.5 is the required probability here.