Question

part b & c， please

A mouse is let loose in the maze of Figure 1. From each compartment the mouse chooses one of the adjacent compartments with equal probability, independent of the past. The mouse spends 1 time unit in each compartment. Let {Xn, n 〉 0} be the Markov chain that describes the position of the mouse for times n 2 0 4 Figure 1: A maze (a) Determine the one-step transition matrix (b) Use matrix multiplication on a computer to evaluate the probability that the mouse will be in compartment 6 at time n 7, starting from compartment 4 at time n 0 (c) What is the probability that the mouse will be found in compartment 6 at some time n far away in the future?

Answer 1

a. The one step TPM is

0 0.5 0.5 0 00 0.5 0 0 0.5 00 0 0 0.33 0 0.33 0.34 0 00 0.5 0 0.5 0 0 0 0.5 0.5 0

b.

P=matrix(c(0,0.5,0.5,0,0,0,1,0,0,0,0,0,0.5,0,0,0.5,0,0,0,0,0.33,0,0.33,0.34,0,0,0,0.5,0,0.5,0,0,0,0.5,0.5,0),nrow=6,byrow=T)
In = matrix(c(0,0,0,1,0,0),nrow=1,byrow=T)
P7 = P%^%7
X = In%*%P7
X

0.055275 0.1082812 0.2171586 0.2198438 0.1979961 0.2014454

Starting from compartment 4 at n=0, the probability that the mouse will be in compartment 6 at time n=7 is 0.2014454

c. To find the steady state of the Markov chain we need to obtain a vector U such that UP = U  

To solve the system of equation with constrain that sum of elements of U is 1.

The following R code is used to solve for U

A = t(P - diag(6))
A = rbind(A, rep(1,6))
b = c(rep(0,6),1)

U = qr.solve(A,b)
U

0.16541353 0.08270677 0.16541353 0.25062657   0.16708438 0.16875522

The probability that the mouse will be found in compartment 6 at some time n far away in the future is 0.16875522