Question

Problem 4: Maze A mouse travels in a maze (shown in figure). At each discrete time-step, the mouse chooses one of the doors from the room it is curently in (uniformly at random), and moves to the chosen neighboring room. Room three has a block of cheese in it (reward for the mouse (a) Model the location of mouse as a DTMC. Ist irreducible and aperiodic? Justify your answers b) Write the one-step probability transition matrix (c) Find the steady state distribution π for this DTMC. 3 4 cheese 5 Figure 1: The maze. (The steady state distribution is defined as a row vector π of probabilities such that π πΡ and Σηι π(i) 1 where P is the one-step transition matrix and n is the nurnber of states.)

Answer 1

Given protess ess is Discrete time markovs chain t01 State spaie transition pyo bability mahr ransIHo) 3 3 2. 0 3 The ransition diaąran From ransiHon araph cn 四乡 Gven morkove chain is irredudible

(m a)| state are penodic c) Guppase 12 2 2ーナ 13 and 5 From and From

3 Therefore
above solution is correct with full explaination so please rate me high.