Question

Problem 2. Tlaloc has 4 umbrellas, each either t home or at work. Each time he goes to work or back, it rains with probability p, independently of all other times. If it rains and there is at least one umbrella with him, he takes it. Otherwise, he gets et. Let Xn be the number of umbrellas at his current location after n trips (so n even corresponds to home and n odd to work. a) Find the transition probabilities for the Markov chain Xn. (b) Show that it is reversible and find the stationary distribution (c) What is the probability that Tlaloc gets wet on any given trip? (d) What value of p maximizes this probability? (e) What happens if he has N umbrellas for N other than 4?

Answer 1

Answer:

Let Xn be the number of umbrellas at home at the beginning of day n. Given that number, Xn+1 depends only on whether it rains or not on that day and this is independent to everything else. Thus Xn is a Markov chain. The transition probabilities are given by: p if j O and i 1 pil-p) ifj-1. . . . , r-land İE(j-1,j+1) 1 2p(1 -p) if j 1,... , r 1and i j p(1 - p) if j r and i r-1 1- p(1 - p) if j r and i r The Markov chain is finite, irreducible, and aperiodic. It is then positive recurrent and admits a unique invariant distribution which is the solution of the balance equations Ttj = (1-2p(1-p))Tj + p(1-p)Tj+1 + p(1-p)지-1, j = 1, . . . , r-1 Re-writing these equations, we get 지=1/2 [TH1 + p(1-p)], j = 1, . . . , r-1 For Ttr we have

Using the value of π-1 in the balance equations we get iterating with respect to r we get. Tr-πτ-1-.. . Now notice that for j-0, π0-(1-p)T0 + p(1-prl e, π0-(1-p)π| Using the fact the Tt is a probability distribution we get: And to (1 - p)Tt1 1- p/1 - p+ r Our man gets wet whenever there is no umbrella at home (wp TO) and it rains in the morning (wp p), and whenever it rains in the afternoon (w.p p) and all the umbrellas are at home (i.e. it did not rain in the morning) (w.p (1- p)rt) Thus Pr(gets wet) ptO (1- p)pT 2p(1 -p)/1 -p r Pr(p) (d) For r 3, we maximize this probability by taking the derivative and letting it equal to zero.