Question

We flip a fair coin 5 times. Let A be the event that at least one T was flipped immediately after an H (i.e. the combination HT appears at least once in your sequence of flips). Use a Markov chain to compute P(A). Hint: Try using the following three states for your Markov chain: State 0: HT has not appeared yet and cannot appear in the next flip; State 1: HT has not appeared yet, but could appear in the next flip; State 2: HT has appeared at least once. Write down the transition matrix P, and express P(A) in terms of P^5

Answer 1

Let A denote the event that at least one T was flipped immediately after an H. It is given that a coin is flipped five times and consider the three states: State O: HT has not appeared yet and cannot appear in the next flip. State 1:HT has not appeared yet, but could appear in the next flip State 2: HT has appeared at least once. The transition diagram is represented below: Ö- Ô-0 0.5 0.5 Markov chain transition diagram The transition matrix is displayed below: 0 1 2 0 0.5 0.5 07 p-1 O 0. 50 210 0 1] P(A) = ps pa - px p 0.5 0.5 0 1 [0.5 0.5 0] - O 0.5 0. 5 0.5 0.5 LO 0 1] Lo 0 1 0.25 0.5 0.25) - O 0.25 0.75 TO 0 1

p = p xp (0.25 0.5 0.25 0.5 0.5 0] | = 0 0.25 0.750 0.5 0.5 (0 0 1 0 0 1 (0.125 0.375 0.5 10 0.125 0.875 pt = px p (0.125 0.375 0.5 0.5 0.5 07 = 0 0.125 0.875 0 0.5 0.5 O 0 1 0 0 1 0.0625 0.25 0.6875 0 0.0625 0.9375 0 0 1 ps = ptx p (0.0625 0 1 0 (0.03125 0 0 0.25 0.6875 0.5 0.5 01 0.0625 0.9375 x 0 0.5 0.5 0 1 0 0 1 0.15625 0.8125 0.03125 0.96875 0 1 The required transition matrix is, (0.03125 0.15625 0.8125 p = 0 0.03125 0.96875 0 0 1