Question

Hello, please use Markov process for the problem. Please make the explanations simple and understandable, I don't have a statistics background. Thank you!

4.10 On a given day Mark is cheerful, so-so, or glum. Given that he is cheerful on a given day, then he will be cheerful again the next day with probability 0.6, so-so with proba- bility 0.2, and glum with probability 0.2. Given that he is so-so on a given day, then he will be cheerful the next day with probability 0.3, so-so again with probability 0.5, and glum with probability 0.2. Given that he is glum on a given day, then he will be so-so the next day with probability 0.5, and glum again with probability 0.5. Let state 1 denote the cheerful state, state 2 denote the so-so state, and state 3 denote the glum state. Let Xn denote Mark's mood on the nth day, then (Xn,n=0, 1,2, ) is a three- state Markov chain. a. Draw the state-transition diagram of the process. b. Give the state-transition probability matrix. c. Given that Mark was so-so on Monday, what is the probability that he wl be cheerful on Wednesday and Friday and glum on Sunday? d. On the long run, what proportion of time is Mark in each of his three moods?

Answer 1

a.

The state transition diagram of the process is,

0.6 0.2 0.3 0.5 0.2 0.2 0.5 0.5

b.

The state transition diagram of the process is,

0.6 0.2 0.2 Р-10.3 0.5 0.2 0 0.5 0.5

c.

Probability that Mark is cheerful on Wednesday given he was so-so on Monday

= P(X2 = 1 | X0 = 2)

= P(X0 = 2, X1 = 1, X2 = 1) + P(X0 = 2, X1 = 2, X2 = 1) + P(X0 = 2, X1 = 3, X2 = 1)

= 0.3 * 0.6 + 0.5 * 0.3 + 0.2 * 0

= 0.33

Probability that Mark is cheerful on Friday given Mark is cheerful on Wednesday

= P(X2 = 1 | X0 = 1)

= P(X0 = 1, X1 = 1, X2 = 1) + P(X0 = 1, X1 = 2, X2 = 1) + P(X0 = 1, X1 = 3, X2 = 1)

= 0.6 * 0.6 + 0.2 * 0.6 + 0.2 * 0

= 0.48

Probability that Mark is glum on Sunday given Mark is cheerful on Friday

= P(X2 = 3 | X0 = 1)

= P(X0 = 1, X1 = 1, X2 = 3) +  P(X0 = 1, X1 = 2, X2 = 3) +  P(X0 = 1, X1 = 3, X2 = 3)  

= 0.6 * 0.2 + 0.2 * 0.2 + 0.2 * 0.5

= 0.26

Probability that Mark is cheerful on Wednesday and Friday and glum on Sunday given he was so-so on Monday

= Probability that Mark is cheerful on Wednesday given he was so-so on Monday * Probability that Mark is cheerful on Friday given Mark is cheerful on Wednesday * Probability that Mark is glum on Sunday given Mark is cheerful on Friday

= 0.33 * 0.48 * 0.26

= 0.041184

d.

Let $\pi$ = [a, b, c] be the steady state vector where a, b and c are the long run proportion of time Mark is in three modes.

Then $\pi$P = $\pi$

and a + b + c = 1

which gives,

0.6a + 0.3b = a => -0.4a + 0.3b = 0

0.2a + 0.5b + 0.5c = b => 0.2a - 0.5b + 0.5c = 0

0.2a + 0.2b + 0.5c = c => 0.2a + 0.2b - 0.5c = 0

and a + b + c = 1

Solving these equations we get,

a = 0.3061224, b = 0.4081633 c =  0.2857143