Question

1) A day on a planet X can be in 2 states: sunny or rainy. The climate of planet X is determined by the following pattern: every sunny day is followed by a sunny day with a probability of 2/3, and every rainy day is followed by a rainy day with a probability of 1/3.

(a) Find the transition matrix that represents the above change in weather pattern.

(b) Find a steady state vector for the above Markov system.

2) Bill and Cam are playing the following game: each player has a coin and decides whether to leave it as heads or tails before showdown (both player reveals their coin simultaneously). If both coins are heads, Bill wins $2. If both are tails, Bill wins $0.50. Otherwise, Cam wins $1. Find the optimal strategy for Bill.

Answer 1

We would be looking at the first question here as:

Q1) We are given the probabilities here as:
P(Sunny --> Sunny) = 2/3, therefore P(sunny --> rainy) = 1 - (2/3) = 1/3

Also, P(rainy --> rainy) = 1/3 and therefore, P(rainy --> sunny) = 1 - (1/3) = 2/3

a) Therefore using the above probabilities the transition probability matrix here is given as:

$P = \begin{pmatrix} (2/3) &(1/3) \\ (2/3) & (1/3) \end{pmatrix}$

This is the required matrix here.

b) Let the steady state probabilities here be X, Y respectively for sunny and rainy days.

Therefore, we have here:
X + Y = 1

From first column, we have here:
X = (2/3)X + (2/3)Y
(1/3)X = (2/3)Y
X = 2Y

Therefore, 3Y = 1

Y = 1/3 and X = 2/3

Therefore the steady state vector here is given as:

$= \begin{pmatrix} 2/3 \\ 1/3 \end{pmatrix}$