Question

(3.) A fair six-sided die is rolled repeatedly. Let R denote the random variable representing the outcome of any particular roll. The following random variables are all discrete-time Markov chains. Specify the transition probabilities for each (as a check, make sure the row sums equal 1) (a) Xn, which represents the largest number obtained by the nth roll. (b) Yn, which represents the number of sixes obtained in n rolls.

Answer 1

Since R represents the random variable representing the outcome of any particular roll , hence R can take any value from 1 to 6

i.e R= {1,2,3,4,5,6}

(a) Let X_n-1 represents the largest number till (n-1)th roll and X_n represents the largest number till nth roll.

Then if X_n-1 = 1 , then X_n can take any value from 1 to 6 with equal probability 1/6 depending on the number obtained in the nth roll.

If X_n-1 = 2, then X_n cannot take 1 since 2>1. Hence X_n will be 2 if nth roll shows 1 or 2 but will be 3,4,5 or 6 if the dice shows 3,4,5 or 6 respectively. Hence X_n =2 with probability 2/6 and X_n = 3,4,5,6 with probability 1/6

If X_n-1 = 3, then X_n cannot take 1,2 since 3>1,2 . Hence X_n will be 3 if nth roll shows 1,2 or 3 but will be 4,5 or 6 if the dice shows 4,5 or 6 respectively. Hence X_n =3 with probability 3/6 and X_n = 4,5,6 with probability 1/6

Similarly proceeding if If X_n-1 = 6, then X_n cannot take any value other than 6 since 6>1,2,3,4,5. Hence X_n will be 6 with probability 1.

Hence our transition probability matrix in this case is

X_n

P = Xn-1  $\begin{bmatrix} 1/6 &1/6 &1/6 &1/6 &1/6 &1/6 \\ 0 &2/6 & 1/6 &1/6 &1/6 &1/6 \\ 0 &0 &3/6 &1/6 &1/6 &1/6 \\ 0&0 &0 &4/6 &1/6 &1/6 \\ 0& 0 &0 &0 &5/6 &1/6 \\ 0 &0 &0 &0 &0 & 1 \end{bmatrix}$

where (i,j)th element represents the probability of reaching state j in the nth step given the process is in state i in the (n-1)th step. For example 3/6 represents the probability that the process will be in state 3 in the nth role given it was in state 3 in (n-1) th roll . Similarly 2/6 represents the probability that the process will be in state 2 in the nth role given it was in state 2 in (n-1) th roll.  

(b) Let Y_n-1 represents the number of sixes obtained till (n-1)th roll and Y_n represents the number of sixes obtained till nth roll.

Then if Y_n-1 = 0 (0 sixes obtained till (n-1)th roll), then Y_n =1 with probability 1/6 (if 6 is obtained in nth roll) and Y_n = 0 with probability 5/6 (if 6 is not obtained)

Similarly if Y_n-1 = 1 (1 six obtained till (n-1)th roll), then Y_n =2 with probability 1/6 (if 6 is obtained in nth roll) and Y_n = 1 with probability 5/6 (if 6 is not obtained) .

Proceeding in this way we get  if Y_n-1 = n-1 (0 sixes obtained till (n-1)th roll), then Y_n =n with probability 1/6 (if 6 is obtained in nth roll) and Y_n = n-1 with probability 5/6 (if 6 is not obtained) .

So our transition probability matrix is given by