Question

A total of 5 balls of two possible colours, black and white, have been put into a box. A ball is chosen uniformly at random. If the ball is black, we replace this ball in the box with a white ball. Similarly, if the ball chosen is white, we replace this ball in the box with a black one. Let {Xn,n 2 0) denote the number of white balls in the box after the n-th trial. This defines a Markov chain with state space S- 10,1, 5 (a) Determine the transition probabilities and draw the transition diagram; (b) Calculate PDG = 41X,-5];

Answer 1

(a)

Let there be m balls at the nth trial. Then Xn = m and number of black balls = 5 - m

For m = 1,2,3,4

Probability to choose a white ball (and replace by a black ball, so the count of white ball is reduced by 1) = m/5

Probability to choose a black ball (and replace by a white ball, so the count of white ball is increased by 1) = m/5

Thus,

P(Xn+1 = m-1 | PXn = m) = m/5

P(Xn+1 = m+1 | PXn = m) = (5 - m)/5

For m = 0,

Probability to choose a black ball (and replace by a white ball, so the count of white ball is increased by 1) = 1

P(Xn+1 = 1 | PXn = 0) = 1

For m = 5,

Probability to choose a white ball (and replace by a black ball, so the count of white ball is decreased by 1) = 1

P(Xn+1 = 4 | PXn = 5) = 1

The transition probability matrix is,

Xn \ Xn+1	0	1	2	3	4	5
0	0	1	0	0	0	0
1	1/5	0	4/5	0	0	0
2	0	2/5	0	3/5	0	0
3	0	0	3/5	0	2/5	0
4	0	0	0	4/5	0	1/5
5	0	0	0	0	1	0

The transition diagram is,

(b)

P[X2 = 4| X0 = 5] = P[X2 = 4, X1 = 4, X0 = 5] (As the transition probability from state 5 to state 4 is 1)

Now, from state 4, there is no transition again to the state 4 in one transition.

Thus,  P[X2 = 4| X0 = 5] = P[X2 = 4, X1 = 4, X0 = 5] = 0