Question

9. Flat distribution, high entropy. Consider the following possible distributions you might observe in N = 4 flips of a coin: (Ph.pt) = (0,1), (1/4, 3/4), (1/2,1/2), (3/4, 1/4), (1,0). Compute W for each case, and show that the flattest distribution has the highest multiplicity. JN Kapu ples Exce mar RD Levi mal

Answer 1

If we compute W(i)=- p(i) * ln (p(i)) for i=1,2 (head or tail) and compute sum of W(i) over all i to get W, then that gives an indication of multiplicity.

So for

(0,1) W=0

(¼, ¾ ) W= 0.34657 + 0.21576 = 0.56234

( ½ , ½) W= 0.346574 + 0.346574 = 0. 693147

( ¾ , ¼ ) W= 0.34657 + 0.21576 = 0.56234

(1,0) W=0

Thus we can see maximum value of W is 0.69 roughly when the distribution is (½ , ½), thus implying the flattest distribution has the highest multiplicity.