Question

Question 3 (12 points 3+5+ 4) not applicable to this year's exam A memoryless source emits messages coming from the set M- mi, m2, mzm4msmej. Every moment of signaling, the probability of emission of each message are 0.25, 0.23, 0.07,0.11, 0.24 and 0.1;. (a) Find the entropy of the source (b) Find a code prefix conditioned to encode the source. (c) Find your code efficiency

Answer 1

(a) As we know that entropy of coding can be obtained from the below formula:

(a) As we know that entropy of coding can be obtained from the below formula: H (z) = -p_m1 log (p_m1) - p_m2 log (p_m2) - p_m3 log (p_m3) - p_m4 log (p_m4) - p_m5 log (p_m5) - p_m6 log (p_m6) H (z) = -0.25 log (0.25) - 0.23 log (0.23) - 0.07 log (0.07) - 0.11 log (0.11) - 0.24 log (0.24) - 0.1 log (0.1) H (z) = 0.729 bits (b) codec prefix can be found using Huffman coding process length m_1 = 2, m_2 = 2, m_3 = 4 m_4 = 3, m_5 = 2, m_6 = 4 now L (3) = sigma^n_i = 1 L (m_i) p (m_i) = 0.25 times 2 + 0.23 times 2 + 0.07 times 4 + 0.11 times 3 + 0.24 times 2 + 0.1 times 4 = 0.5 + 0.46 + 0.28 + 0.33 + 0.48 + 0.4 = 2.45 bits

(b)

(c) Coding efficiency is given by