Question

1. Suppose there are 3 characters in a language L, and we have built a unigram model. The probabilities for the 3 characters are given by the models are P(“A”) = 0.25, P(“B”) = 0.50, and P(“C”) = 0.25. What will be the expression to calculate the perplexity of for the sequences “AAA” and “ABA”?

Answer 1

perplexity of sequence is given by following formula :

P(S) = ( 1 / (p(s₁)p(s₂)..p(s_n)) )^(1/n)

(i) For S₁ = "AAA", perplexity is :

      ( 1 / ( 0.25 * 0.25 * 0.25 ) )^(1/3)

      ( 1 / 0.25 ) =   4

     Perplexity for sequence "AAA" is 4

(ii) For S₂ = "ABA", perplexity is :

      ( 1 / ( 0.25 * 0.5 * 0.25 ) )^(1/3)

      ( 1 / 0.31498 ) =   3.1748

     Perplexity for sequence "ABA" is 3.1748

Note: If you have any doubts please comment.

It will be great help If you like.