Question

Simple Bayesian Statistics problem:

It is agreed that Hamilton wrote 51 Federalist papers and Madison wrote 14 Federalist papers. However, there’s a dispute over how to attribute 12 other papers between these two authors. In Hamilton’s 51 papers, the word “upon” was used 3.24 times per 1000 words. In Madison’s 14 papers, the word “upon” was used 0.23 times per 1000 words.

a.) Based on the sample proportion of papers with known authorship, 51/(14+51) is a reasonable prior probability for a disputed paper to be attributed to Hamilton. Give a situation where this prior should be used over the prior that gives equal probability to both authors, and give another situation where the equal probability prior is referred.

b.) If the word “upon” is used 3 times in a disputed text of length 1,000 words and we assume the equal probability prior, what is the posterior probability the paper was written by Hamilton?

c.) Give one assumption you are making in part b that is likely unreasonable.

d.) In part b, if we change the number of “upon”s from 3 to 1, does the posterior probability increase, decrease, or stay the same?

e.) In part b, if we change the number of “upon”s to 30 and the text length to 10,000 words, does the posterior probability increase, decrease, or stay the same?

Answer 1

a) If we are supposed to predict the author of the papers based solely on the number of earlier papers already published and clearly identified, it might be appropriate to use the prior probability of =51/(14 + 51) as the prior probabilities of Hamilton. As an example, if the other 12 papers are also Federalist, we can use the above priors.

However, if we believe the number of earlier publications are not necessarily the indication of likelihood of writing another paper, with lack of other information we may believe both are equally likely to write a paper, we can choose an equal probability. This can happen if we are considering a paper in fairly new areas, (which the earlier papers are not covered), such as Non-Federalist areas, we should use equal probability due to lack of information.

b)

We are given,

$\mathbb{P}('upon' \: | Hamilton ) = 3.24 / 1000$

$\mathbb{P}('upon' \: | Madison ) \, = \, 0.23 / 1000$

$\mathbb{P}('upon' =3 ) \, = 3 / 1000$

$\mathbb{P}(Hamilton ) \, = 51 / (14+51)$

$\mathbb{P}(Madison ) \, = 14 / (14+51)$

Now we have to find $\mathbb{P}( Hamilton | 'upon'=3 ) = ?$

We know that $\mathbb{P}(A | B ) = \mathbb{P}(B | A ) \mathbb{P}(A) / \mathbb{P}( B )$ where A and B are events and $\mathbb{P}( B ) \neq 0$

Now to solve this problem we will take help of liner interpolation,

c) Here the assumption is the probability of the paper being written by Hamilton varies linearly with the observed frequency of the word "upon" in the neighborhood. This is unlikely to be true for values very far from the 3.24. We need to use a different distributional assumptions such as normal to interpolate the above probabilities. However, for that we would need the frequencies of "upon" for every document to calculate the standard deviation of the estimated normal distribution. The mean of the estimated normal distribution would be the observed total mean i.e. 3.24/1000 for Hamilton.