Question

Suppose that (X1, X2,,,,Xn) are iid random variables. Find the maximum likelihood estimator of theta for the following distributions

1) Poi(theta)

2) N(Mu, theta)

3) Exp(theta)

Answer 1

(1)

The pdf of Poisson distribution is

Here we have

.

.

.

So the likelihood function will be

Taking log of both sides gives

Differentiating both sides gives

  

Equating this equal to zero gives

That is required MLE is

(2)

The pdf of normal distribution is

f(x) =-1

Let X1, X2, ...Xn is a random sample from the normal distribution. So we have

.

.

.

The likelihood function is

Taken log of both sides gives

Differentiating above with respect to mu gives

Equating it to zero gives

$\sum_{i=1}^{n}(x_{i}-\mu)=0$

$\sum_{i=1}^{n}x_{i}-n\mu=0$

$\hat{\mu}=\frac{1}{n}\sum_{i=1}^{n}x_{i}=\bar{x}$

-------------------------

Differentiating above with respect to sigma gives

Equating it to zero gives

$heta^{2}=rac{sum_{i=1}^{n}(x_{i}-mu)^{2}}{n}$

Putting estimator of my gives

So required estimate is

(3)

From the definition of pdf we have

$f(x_{1})=\frac{1}{\theta} e^{-x_{1}/\theta}$

$f(x_{2})=\frac{1}{\theta} e^{-x_{2}/\theta}$

.

.

.

$f(x_{n})=\frac{1}{\theta} e^{-x_{n}/\theta}$

So the likelihood function will be

$L(\theta)=f(x_{1})f(x_{2})...f(x_{n})=\left [\frac{1}{\theta} e^{-x_{1}/\theta} \right ]\left [\frac{1}{\theta} e^{-x_{2}/\theta} \right ]...\left [\frac{1}{\theta} e^{-x_{n}/\theta} \right ]=\frac{1}{\theta^{n}} e^{-(x_{1}+x_{2}+...x_{n})/\theta}$ Taking log of both sides gives

$ln\left [L(\theta) \right ]=ln\left [\frac{1}{\theta^{n}} e^{-(x_{1}+x_{2}+...x_{n})/\theta} \right ]=ln(1)-nln(\theta)-\frac{\sum_{i=1}^{n}x_{i}}{\theta}$

Differentiating above with respect to $\theta$ gives

$\frac{d}{d\theta}ln\left [L(\theta) \right ]=-\frac{n}{\theta}+\frac{\sum_{i=1}^{n}x_{i}}{\theta^{2}}$

Equating above to zero gives

$-\frac{n}{\theta}+\frac{\sum_{i=1}^{n}x_{i}}{\theta^{2}}=0$

$\theta=\frac{\sum_{i=1}^{n} x_{i}}{n}=\bar{x}$

Hence, required estimate is

$\tilde{\theta}=\frac{\sum_{i=1}^{n} x_{i}}{n}=\bar{x}$.