Question

Question 3: A random variable X has a Bernoulli distribution with parameter θ є (0,1) if X {0,1} and P(X-1)-θ. Suppose that we have nd random variables y, x, following a Bernoulli(0) distribution and observed values y1,... . Jn a) Show that EIX) θ and Var[X] θ(1-0). b) Let θ = ỹ = (yit . .-+ yn)/n. Show that θ is unbiased for θ and compute its variance. c) Let θ-(yit . . . +yn + 1)/(n + 2) (this estimator comes from something called Laplace's rule d) Compute the risk (bias squared plus variance) of θ and Comment on the quality of the two Note: Bias is the difference between the expected value of an estimator and the parameter it is of succession). Show that is based but that it has less variance that θ estimators for θ. estimating

Answer 1

a)

E[X] = 0 * P(X = 0) + 1 * P(X = 1) = 1 * $\theta$ = $\theta$

E[X²] = 0² * P(X = 0) + 1² * P(X = 1) = 1 * $\theta$ = $\theta$

Var[X] = E[X²] - E[X]²  = $\theta$ - $\theta^2$ = $\theta(1 - \theta)$

b)

$E[\hat{\theta}] = E[\bar{y}] = E[y] = \theta$ (By central limit theorem, $E[\bar{y}] = E[y]$ )

Thus, $\hat{\theta}$ is unbiased estimator of $\theta$

$Var[\hat{\theta}] = Var[\bar{y}] = Var[y]/n = \theta(1- \theta)/n$

c)

$E[\tilde{\theta}] = E[(y_1+ ..+y_n+1)/(n+2)] = E[(y_1+ ..+y_n)+1]/(n+2)$

$= E[n((y_1+ ..+y_n)/n)+1]/(n+2) = E[n\bar{y}+1]/(n+2) = (nE[\bar{y}] + 1)/n+2$

$E[\tilde{\theta}] = (nE[y] + 1)/n+2 = (n \theta + 1)/(n+2)$

As, $E[\tilde{\theta}] \ne \theta$ , $\tilde{\theta}$ is biased estimator of $\theta$.

$Var[\tilde{\theta}] = Var[(y_1+ ..+y_n+1)/(n+2)] = Var[(y_1+ ..+y_n)+1]/(n+2)^2$

$= Var[n((y_1+ ..+y_n)/n)+1]/(n+2)^2 = Var[n\bar{y}+1]/(n+2)^2$

$= (n^2Var[\bar{y}])/(n+2)^2 = (n^2 Var[y]/n) / (n+2)^2 = n \theta (1- \theta)/(n+2)^2$

$Var[\tilde{\theta}] = n^2 \theta (1- \theta)/n(n+2)^2 = (n / n+2)^2 Var[\hat{\theta}]$

For n > 1, (n / n+2)² < 1

thus,

$Var[\tilde{\theta}] < Var[\hat{\theta}]$

d)

Bias for $\hat{\theta}$ is,

b = $E[\hat{\theta}] - \theta = \theta - \theta = 0$

Risk = b² + Var($\hat{\theta}$) = $\theta(1- \theta)/n$

Bias for $\tilde{\theta}$ is,

b = $E[\tilde{\theta}] - \theta =(n \theta &plus; 1)/(n&plus;2) - \theta = ( 1 - 2\theta)/(n&plus;2)$

Risk = b² + Var($\tilde{\theta}$) = $( 1 - 2\theta)^2/(n&plus;2)^2 &plus; n \theta (1- \theta)/(n&plus;2)^2 = (( 1 - 2\theta)^2 &plus;n \theta (1- \theta) )/(n&plus;2)^2$

For low values of n and large values of $\theta$, the risk for $\hat{\theta}$ will be larger than that of $\tilde{\theta}$, but for high values of n, the risk for $\hat{\theta}$ will be lesser than that of $\tilde{\theta}$.