Question

Suppose that E h ˆθ1 i = E h ˆθ2 i = θ, Var h ˆθ1 i = σ 2 1 , Var h ˆθ2 i = σ 2 2 , and Cov h ˆθ1, ˆθ2 i = σ12. Consider the unbiased estimator ˆθ3 = aˆθ1 + (1 − a) ˆθ2. What value should be chosen for the constant a in order to minimize the variance and thus mean squared error of ˆθ3 as an estimator of θ?

Answer 1

$E(\hat{\theta}_1)=E(\hat{\theta}_2)=\theta,~Var(\hat{\theta}_1)=\sigma_1^2,~Var(\hat{\theta}_2)=\sigma_2^2,~Cov(\hat{\theta}_1,\hat{\theta}_2)=\sigma_{12}\\ Var(\hat{\theta}_3)=a^2Var(\hat{\theta}_1)+(1-a)^2Var(\hat{\theta}_2)+2a(1-a)Cov(\hat{\theta}_1,\hat{\theta}_2)=\Delta ,~say\\ \frac{\mathrm{d} \Delta }{\mathrm{d} a}=2aVar(\hat{\theta}_1)+2(1-a)Var(\hat{\theta}_2)+2(1-2a)Cov(\hat{\theta}_1,\hat{\theta}_2)\\ =2a\left(Var(\hat{\theta}_1) -Var(\hat{\theta}_2)-2Cov(\hat{\theta}_1,\hat{\theta}_2)\right )+2\left(Var(\hat{\theta}_2)+Cov(\hat{\theta}_1,\hat{\theta}_2) \right )=0\\ or,~a=\frac{\left(Var(\hat{\theta}_2)+Cov(\hat{\theta}_1,\hat{\theta}_2) \right )}{\left(Var(\hat{\theta}_2)+2Cov(\hat{\theta}_1,\hat{\theta}_2)-Var(\hat{\theta}_1) \right )}\\ \frac{\mathrm{d}^2 \Delta }{\mathrm{d} a^2}=2\left(Var(\hat{\theta}_1) -Var(\hat{\theta}_2)-2Cov(\hat{\theta}_1,\hat{\theta}_2)\right )>0\\ ~if~Var(\hat{\theta}_1)>Var(\hat{\theta}_2)+2Cov(\hat{\theta}_1,\hat{\theta}_2)\\$

$Hence~if~Var(\hat{\theta}_1)>Var(\hat{\theta}_2)+2Cov(\hat{\theta}_1,\hat{\theta}_2)\\ then~a=\frac{\left(Var(\hat{\theta}_2)+Cov(\hat{\theta}_1,\hat{\theta}_2) \right )}{\left(Var(\hat{\theta}_2)+2Cov(\hat{\theta}_1,\hat{\theta}_2)-Var(\hat{\theta}_1) \right )}~minimizes~\Delta.$