Question

3. Let Ya» . . . , Yn be independent normally distributed random variables with E(X) Gai and V(X)-1. Recall that the normal density with mean μ and variance σ given by TO 202 (a) Find the maximum likelihood estimator β of β (b) Show that ß is unbiased. (c) Determine the distribution of β (d) Recall that the likelihood ratio test of Ho : θ 02] L1] L2] θ° is to θ0 against H1: θ reject Ho if L(e) for some k, where θ is the maximum likelihood estimator of θ. Show that the likelihood ratio test of Ho : β--As against H1: β Ro is equivalent to rejecting Ho if for some constant k, which may depend on z1,... ,xn 4] (e) Specify k so that the likelihood ratio test has size o

Answer 1

$(a)~The~likelihood~function~of~\beta=L(\beta)=\frac{1}{(2\pi \sigma^2)^{n/2}}\exp\left(-\frac{1}{2\sigma^2}\sum_{i=1}^n(y_i-\beta x_i)^2\right)\\ \log L(\beta)=l(\beta)=-\frac{n}{2}\log(2\pi \sigma^2)-\frac{1}{2\sigma^2}\sum_{i=1}^n(y_i-\beta x_i)^2\\ \frac{\mathrm{d} l(\beta)}{\mathrm{d} \beta}=\frac{1}{2\sigma^2}\sum_{i=1}^n(y_i-\beta x_i)x_i=0\Rightarrow \hat{\beta}=\frac{\sum_{i=1}^nx_iy_i}{\sum_{i=1}^nx_i^2}\\ \frac{\mathrm{d}^2 l(\beta)}{\mathrm{d} \beta^2}=-\frac{1}{\sigma^2}\sum_{i=1}^nx_i^2<0~for~\beta=\hat{\beta}.\\$

$Hence~MLE~of~\beta=\frac{\sum_{i=1}^nx_iy_i}{\sum_{i=1}^nx_i^2}.\\ (b)~E(\hat{\beta})=\frac{1}{\sum_{i=1}^nx_i^2}\sum_{i=1}^nx_iE(y_i)=\frac{1}{\sum_{i=1}^nx_i^2}\sum_{i=1}^nx_i^2\beta=\beta.\\ Hence~\hat{\beta}~is~MLE~of~\beta.\\ (c)~Var(\hat{\beta})=\frac{1}{(\sum_{i=1}^nx_i^2)^2}\sum_{i=1}^nx_i^2\\~since~y_i's~are~independent~and~Var(y_i)=1~\forall~i.\\ =\frac{1}{\sum_{i=1}^nx_i^2}.\\$

$Again,~\hat{\beta}~is~a~linear~function~of~n~independent~y_i's~so\\ \hat{\beta}\sim N\left(\beta,~\frac{1}{\sum_{i=1}^nx_i^2}\right ).\\ (d)~\\ \lambda=\frac{L(\theta_0)}{L(\hat{\theta})}=\exp\left(\frac{1}{2}\sum_{i=1}^n\left((y_i-\hat{\beta}x_i)^2-(y_i-\beta_0x_i)^2 \right ) \right );~since~\sigma^2=1\\ =\exp\left((\beta_0-\hat{\beta})\left(\sum_{i=1}^nx_iy_i -\frac{\hat{\beta}&plus;\beta_0}{2}\sum_{i=1}^nx_i^2\right )\right)=\exp\left((\beta_0-\hat{\beta})\left(\hat{\beta}-\frac{\hat{\beta}&plus;\beta_0}{2}\right )\sum_{i=1}^nx_i^2\right),\\ ~since~\hat{\beta}=\frac{\sum_{i=1}^nx_iy_i}{\sum_{i=1}^nx_i^2}\\$

$=\exp\left(-\frac{1}{2}(\beta_0-\hat{\beta})^2\sum_{i=1}^nx_i^2\right)\leq k\\ or,~(\hat{\beta}-\beta_0)^2 \geq c\\ were~c~is~obtained~as\\ P\left((\hat{\beta}-\beta_0)^2\geq c|H_0\right )=\alpha\\ (e)~since~\sqrt{\sum_{i=1}^nx_i^2}(\hat{\beta}-\beta_0)\sim N(0,1)~underH_0~then\\$

$\sum_{i=1}^nx_i^2(\hat{\beta}-\beta_0)^2\sim \chi^2_1.\\ Hence~c=\chi^2_{\alpha,1}$