Question

1. Let X and Y b e random variables, with μΧ = E(X), μΥ = E(Y), σ炙= Var(X) and σ Var(Y)

Answer 1

$Q\equiv Q(b_0,b_1)=E(Y-b_0-b_1X)^2=\int_{-\infty}^\infty \int_{-\infty}^\infty(y-b_0-b_1x)^2 f(x,y)dxdy\\ where,~f(x,y)~isa~joint~distribution~ofX~and~Y.\\ Now,~minimize~Q(b_0,b_1)~with~respect~to~b_0~and~b_1~i.e.\\ \frac{\partial Q(b_0,b_1)}{\partial b_0}=0\Rightarrow -2\int_{-\infty}^\infty \int_{-\infty}^\infty(y-b_0-b_1x) f(x,y)dxdy=0\Rightarrow \\ E(Y)-b_0-b_1E(X)=0~or,~b_0=\mu_Y-b_1\mu_X.....(1)\\ \frac{\partial Q(b_0,b_1)}{\partial b_1}=0\Rightarrow -2\int_{-\infty}^\infty \int_{-\infty}^\infty(y-b_0-b_1x)x f(x,y)dxdy=0\Rightarrow \\ E(XY)-b_0E(X)-b_1E(X^2)=0~\\or,~E(XY)-\mu_X(\mu_Y-b_1\mu_X)-b_1 E(X^2)=0~(using~(1))\\ or,~\left(E(XY)-\mu_X\mu_Y \right )-b_1(E(X^2)-\mu_X^2)=0~or,~Cov(X,Y)-b_1Var(X)=0\\ or,~b_1=\frac{Cov(X,Y)}{\sigma_X^2}....(2)\\ From~(1)~and~(2),~b_0=\mu_Y-\frac{Cov(X,Y)}{\sigma_X^2}\mu_X\\$

$(b)~Q_{min}=E(Y-\mu_Y-b_1(X-\mu_X))^2=\sigma_Y^2+b_1^2\sigma_X^2-2b_1Cov(X,Y)\\ =\sigma_Y^2+\frac{Cov^2(X,Y)}{\sigma_X^2}-2\frac{Cov^2(X,Y)}{\sigma_X^2}=\sigma_X^2-\frac{Cov^2(X,Y)}{\sigma_X^2}$