Question

II. Derivations (You must show all your work for full credit.)

Answer 1

Let

X be a nXk matrix

Y,E and $\beta$ be nX1 vectors

X' is the transpose of X

$\beta$ ^ is the estimate for $\beta$

Now,

€1 Y1 Y2 1 X 11 X11 X21 1 X 12 X 22 Xk1 Xk2 B1 B2 €2 ... ... ... ... ... : + ... 1 Xa Xan Xin Bin nxl nxk kx 1 En nx1

or Y= X$\beta$+E

i. Using principle of ordinary least squares we minimise the sum of squared residuals :

E=Y- X$\beta$ (Residuals)

Sum of squared residuals: E'E=(Y- X$\beta$^)' (Y- X$\beta$^)= Y'Y-$\beta$^'X'Y-Y'X$\beta$^+$\beta$^'X'X$\beta$^ = Y'Y-2$\beta$^'X'Y+$\beta$^'X'X$\beta$^

To minimise, we partially take the derivative with $\beta$ ^

dele მ3 =—2X'ყ+2XxX = 0

Now,we get the normal equation as

(X'X)$\beta$^=X'Y

or $\beta$ ^=((X'X)^-1)*X'Y ---1

ii.

Taking expectations on both sides of equation 1

E( $\beta$ ^)=E(((X'X)^-1)*X'Y)

=((X'X)^-1)X' E(Y)= ((X'X)^-1)X' (X$\beta$+E)= ((X'X)^-1)X'X$\beta$= $\beta$ (E(E)=0)

Hence E( $\beta$ ^)= $\beta$

Therefore unbiasedness is proved

iii. Variance $\beta$ ^ = E(( $\beta$ ^-$\beta$)( $\beta$ ^-$\beta$)')= E( ((X'X)^-1)X'u)((X'X)^-1)X'u)')= E((X'X)^-1)X'uu'X(X'X)^-1)= (X'X)^-1)X'E(uu')X(X'X)^-1=(X'X)^-1)X'(sigma sq)X(X'X)^-1=(sigma sq)(X'X)^-1)X'X(X'X)^-1= (sigma sq)(X'X)^-1 which is the desired variance covariance matrix.