Question

The random vector Y = (Y_{1, ...,} Y_n)^T is such that Y = Xβ + ε, where X is an n × p full-rank matrix of known constants, β is a p-length vector of unknown parameters, and ε is an n-length vector of random variables. A multiple linear regression model is fitted to the data.
(a) Write down the multiple linear regression model assumptions in matrix format.
(b) Derive the least squares estimator β^ of β.
(c) Using the data:
Y =(11 17 25),  X =(1 3 1 7 1 10)
calculate β^.
(d) Obtain the expectation and variance of β^.
(e) Write down the expression of the hat matrix H, and explain why hat matrix is important. Verify that the matrix H is symmetric, idempotent and of rank p.
(f) Find H for the data in part (c).

The random vector Y = (Y1,..., Y.)" is such that Y = X3 + €, where X is an n x p full-rank matrix of known constants, B is a p-length vector of unknown parameters, and € is an n-length vector of random variables. A multiple linear regression model is fitted to the data. (a) Write down the multiple linear regression model assumptions in matrix format. (b) Derive the least squares estimator 8 of B. (c) Using the data: 11 17 25 Y = 1 1 3 7 1 10 X= calculate B. (d) Obtain the expectation and variance of B. (e) Write down the expression of the hat matrix H, and explain why hat matrix is important. Verify that the matrix H is symmetric, idempotent and of rank p. (f) Find H for the data in part (e).

Answer 1

Axl not px x Y = Xßte @) Linear Regression nX1 known. model assumptions: w Nn (9,*n), o Ey's are independently distributed, (xx) is of full rank, ie row rank of x. To find is estimator P of h : We minimize E:e & worth to get Å. Note Hat, 30's) = (x-xp) *(x-x85] B12BRY] CP*] 0-2xy + 2x'x & >> - 2 x'Y+ 2x*8 = Xx2 = XY : Ls estimator of ß is :( = (xx-'x'Y. y = 17 25 (3) xx 158 3 20 20 158 , (xx) -1 = 1 -20 20 3. X'Y= 53 402

- 20 53 Å = (x*x) 'x'Y 158 -20 3. (4.513) 116.027 (Aus) (2) ECÂ) x*x] - [6x)" x'Y] = (xx) *x' E(X) = (x*x)" x XP D(A) = var & D = D ((x*x) *x'Y) *xx'x' D(2)3399-8 x'j' = ()4x' D(E). X (x'x)" **x) \ x'x . (x*x)* 0² Expectation of  (3) variance of per DCÂ) = (2x8)+1 le) Hat matrix : H = =HY, Also it's simple to compute, + X(XX)X? It's important, because ŷ Predicted value of L ie, to find î as His idempotent & synner His Symonetrie: [x(xx)=x]' x(x'x) -Tx! His idempotent : H22 x[x'x)-lx'x(xx)"ye! = x(xx)'X'= H H.

rank (6) = bi rank (H) = trace (H) trace [ xxxx) xx trx*831(x80] =tr - tx (I) 42 (1) H= x(xx)*x' 31/158/74 -20/74 - 17 10 3 7 10 1-20/74 3/741 = - 98/74 -11/74 18/74 1/74 -42/74 10/74? 37 10 ē 65/74 21174 -12/74) Aus 21/74 25/74 28/74 - 12/74 28/74 -22/74