Question

7. What is multicollinearity and why is it a problem? What techniques would you use to detect multicollinearity? Give an example to explain your answer

Answer 1

1) What is Multicollinearity?

In statistics, multicollinearity (also collinearity) is a phenomenon in which one predictor variable in a multiple regression model can be linearly predicted from the others with a substancial degree of accuracy.

2) Why is it a problem?

Multicollinearity occurs when independent variables in a regression model are correlated. This correlation is a problem because independent variables should be independent. If the degree of correlation between variables is high enough, it can cause problems when you fit the model and interpret the results.

3)How to detect multicollinearity?

4)Example of multicollinearity?

For the two questions refer the following notes.

The notes give the complete elaboration of the given questions.

4. Employ generalized inverse If rank (X"X)<k, th en the generalized inverse can be used to find the inverse of X'X . Then B can be estimated by In such case, the estimates methods of fi is also not known that which method of findin will not be unique except in the case of use of Moore-Penrose inverse of (X"X). Different nding generalized inverse may give different results. So applied workers will get different results. Moreover. it g generalized inverse is optimum. 5. Use of principal component regression The principal component regression is based on the technique of principal component analysis. The are transformed into a new set of orthogonal variables called as principal component explanatory variables s. Usually this techniq ue is ue e dimensionality of data by retaining some levels of variability of explanatory variables which is expredby he bility in study variable. The principal components involves the determination of a set of linear comibiretions explanatory variables such that they retain the total variability of the system and these linear combinations are mutualiy independent of each other. Such obtained principal components are ranked in the order of their importance. The imporianca being judged in terms of variability explained by a principal component relative to the total variability in the system. The procedure then involves eliminating some of the principal components which contribute in explaining relatively ess variation.. After elimination of the least important principal components, the set up of multiple regression is used by reag the explanatory variables with principal components.

reced ing ia s such that they are orthogonal to their p e continue with such process and obtain k such linear combination combinations and satisfy the normality Z,,Z,.., Zt and for them, Var(Z,)> Var(z)> principal component. The linear combination h and so on. These principal components have the property that correlated but Z,,Z,.., Z4 are orthogo .>Var(Z,). The linear combination having the variance is the first aving the second largest variance is the second largest principal component ition. Then we obtain their variances. Suppose such linear combinations are ŽVar(z,)-Var (X,). Also, the X, X.,.. are onal or uncorrelated. So there will be zero multicollinearity among Z,,Z....Zk . i-1 ,X, are not independent. Since the principal components based combat use , X2, on X, , X2, ,X, are mutually independent, so they can be used as explanatory the multicollinearity. variables and such regression will Let λ ,A,, , A be the eigenvalues of X'X,As diag(λι,A,, ,h) is kxk diagonal matrix, Visa kxk orthogonal matrix whose columns are the eigenvectors associated with λ|,A,, ,A, Consider the canonical form of the linear model where Columns of Z (Z , , Z. ) define a new set of explanatory variables which are called as principal component.