Question

Additional Problem A researcher collected data on Y and four X-variables: X1, X2, X3, X4, and he wants to obtain a regression model. However, he is not sure if all the four X-variables should be included in the model. He provides you with the information shown below, namely, the SSR obtained when Y was regressed on each subset of X-variables. Also given: SST-100, and that the sample size is n 12. Your task Apply the Forward-Stepwise selection method, with a-to-enter- o.o5, pwise selection method, with a-to-enter 0.05 to determine which X-variables should be in the regression model State which variable entered first, which entered second, etc., then state the set of variables that were selected. Subset of variables SSR Subset of variables SSR X1, X2, X3 X1, X2, X4 X1, X3, X4 X2, X3, X4 X1, X2, X3, X4 80 83 76 82 84 52 48 54 36 75 69 76 74 78 72 X1 X2 X3 X4 X1, X2 X1, X3 X1, X4 X2, X3 X2, X4 X3, x4

Answer 1

We select the subset of variables in each step such that the residual sum of squares (SSR) is minimised. We select those subsets with minimum value of SSR.

In first step of forward stepwise selection method, we allow X4 to enter

Then X4 is fixed, and X3 is allowed to enter (Since 72 is minimum SSR amongst the pairs in which X4 is present)

In the third step, similarly X4 and X3 is fixed and X1 is allowed to enter the regression model

In the last step, we have a regression model with all 4 independent variables!

We choose the model which maximises Cp or R sq or Adjusted R sq

R ^ 2 = SSM / SST = (SST - SSR) / SST

Here SST = 100

Steps	SSR	R square
X4	36	0.62
X4 , X3	72	0.28
X4, X3, X1	76	0.24
X4, X1, X3, X2	84	0.16

Therefore we select the model in step 1 according to R square criteria

According to adjusted R square criteria, we select the same model in step 1 with one independent variable X4

adj R^2 = 1 - { (1 - R^2) (N - 1) / (N - P - 1) }

Steps	adj R sq	R square
X4	0.4028	0.62
X4 , X3	- 0.131	0.28
X4, X3, X1	-0.194	0.24
X4, X1, X3, X2	-0.32	0.16